In this contribution, I explain why I think it is important for a scientist to strive towards an optimal degree of generalization. There is enormous variation in human behavior, yet there is quite a bit of similarity too across individual people. Finding the most useful focus in this dynamic relation between difference and similarity is one of the main challenges for any researcher of human behavior. 

Explaining complexity

One thing I wouldn’t be particularly keen on doing is to go on national television and comment on some issue as an academic researcher. The reason is not just that I wouldn’t want to have the camera on me, but more so that the realities of media debates and academic research are a bad match. Where one seeks clear black-and-white statements, the other invariably accumulates enough evidence to conclude that things are not black and white. The message that things are complex, and that there is a lot we don’t know yet, and that we therefore shouldn’t rush to conclusions, doesn’t come across well in the fast-paced world of modern media. They ask academics to lend some objective credence to strong claims, but the actual academic message is not very welcome. It’s not clear-cut, and it takes way too long to say.

Perhaps I’m painting too much of a caricature of the media here, and there sure are plenty of outlets that provide ample room for nuance and actually engage with the different sides of an issue that academics are so well-placed to present (think of the better documentaries or old-fashioned magazines like the New Yorker or the Groene Amsterdammer), but what I want to focus on here are the reasons why there is so much indeterminacy, why that is a good thing, and with what methods we academics explore this complex reality. My take-home message will be that the only thing that matters is that your methods match your goals. One and the same issue can be studied legitimately with different methods; in fact that pluralism should be the preferred way of doing it because it allows for better generalization. On the other hand, it should be clear that total generalization is an illusion: there are no definitive answers.

Saying that things are complex can easily be construed as a cop-out. Maybe you just don’t want to commit to a strong position. But most things really are complex, and therefore that is the message that should be made forcefully, backed up by scientific findings. Things are complex, and if we manage to understand that they are, and why they are, maybe as a society we are better equipped to resist quick fixes and simplifying messages. The challenge is how to convey this without killing all discussion. The challenge is to appreciate this complexity.

Asking for knowledge? 

The first methodological lesson we learn as students is that the research methods we use should be appropriate with regard to the question we are trying to answer. If you are interested in people’s opinions, you need to ask them, and you can do this through questionnaires, interviews or observations. Each method has its advantages and disadvantages, and combining methods is often a good idea. If, like me, you’re not so much interested in what people think but in what they do, in human behavior that is, questionnaires and interviews are perhaps not very informative, since you can never know whether what people say about what they do actually corresponds to what they do. They don’t even have to be lying: much of our behavior proceeds in relatively automatic fashion, more or less beyond our conscious control. 

If you would ask me which parts of the newspaper I read during my breakfast this morning I would not be able to give you an exhaustive list of the articles and reports I looked at. I could tell you I read a review of a concert by the Drive-By Truckers, because I particularly like that band and I was therefore happy to see there was a review. This article was salient for me, it stood out, and therefore I remember it well. But I read many other things in the paper, and much of it is a blur now. Something on Trump, something on the battle for Mosul, something on the Dutch elections, more interviews with so-called zwevende kiezers (‘floating voters’). I was aware of it all when I was reading it, of course, but an interview with me afterwards, even just a couple hours, would not give you an accurate picture. To find out about human behavior, you need to observe human behavior.

Such is the messy business of everyday science: every little project contributes building blocks to the larger joint project of understanding the world around us

One classical way of doing that is to take notes as you’re observing, the classical anthropological method. This is a very pure way of doing research, of course, especially if you make as sure as possible that you’re just a fly on the wall (unless you want to know how your presence as a researcher affects people’s behavior), but it also gives you little control over what behavioral data you’re going to get. That’s okay if you don’t have a preference for one particular aspect of behavior, but for many more specific research questions you will want to elicit particular kinds of behavior, while still keeping the research setting in which you collect your data as natural as possible. Most of my own questions are of the latter kind.

How do people use language?

As a linguist, I’m primarily interested in how people use language. We all use language all through the day every day, so in principle there is an unlimited amount of data I could analyze. Obviously, my research questions need to be a little more specific than just ‘how do people use language?’ For me, one of the most intriguing aspects of language is that it changes constantly. That interest doesn’t come out of the blue. It has to do with the theories in linguistics that were dominant when I was a PhD student and which had a pretty unsatisfactory account of change. Similarly, those theories didn’t place great importance on the relationship between language use and how our minds store language. Third, I was working with Turkish immigrants and noticed that their Turkish was influenced quite a bit by their Dutch, and slowly but surely it dawned on me that my own Limburgian was similarly influenced by Dutch. Out of all this serendipity came my research focus on how languages change under the influence of another language. By now, my research questions are a lot more sophisticated than that, but as a general indication of what I work on it will suffice for now.

I dwelled on the research questions a bit because one cannot talk about methods without talking about research questions. But let’s say I want to know how Turkish as spoken by the immigrant population in Holland changes under the influence of Dutch (and let’s take for granted for now that it is an interesting question, both for linguistics, as the changeability of language holds inherent interest, and for social and cultural studies because speaking your language in violation of expected norms comes with all kinds of social costs and cultural implications. How do I go about it? At the end of my study, I would like to conclude something that is representative: if I just followed one particular person from the immigrant community around for a few days noting down everything he or she says I might get a representative sample of how that person speaks at this point of time, but I have no way of knowing whether that person is typical of the community. Ideally, you want a lot of observations of language use of that person, in all kinds of situations (informal face-to-face conversation with friends and family, informal writing on social media, relatively formal interactions at school or work, etc.) over an extended period of time (changes can take years, and rarely have an end-point), and you want this for as many members of the community as possible. The latter is important because you want to know how much variation there is, and the upshot is often that there is so much variation that you might not be able to speak of a community at all: there are just many individuals that cluster together in various ways because their behavior is relatively similar.

Settling for a lower degree of generalization

Obviously, this way my research quickly becomes unmanageable. There is no way you can collect this amount of data, make transcriptions of all the conversations you record, and analyze the enormous corpus that results. There are two ways to remedy the problem somewhat, and I have explored both in my work. One is to design experiments, but I will reserve that for another discussion. The other is to settle for a lower degree of generalization. This is the pragmatic option: you collect the data you need, and you collect as much of it as you can handle. In the course of a single research project, say a four-year PhD project, you can handle such data from maybe 30-50 people. If your focus is on the Turkish community in Holland, those 50 people are not exactly a reliable sample of the entire community of several hundred thousands, but it’s better than two or three. Those fifty people produce speech and social media writing perhaps for up to fifteen hours a day, but you’ll be lucky if you can handle a few hours of speech by each in total. Again, the sample of language use you will get from them is hardly representative for their linguistic life, but it’s better than nothing. The result, if you do your analysis well, is a decent snapshot of language use in the community, and if you present it with the suitable reservations about generalizability, we end up with a general idea of language use by Dutch Turks, including information about how Dutch seems to be influencing Turkish.

Those reservations are not always voiced, though. If you ask me, there is a tendency in science to generalize more than we should. One reason is that it is difficult to get your results published if you hedge too much about how it doesn’t really allow you to say much about the general picture. This takes us right back to my introduction: if a journalist or politician asks, I should say that I should be very careful generalizing my findings to the larger population, but that message is not popular.

But is it bad to have reservations about how much you can generalize? I don’t think so. Such is the messy business of everyday science: every little project contributes building blocks to the larger joint project of understanding the world around us, human behavior in our case, and in my specific case human linguistic or communicative behavior in conditions of multilingualism. The results of my work tell us something about how different and similar we are, how knowing two or more languages influences how you speak, how barriers between languages can get broken down, how dynamic language is, how it interacts with other aspects of communication, how language change affects identity and the larger social issues identity is relevant to, etc. They don’t give any definitive answers, but very few scientific studies do. Instead, they falsify some earlier ideas, they add new insights and new knowledge, they show that things are rarely simple and unproblematic, and they raise new questions and hypotheses. This is a never-ending process. Generalization, in the sense of knowing all there is to know, is an illusion, but it is an illusion worth chasing because the knowledge we gain along the way holds plenty of excitement already.