Traditionally, periods of intense political activity, conflict and/or social change are accompanied by increased interest in the results of public opinion polls. The events of February 2022 generated a demand for information on the attitudes of Russians towards the war against Ukraine. The publication of such data has been accompanied not only by discussions about its interpretation, but also by publicly expressed doubts about the validity, ethics and expediency of such polls.
Public opinion polls produce two different types of public information or knowledge: on the one hand, the results of the polls themselves, and on the other, their interpretations. There are certain requirements to be followed for the publication of research results. It is necessary to specify the exact wording of the question, the type and size of the sample, the general population of the research, etc. If all the professional requirements for conducting research are met and the necessary accompanying information is disclosed, the published data become a social fact, and the responsibility for recording it lies with the research organisation. The quality of the data — and therefore the validity of the results — is controlled by the commissioner of the survey (if there is one) and the professional community.
A wide range of people, including commentators who are not even associated with the research industry, publicly interpret the results of public opinion research. In fact, anyone can do this and there are no barriers or restrictions. Interpretations are often contradictory and even polarised, and their authors are not obliged to claim objectivity and professionalism.
What follows is not a discussion of the interpretation of collected data. We propose to examine issues related only to the very execution of surveys, the part of the process itself where, in our view, consensus among the various participants in the polling industry is not only possible, but expected.
To illustrate the importance of such a discussion, let us begin with a case to which the epithet «historic» can be applied today: the 2012 presidential election.
«Open Opinion»
In December 2011, the well-known Russian sociologist Igor Zadorin wrote in his personal blog on the LiveJournal platform: «Russian applied sociology has been around for some quite time already, and in these years we have understood a lot about it, and are no longer surprised by many things. In particular, we are no longer surprised that when political struggles become particularly acute (for example, during elections), when political subjects are forced to seek an additional resource in the form of public support, our professional field is bombarded with a lot of allegations, accusations and outright lies. This is understandable, because the assessment of the public resource that politicians and parties measure among themselves becomes an arena of struggle. The electoral rating is like share price, and political subjects are not indifferent to its political capitalisation. Naturally, those ratings that the political players do not like are publicly questioned and discredited, and those who measure public support (you and me) are called hacks, charlatans and even corrupt liars. And it is not only politicians who do not trust sociologists. Often enough the very public whose opinions we measure cannot believe our data about them. This is humanly understandable, many people by their very nature cannot accept that there are other points of view, other opinions on a question to which the answer seems obvious to them.»
These words seem very timely and relevant in today’s context. However, Zadorin was not just describing the current situation. He made a proposal to all willing representatives of the professional community to conduct an independent and open poll of public opinion during the election period. This proposal was put into practice, the results were published in February 2012, and the group of sociologists gathered around this task was named «Open Opinion.»
To better understand the social impact of the research in question, a few words should be said about the context in which it was conducted. Twelve years ago, three organisations were involved in the study of public opinion during the election period: VTsIOM, the Public Opinion Foundation (FOM) and the Levada Centre. The election ratings published by each of them differed considerably. For example, according to VTsIOM data for 11−12 February 2012, 55% of respondents were ready to vote for Putin, according to FOM data for the same dates their share was 50%, and according to Levada-Centre data for 20−23 January 2012 — 43%. The difference of 12% is too large to be explained by measurement error alone. There was a widespread interpretation in the media that the FOM and VTsIOM data show that Putin will be elected in the first round, while the Levada Centre data show that there will be a second round.
However, the results published by «Open Opinion» (according to which 48% of Russians were ready to vote for Putin) did not tell us anything new about the state of public opinion in the country. Their value lay elsewhere. First of all, the pollsters revealed their methodological tools, which made it possible to detect discrepancies in the indicators. VTsIOM and FOM asked closed-ended questions with a limited list of candidates as they were about to appear on the ballot paper. The Levada Centre had an open-ended question, allowing virtually all possible answers, resulting in a higher number of those who found it difficult to give any answer at all. Both approaches are methodologically valid.
Moreover, the invalidity of the above mentioned interpretation of the Levada Centre data has been publicly demonstrated. In elections, votes are counted not from all Russians eligible to vote, but from those who actually voted. This research organisation published the data of all those who took part in the survey. Recalculating the number of those willing to vote for Putin from those who intended to vote, their share was more than 60%.
The history of «Open Opinion» allows us to draw several conclusions that are important for our subsequent considerations.
- Russia’s leading «polling factories» engaged in public opinion research operate in good faith. This conclusion was confirmed in 2013, when the «Open Opinion» organised a professional audit of public opinion surveys of Muscovites during the Moscow mayoral election. All research organisations (in addition to the three mentioned above, the list also included Synovate Comcon) agreed to participate in it. The results are available on the «Open Opinion» website.
- Various actors active in the public space, who for certain reasons are not satisfied with the results of public opinion research, can promote controversial and even deliberately false interpretations that can be successful in the public space. In 2012, Levada Centre results were used to underestimate Putin’s actual electoral potential, as well as to create a «false intrigue» (the possibility of a second round in the voting) that fuelled interest in the election campaign.
- Public opinion researchers are ready for an open professional conversation about methodology, quality, and measurement results, and are willing to reach consensus on the basic interpretation of the results. At the same time, there is no permanent platform for forming this consensus. «Open Opinion» used to play this role successfully throughout the 2010s.
A new reality
Let us go back to the beginning of 2024, a time when the second year of full-scale military operations in Ukraine is coming to an end and the next presidential elections are approaching. What has changed in the field of public opinion research in the country compared to twelve years ago?
All three of the research organisations mentioned above that were active in 2012 are continuing their public opinion monitoring projects. Moreover, since the beginning of the invasion of Ukraine, the number of measurements has increased. At least two new participants in the process deserve to be introduced, as they have managed to prove their professional competence to the expert community: «Russian Field», headed by Artemiy Vvedensky, and ExtremeScan with Elena Koneva and the helm (previously she was the head of Synovate Comcon).
At the same time, government pressure on pollsters has increased significantly over the past twelve years. On the one hand, up to a certain moment the mass opinion research industry in Russia has enjoyed relative freedom. On the other hand, it is the segment of socio-political research that the state, as a key player in the political field, has always sought to control. The number of organisations capable of carrying out nationally representative surveys is estimated at dozens, if not hundreds. However, they do not enter this segment because of the high political risks.
In formal terms, the pressure is primarily related to the «foreign agent» status. The government registered Levada Centre as a «foreign agent» back in 2016. In addition to managerial and financial difficulties, the status of «foreign agent» for a research organisation also means restrictions on the dissemination of its data: government-funded or government-affiliated organisations will not use or publish data obtained by «foreign agents. In January 2024, the State Duma passed a bill in first reading that would affect the activities of research organisations from so-called «unfriendly jurisdictions.» The need for amendments was justified by the fact that, in the current circumstances, the information collected by these organisations and sent abroad could be of strategic importance. The processing of data obtained as a result of research, its systematisation, analysis and storage must be carried out in technical facilities located on the territory of Russia. The organisations covered by this law mainly specialise in marketing research and do not publish public opinion surveys data. However, the trend is clear. The aim of this policy is to divide the professional community, to separate «their own» from «outsiders», i.e. the research groups under governmental control from the independent groups not controlled by th Kremlin.
Since February 2022, it is hard not to notice a general trend related to the shrinking of the public realm. These include the adoption of the law that establishes administrative and criminal punishments for «discrediting» or dissemination of «unreliable information» about the Russian Armed Forces, as well as the closure of a significant number of independent media outlets, and the classification of information, including budget expenditures and Rosstat statistical indicators. In this context, survey results are also considered strategic information.
It should be noted that the «Open Opinion» group, which implemented a dozen and a half projects in the 2010s, including public opinion polls on various topics and professional expertise, has not announced any new projects since the beginning of the war. At the same time, neither Zadorin nor other members of the community have made any official statements indicating that the group has ceased its activities. We assume that the lack of activity is due to the fact that conducting such a civic initiative is simply no longer safe in the context of this «new reality.»
«Insiders» and «outsiders»
The Russian state is the major commissioner of opinion polls. It has all the data it needs, which allows it to ignore the polls conducted by the outsiders. At the same time, only the flattering complimentary (to the authorities) part of the information approved for publication and confirming the public’s support for the state’s policy is disseminated to the public.
The media not linked to the Russian authorities also seem to have adopted the game of «insiders» and «outsiders,» ignoring the social facts recorded by state researchers as if they were fundamentally different from the results of polls conducted without state participation. Some are even actively engaged in a campaign meant to discredit public opinion research as such, regardless of who conducts it.
The arguments that have been put forward do not stand up to criticism. Asked what has changed for pollsters since February 2022, Lev Gudkov, research director of the Levada Centre, replied: «The methodology is the same, peoples’ reaction to sociologists conducting surveys has not changed. The only thing is that at first we did not use the word „war“ in the questions, but only „special military operation“, to avoid setting people up. Otherwise it’s all the same.»
Polling professionals face two constraints. The first is the principle of voluntary participation: the respondent must give informed consent to take part in the research. The second is that the research must be safe for its participants, both the respondents and the researchers themselves.
The research data does not confirm that the war is a factor in Russians’ reluctance to participate in surveys. At the same time, the war is a sensitive topic for respondents, but pollsters know how to handle such issues. It is also not experimentally confirmed that there is a special category of people out there who never ever participate in surveys. Some sections of the population are known to be harder to recruit than others. For example, men are more likely than women to refuse to take part in surveys. Pollsters are aware of this and take it into account when conducting surveys.
The discussion about the validity of current sociological surveys in Russia brings to mind two texts, both authored by well-known pollsters.
The first is a speech given by George Gallup, one of the founding fathers of mass opinion polling, to the American Philosophical Society in April 1942, entitled «How Important is Public Opinion in Time of War?». For Gallup, the question is not whether it is possible to study public opinion in time of war. He believes this to be an important and necessary task. The importance of studying public opinion lies in the fact that it must be taken into account in decision-making, a right reserved for competent professionals.
The second text was published in 1992 by Boris Grushin, the patriarch of Russian polling industry, and is entitled «Why Most Polls Conducted in the Former USSR Should Not Be Trusted.» Grushin points out that public opinion is «an organic product of social life, a kind of collective judgment formed in the process and as a result of highly complex social communication.» In our realities there are often no conditions for the formation of this «product» i.e. of public opinion, so that, at best, sociologists are not dealing with opinion, but with «the cry or groan of the crowd», and, at worst, with «artifacts fabricated in the course of research, i.e. a more or less falsification of the real state of public consciousness.» It should be emphasized that Grushin does not claim that polls were impossible in the undemocratic Soviet Union; this assumption is refuted by his own professional activity. Rather, he argues that polls must be properly organized and interpreted.
Much has been said about how the Russian authorities manipulate the results of public opinion polls. The issue of the representation of the results of surveys in non-state media is much less frequently discussed. In what follows we shall critically interrogate two recent publications that illustrate our point.
The first publication appeared on the SVTV.org website and is entitled «The Majority of Russians Refuse to Participate in Social Surveys About the War: «Russian Field». The article is based on the analysis of the reachability of respondents and contains a link to the primary source: the independent «Russian Field» project. A comparison of the primary source and the article highlights one important discrepancy. On their website the «Russian Field» pollsters do indeed provide data indicating that the number of interview refusals exceeds the number of consents. However, unlike the journalists, the «Russian Field» researchers do not explain the low level of agreement to interviews with the topic of the survey or the situation in the country. The low level of consent to interviews in telephone surveys is a global phenomenon. At the same time, the research community recognizes that the quality of the data collected remains high. SVTV is misusing the analysis produced by the «Russian Field» in order to groundlessly and unfairly undermine the credibility of the research organization’s data, and at the same time of other public opinion researchers in Russia.
The second publication is the article authored by Yekaterina Pachikova and Nadezhda Kolobaeva «The Silent Nation: Why Wartime Opinion Polls Cannot be Trusted.«The main idea of the article is developed in the lead: «Since the beginning of the war, opinion polls show a stunning 80% support for Putin’s actions in Ukraine. However, experts agree that surveys do not reflect the real opinion of the people. In reality, most Russians are reluctant to speak to sociologists, and this makes surveys entirely unrepresentative of popular opinion. Sociologists also argue that in a political regime such as Russia’s, public opinion has no meaning, since a significant portion of the population makes no effort to form an attitude to political issues they believe they have no influence over. Moreover, these surveys can be detrimental, as they help the Kremlin to persuade the populace that the government enjoys widespread support.»
Needless to say, the opinion of the professional sociological community is the exact opposite of that quoted in the text. Professionally conducted surveys are an important and informative source of information on the state of public opinion in the country, and their representativeness is beyond doubt. Interestingly, the article quotes experts Elena Koneva of ExtremeScan and Daria Pavlova of the «Russian Field», who discuss the details of how public opinion research is conducted today, but neither of their quotes supports the main conclusions drawn by the journalists. The most quoted expert in the article, however, is Grigory Yudin, who is neither a representative of the survey industry nor an auditor of research projects. Yudin’s arguments are a collection of critical points from social theory and hypotheses, some of which do deserve attention and further discussion. However, this does not make Yudin a mouthpiece of the conventional opinion of the research community.
Public opinion researchers in Russia certainly face certain difficulties. One of them is that a meaningful discussion around their findings, free from ideological pressure of any kind, is almost impossible to date in the open public space. This does not mean that there are absolutely no publications with balanced, conscientious analysis, but for a non-specialist they remain invisible in the stream of ideologized interpretations. However, this does not make the work of Russian pollsters either pointless or futile.
