#4. What the ratings won't tell you

Updated: Oct 9

Post-release #4 dated April 2, 2022

Elena Koneva

 
Based on the Chronicle survey, Wave #3 26.03-29.03

Recently, research centers have been publishing exceptionally high political ratings. They include the level of support for the military operation, which is largely the result of support for the government, and the dynamics of the two are in sync.

The phenomenon of these polls lies not only in the high numbers, but also in the rapid dynamics, literally in a matter of weeks. Virtually all liberal commentators talk about the social desirability effect of military self-censorship and the unreliability of these numbers, but immediately move on to discuss the reasons for such high support.

Today's situation should encourage the research audience to accept figures and analysis critically, but without prejudice, without losing the hope of extracting predictive knowledge from them.

For this purpose, it is useful to consider persistent stereotypes in the perception of political sociology.

Stereotype #1. Opinion polls cannot be conducted in a country that is undergoing a military operation/ a war

The need to understand social processes is now extremely high, so we cannot give up on research. We need to look for ways to overcome the technical and psychological difficulties of conducting surveys.

Political (and not only) decisions in Russia are accompanied by public opinion polls. It is impossible to establish a moratorium on the production of numbers, especially for propaganda purposes. In order to maintain a balance, independent data must too be produced and analyzed.

The numbers are a necessary but a superficial picture of reality. What does the figure of 60% support for the military operation today, on the 38th day of it, tell us? About the effectiveness of the propaganda of the military campaign in Ukraine, about the high rating of Putin himself, about the fact that there are more people who support it than those who do not? About the willingness to enlist in a volunteer battalion or to endure economic hardship?

About the fact that the deterioration of the economic situation and of one's personal financial status, the loss of jobs, the impending isolation of the country and the rapid evacuation of hundreds of thousands of more mobile compatriots abroad - all these consequences in the perception of many people exist separately from the military action to 'protect the Russians in the Donbass'.

We are still surveying the reflection of the television picture. According to 29 percent of the respondents, Ukrainians greet the Russian troops in a friendly manner, while 15 percent claim that combat operations are only taking place on the territory of the Donbass.

Stereotype #2. The institutions under government control manipulate the data, while independent companies would provide numbers that are less positive for the government

Below are data from several research centers.

The wording of the questions differs slightly and is fairly comparable.

So, the question of support for the military operation in Ukraine.

High numbers and sharp dynamics are shown by almost all measurements. At the beginning of the military operation, the support numbers were in the range of 58-65%

On March 4, 2022, the president of Russia signed a law criminalizing fakes about the actions of the Russian military in special operations as well as calls for anti-Russian sanctions.

Support, like other political ratings, rises to a level of 73-81% in subsequent measurements.

Here we give detailed data for the three waves of the Chronicles project, including individual data for the three experimental subsamples of the second wave.

The first part of the experiment entailed the different positioning of our symbolic question about support for the military operation at the beginning of the interview versus at the end.

The second experiment consisted in changing the wording of the question about the support of the military operation.

For 50% of the sample, the question was as follows:

"Please answer, do you support or do you not support Russia's military operation in Ukraine?" 

The other half of the respondents were asked the question differently

"Please answer, do you support, do you not support Russia's military operation on the territory of Ukraine, do you find it difficult to answer unequivocally or do you not want to answer this question?" 

Table 2.1. Distribution of responses about the support depending on the wording of the question (%)

As a result, 12% of respondents exercised their articulated right not to answer this question

In the third wave, the tested wording was used with the explicit alternative of 'you can refuse to answer this question," and the question was positioned at the end of the interview. 'supporting' respondents have no reason to hide their position out of fear, it can be assumed that these 7% of respondents were only expressing the 'administratively approved'' opinion, without actually sharing it.

There are no statistically significant differences in the gender and age of the respondents within either subgroup, i.e. the same audience was surveyed in both cases.

In the third wave, the tested wording was used with the explicit alternative of 'you can refuse to answer this question', and the question was positioned at the end of the interview.

We hope that the Chronicles have succeeded in balancing out the social desirability effect of endorsing government policies to a significant degree.

The results of the experiment are illustrative of the impact of technical factors, in addition to criminal penalties for unauthorized interpretation of the military operation

We invite key pollsters to an open discussion of the methodology for measuring public opinion in the current circumstances. A similar round table has shown its effectiveness following the 2013 Moscow mayoral elections, when standard approaches and models had shown the predictions that were far from the election results under the changed circumstances.

Stereotype #3. During a critical period, respondents tell untruths or refuse to participate in interviews

All the available experience of the researchers in Russia over the past month suggests that the refusal rate has increased, but remains at an acceptable level. This topic deserves a separate publication.

In the press there are comments saying '90% of respondents hang up at the beginning of the interview'. Such refusal rate indicates the lack of professionalism of the call center, not the level of consent.

As long as people respond, we should benefit from their goodwill.

Stereotype #4. People are divided into monolithic parties supporting or not supporting the military operation

This concept is widely covered in propaganda texts.

Even if we take the current minimum estimate of 60% support for the military operation, this group is very heterogeneous. We are working on the necessary segmentation, the results of which we will present in the next publication.

Declaring support for the military operation does not prevent 32% of this group from wanting to end the operation as quickly as possible without seeking a military surrender of Ukraine.

33% of them believe that for the economic development of Russia it is necessary to cooperate with the West, a third of them expect a strong negative impact of the sanctions.

37% of approvers expect Ukrainians to greet the Russian army in a friendly manner, compared to 22% who expect them to be hostile.

As an additional illustration, we present the table showing the responses distribution only for the respondents who support the military operation.


Stereotype #5. Russian society is rallying 'round the flag', and the resource for the approval of the military operation has not yet been exhausted

Foreign policy goals (almost in a Maslowian way) have a limited strength

It is very important to look at the 'inner life' of Russians, because it is what remains a basic value for the people.

There are many changes in the personal lives of the Russians, which will evolve and affect the perception of the foreign policy.

The sources of approval have been used. The sources of disapproval are in growing dynamics. In this regard, positive research reports should be taken very carefully, they can be disorienting.

Throughout the military operation, multidirectional factors are at work. The military consolidation around the leader is balanced by the unfavorable changes in the lives of Russians. As a result, there are virtually no dynamics at work.