PSR Podcast #21: Dr Tsung-han Tsai, When “Don’t Know” Indicates Nonignorable Missingness: Using the Estimation of Political Knowledge as an Example

In survey research, researchers usually design a battery of questions to measure some concepts such as democratic values and political knowledge. Owing to the limitations of the questionnaire length, three to five questions are used to measure a defined concept.  Since there are only limited questions for a concept, responses to these questions matter. However, respondents sometimes provide nonresponses to these questions such as don’t know. One widely used approach to deal with nonresponses is to treat them as missing values. Treating nonresponses as missing values indicates that there is no information extracted from these questions.

In some cases, don’t know can be considered valid responses rather than missing values. For example, in the case of measuring political knowledge, the conventional approach is categorizing don’t know into incorrect responses. However, more and more research suspects whether it is appropriate to treat don’t know as an absence-of-knowledge category. This research pays attention to partial knowledge hidden within don’t know.

In this paper, I propose a model to extract the information from don’t know responses, on the one hand, and to formally test partial knowledge within DK. In specific, I combine item response theory and the shared-parameter approach which is presented in the literature on missing data mechanisms. Unlike the conventional approach, I treat DK as missing values and assume that they are missing not at random. The logic is that whether a response to political knowledge questions is correct or not and whether a don’t know the response is provided is determined by knowledge levels.

I applied the proposed model to analyze survey data from Taiwan’s Election and Democratization Study project. In specific, I study the gender gap in political knowledge. It has been argued that men appear to know more about politics than women. Even though some studies recognize the gender gap in knowledge, others argue that the higher percentage of DK responses from women exaggerates the gap in political knowledge.

That is if there is knowledge hidden within DK responses, treating DKs as incorrect responses would make women appear less knowledgeable than they actually are. According to the results of the analysis in this article, we do find hidden knowledge within nonresponses for women. This phenomenon, however, occurs only in one of the three political knowledge questions. These results suggest that the gender gap in political knowledge is not seriously exaggerated by women’s higher percentages of nonresponses because most of the time these nonresponses indicate the absence of knowledge.