“MRP is a model-based technique, so if you have a really poor model of the opinion you’re examining, that’s going to hurt you” – claims Prof. Hanretty. “Hopefully, everyone using MRP will have at least some substantive knowledge of the demographic and geographic determinants of public opinion” – he adds. In this interview, the author explains complexities, the potential and flaws of multilevel regression and post-stratification. A fuller analysis of this emerging technique can be found in Prof. Hannerty’s PSR article.
Political Studies Review: How would you describe the basic idea behind multilevel regression and post-stratification (MRP) technique?
Chris Hanretty: There are two basic steps in MRP: (1) you learn about voter opinions from a large national sample, and in particular, the opinions of certain types of voters; (2) and you go look up other sources of information (often a census or something similar) to find out how many voters of each type there are in each area. If I know (on the basis of my national sample and some model) that 55-64-year-old men with a high school education are very likely to vote Conservative, and if I know how many such men there are in a particular seat, then that gets me part of the way to understanding how that seat as a whole will vote. I just need to repeat the exercise for all the different voter types implicit in my model.
That’s the idea in a nutshell. In practice, it’s more complicated, and often a lot of the added value comes not from knowing information about individual voter types, but information about the types of the area they live in. The single best predictor of Conservative vote share in a seat is the Conservative vote share in the last election. MRP really benefits from having these predictors alongside demographic predictors, but I lead with the demographic picture because that’s much more intuitive.
You wrote that MRP has been developing for the past 15–20 years. It has made it possible to pose and answer questions related to public opinion in small areas that have not been possible before. How was this method popularised, and what influenced its development? Is it becoming a prevalent statistical technique?
I think Andrew Gelman at Columbia has been an outstanding popularizer of MRP. I think technical and software developments have always played their part. There are now a lot more packages which allow researchers to estimate multilevel models of the kind used in MRP.
The major benefit of MRP seems that it allows avoiding the need for surveys at a sub-regional level. Are there any other benefits?
For me, it’s hard to see past that benefit. If you want to know about constituency opinion in the UK, it’d be impossible to field a standard 1,000 person survey in all those seats. No company has that polling capacity. Maybe for some contexts – say, US states – you could think about conducting state polls and aggregating those. But then you’d have to think about varying dates of fieldwork, different weighting targets in those states – urgh, it makes me shudder to think of it.
What are the possible limitations of this method?
MRP is a model-based technique, so if you have a really poor model of the opinion you’re examining, that’s going to hurt you. Hopefully, everyone using MRP will have at least some substantive knowledge of the demographic and geographic determinants of public opinion.
Maybe for some contexts – say, US states – you could think about conducting state polls and aggregating those. But then you’d have to think about varying dates of fieldwork, different weighting targets in those states – urgh, it makes me shudder to think of it.
Another limitation is that you might not always have the post-stratification data you need. You might want to create estimates just for adult citizens, but your national census office might only release breakdowns for the adult population. There’s often a tension between what you want to include in the model and what’s available from official statistics.
What are other contributions your article brings to the field you’d like to highlight?
I’m just happy to have some code out there which takes people through the whole process. Written descriptions of procedures in peer-reviewed journals are obviously important, but additional documented code is the cherry on the cake!
MORE
Article: Hannerty C. (2020), An Introduction to Multilevel Regression and Post-Stratification for Estimating Constituency Opinion, Political Studies Review 2020, Vol. 18(4) 630–645.
ABOUT
Professor Christopher Hannerty – Professor of Politics at Royal Halloway, University of London.
His research areas concern representation and the politics of the judiciary. More
Questions and production
Dr Eliza Kania, PSR/Brunel University London