Presenter: Boris Sokolov (Laboratory for Comparative Social Research, Higher School of Economics)
Time: 27 January (Thursday), 12:00-13:30 CET
Abstract: In many social scientific disciplines, scholars often aggregate individual-level survey-based attitudinal measures to approximate nation-level constructs, such as culture (broadly defined). These aggregated scores are then used as inputs in nation-level or multilevel regression analyses. This practice often leads to controversies, since it has been observed multiple times that at the individual-level particular measures may correlate very weakly and at the same time result in highly robust aggregate-level dimensions. Some scholars find this situation problematic since if individual-based measures do not exhibit meaningful patterns at the individual level then they may simply be incomparable cross-nationally. Thus, aggregating them may result in spurious and therefore misleading group-level associations. Other scholars assert that culture is essentially aggregate-level phenomenon and therefore individual-level patterns, or their absence, does not matter at all for the validity of group-level measures. Instead, the latter may be claimed as valid if they exhibit strong and theoretically consistent correlations with other group-level variables (e.g., objective indicators of economic and societal development or reliably measured historical/geographical variables). A potential problem with the latter argument is that aggregation is often done via simple within-country averaging over multiple observed indicators of a construct and then over individuals. This approach does not account for measurement error, either at the individual or group level. How critical is ignoring measurement error when measuring group-level constructs using individual-level data? Using a simple formal model based on the classical test theory and multilevel regression theory and statistical simulations, I study under which conditions the aggregate of individual observed scores recovers the true group attitude sufficiently well. I also extend this analysis to the setting where one is interested in estimating the correlation/regression relationship between two aggregate-level variables, one of which is measured using individual survey data. My results suggest that when various types of measurement errors are randomly distributed across individuals and groups, the reliability of the aggregate measure is high, and the estimate of its correlation with another group level variable is close to the true value. I also identify conditions under which this is not the case, and discuss some tools that can be used by researchers in order to assess if these conditions are present in their data sets.
Registration: Sign up here if you are not on the mailing list already.