Some common errors in organizational research include misspecification errors:
- blind aggregation of individual-level measures to represent unit-level constructs,
- use of unit-level measures to infer lower-level relations (the well-known problems of aggregation bias and ecological fallacies),
- and use of informants who lack unique knowledge or experience to assess unit-level construct (Kozlowski & Klein, 2000).
In the past, organizational studies have primarily concentrated on single-level analysis. However, with the advancements in statistical software and techniques, conducting just a simple single-level analysis is becoming harder to justify. Single-level research studies are being replaced today with the more complex multilevel analysis techniques. In hierarchical systems, nested systems, such as in an organization, when a change is made in one part of the system each adjoining system is also effected, changing the whole system - the organization. By concentrating only on a single-level study, the researcher is ignoring the surrounding environment, the effect that the individual has on the group and organization, and alternatively, the effect that changes in the organization has on the team and on the individual.
Klein & Kozlowski (2000) highlighted the benefits of addressing organizational research using multilevel analysis as being able to better understand the complexity of the phenomenon that takes place across levels in organizations.
"Organizations are hierarchically nested systems. To neglect these systems' structure in our conceptualization and research designs is to develop incomplete and misspecified models" (p. 232).
Misspecification occurs when measures taken at one level, say measures taken at the individual level, are used to make generalizations or inferences at a separate level, say at the team level. To begin with a properly specified model one needs to begin with the level of analysis that the researcher is interested in: "the outcome variable is measured at the lowest level of interest to the researcher" (Hofmann, Griffin, & Gavin, 2000, p. 489). The dependent variable(s) should be measured at the level the researcher is interested. Hence, if the researcher is interested in how team constructs effect individual team members then the dependent variable needs to be an individual measure. Resulting in a two-level study with the dependent variable at the individual level, measures representing the individual team members as level-1 measures, and team constructs represented as level-2 measures. Hypotheses can test any proposed interaction that may take place between levels. Klein and Kozlowski (2000) identified:
"Hypotheses in multilevel research are level-specific. Thus, hypotheses describe not simply the direction - positive or negative - of the relationship between constructs but also the level or levels of each predicted relationship: single, cross-level direct, cross-level moderating, or multilevel homologous" (p. 233).
Unit level constructs need to be clearly defined in the preliminary stages of specifying any model. Kozlowski and Klein (2000) identified unit-level constructs consisting of three basic types: global, shared, and configural unit properties. Global unit properties are those constructs that are measured at the unit level and do not originate at any lower level. Group size and group type are two examples identified as global units according to Kozlowski and Klein (2000). Shared unit properties are measures that originate at one level and can have a similar (isomorphic) meaning at the next level. Examples of shared unit properties include team performance (Kozlowski & Klein, 2000), team cohesion, team norms, team climate, and team mental models (Klein & Kozlowski, 2000). Individual performance, for example, can be aggregated to represent team performance, an isomorphic construct. Configural unit properties also originate at the lower lever, as in shared unit properties, but the upper level is dissimilar (non-isomorphic, or homology) to the lower level construct. Examples include diversity (Kozlowski & Klein, 2000), team personality composition, team interpersonal network density (Klein & Kozlowski, 2000) and team culture. Each of these constructs can take on different properties at the individual level when compared to the team level or organizational level. Configural unit properties cannot be aggregated, or summed, since they take on different meanings at different levels.
Each measure representing the constructs in the model needs to be identified with their unit properties correctly specified. Before aggregating a measure from the individual level to the team level, for example, a shared unit must be specified whereas a configural unit can not be aggregated, this would lead to model misspecification. Prior to aggregating shared units correct statistical procedures need to be followed. Klein and Kozlowski (2000) provide methods and guidelines for aggregating measures from one level to the next level. These guidelines include rwg, rwg(j), ICC(1), ICC(2), and WABA reliability measures. While no single reliability measure covers all possible scenarios, it is recommended that more than one reliability measure should be calculated. I typically prefer to calculate either rwg or rwg(j) followed by ICC(1) and ICC(2) calculations. More details on each of the reliability measures will be provided in future blog posts.
References:
Hofmann, D. A., Griffin, M. A., & Gavin, M. B. (2000). The application of hierarchical linear modeling to organizational research. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and mthods in organizations: Foundations, extensions, and new directions (pp. 467-511). San Francisco: Jossey-Bass.
Klein, K. J., & Kozlowski, S. W. J. (2000). From Micro to Meso: Critical Steps in Conceptualizing and Conducting Multilevel Research. Organizational Research Methods, 3(3), 211-236. doi:10.1177/109442810033001
Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and mthods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
