Multilevel Units for Organizational Research - Beware of Misspecification Errors

Some common errors in organizational research include misspecification errors:

• blind aggregation of individual-level measures to represent unit-level constructs,
• use of unit-level measures to infer lower-level relations (the well-known problems of aggregation bias and ecological fallacies),
• and use of informants who lack unique knowledge or experience to assess unit-level construct (Kozlowski & Klein, 2000).

In the past, organizational studies have primarily concentrated on single-level analysis. However, with the advancements in statistical software and techniques, conducting just a simple single-level analysis is becoming harder to justify. Single-level research studies are being replaced today with the more complex multilevel analysis techniques. In hierarchical systems, nested systems, such as in an organization, when a change is made in one part of the system each adjoining system is also effected, changing the whole system - the organization. By concentrating only on a single-level study, the researcher is ignoring the surrounding environment, the effect that the individual has on the group and organization, and alternatively, the effect that changes in the organization has on the team and on the individual.

Klein & Kozlowski (2000) highlighted the benefits of addressing organizational research using multilevel analysis as being able to better understand the complexity of the phenomenon that takes place across levels in organizations. 

"Organizations are hierarchically nested systems. To neglect these systems' structure in our conceptualization and research designs is to develop incomplete and misspecified models" (p. 232).

Misspecification occurs when measures taken at one level, say measures taken at the individual level, are used to make generalizations or inferences at a separate level, say at the team level. To begin with a properly specified model one needs to begin with the level of analysis that the researcher is interested in: "the outcome variable is measured at the lowest level of interest to the researcher" (Hofmann, Griffin, & Gavin, 2000, p. 489). The dependent variable(s) should be measured at the level the researcher is interested. Hence, if the researcher is interested in how team constructs effect individual team members then the dependent variable needs to be an individual measure. Resulting in a two-level study with the dependent variable at the individual level, measures representing the individual team members as level-1 measures, and team constructs represented as level-2 measures. Hypotheses can test any proposed interaction that may take place between levels. Klein and Kozlowski (2000) identified:

"Hypotheses in multilevel research are level-specific. Thus, hypotheses describe not simply the direction - positive or negative - of the relationship between constructs but also the level or levels of each predicted relationship: single, cross-level direct, cross-level moderating, or multilevel homologous" (p. 233).

Unit level constructs need to be clearly defined in the preliminary stages of specifying any model. Kozlowski and Klein (2000) identified unit-level constructs consisting of three basic types: global, shared, and configural unit properties. Global unit properties are those constructs that are measured at the unit level and do not originate at any lower level. Group size and group type are two examples identified as global units according to Kozlowski and Klein (2000). Shared unit properties are measures that originate at one level and can have a similar (isomorphic) meaning at the next level. Examples of shared unit properties include team performance (Kozlowski & Klein, 2000), team cohesion, team norms, team climate, and team mental models (Klein & Kozlowski, 2000). Individual performance, for example, can be aggregated to represent team performance, an isomorphic construct. Configural unit properties also originate at the lower lever, as in shared unit properties, but the upper level is dissimilar (non-isomorphic, or homology) to the lower level construct. Examples include diversity (Kozlowski & Klein, 2000), team personality composition, team interpersonal network density (Klein & Kozlowski, 2000) and team culture. Each of these constructs can take on different properties at the individual level when compared to the team level or organizational level. Configural unit properties cannot be aggregated, or summed, since they take on different meanings at different levels.

Each measure representing the constructs in the model needs to be identified with their unit properties correctly specified. Before aggregating a measure from the individual level to the team level, for example, a shared unit must be specified whereas a configural unit can not be aggregated, this would lead to model misspecification. Prior to aggregating shared units correct statistical procedures need to be followed. Klein and Kozlowski (2000) provide methods and guidelines for aggregating measures from one level to the next level. These guidelines include r_wg, r_wg(j), ICC(1), ICC(2), and WABA reliability measures. While no single reliability measure covers all possible scenarios, it is recommended that more than one reliability measure should be calculated. I typically prefer to calculate either r_wg or r_wg(j) followed by ICC(1) and ICC(2) calculations. More details on each of the reliability measures will be provided in future blog posts.

References:

Hofmann, D. A., Griffin, M. A., & Gavin, M. B. (2000). The application of hierarchical linear modeling to organizational research. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and mthods in organizations: Foundations, extensions, and new directions (pp. 467-511). San Francisco: Jossey-Bass.

Klein, K. J., & Kozlowski, S. W. J. (2000). From Micro to Meso: Critical Steps in Conceptualizing and Conducting Multilevel Research. Organizational Research Methods, 3(3), 211-236. doi:10.1177/109442810033001

Kozlowski, S. W. J., & Klein, K. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and mthods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass. 

Team Cognition Conflict

I will be attending the 2014 AHRD International Conference in the Americas. This AHRD conference will take place in Houston, TX, from 2/19/2014 to 2/22/2014. 

I will be introducing a new construct to the literature on team conflict. The current literature identifies team conflict as being multidimensional, consisting of task, relationship, and process conflict (Behfar, Mannix, Peterson, & Trochim, 2011; Greer, Jehn, & Mannix, 2008; Jehn & Chatman, 2000; & Song, Dyer, & Thieme, 2006). Task conflict looks primarily at work related issues, relationship conflict looks at personal or social issues not relating to work, and process conflict relates to procedural issues.  

In one of my areas of interest/study, team cognition, there have been many advances in the literature identifying the different cognitive processes that take place in teams and small groups. From these advances I fell that the addition of a new construct, team cognition conflict, should be incorporated into the team conflict literature. The cognition conflict construct is a separate construct from those that have been previously identified in the literature, placing the team conflict constructs as having four main sub-dimensions: task, relationship, process, and cognition conflict. By further differentiating team conflict into better defined dimensions researchers will be able to clearly identify team conflict, providing better predictive measures for team performance and decision making abilities. This addition to the team conflict literature also responds to Song, Dyer, and Thieme's (2006) call for further research identifying different types of team conflict. 

The model presented below introduces the outline of the team conflict theoretical framework that will be presented in the AHRD conference in a roundtable format. 

(Turner, J. R., 2013, Figure 1)

References:

Behfar, K. J., Mannix, E. A., Peterson, R. S., & Trochim, W. M. (2011). Conflict in small groups: The meaning and consequences of process conflict. Small Group Research, 42, 127-176. doi:10.1177/1046496410389194

Greer, L. L., Jehn, K. A., & Mannix, E. A. (2008). Conflict transformation: A longitudinal investigation of the relationships between different types of intragroup conflict and the moderating role of conflict resolution. Small Group Research, 39, 278-302. doi:10.1177/1046496408317793

Jehn, K. A., & Chatman, J. A. (2000). The influence of proportional and perceptual conflict composition on team performance. The International Journal of Conflict Management, 11, 56-73. doi:10.1108/eb022835

Song, M., Dyer, B, & Thieme, J. R. (2006). Conflict management and innovation performance: an integrative contingency perspective. Journal of the Academy of Marketing Science, 34, 341-356. doi:10.1177/00092070306286705

Turner, J. R. (2014). Team cognition conflict: A conceptual review identifying cognition conflict as a new team conflict construct. Paper to be presented at the 2014 AHRD International Conference in the Americas, April 2014. [forthcoming presentation]

Team Shared Cognition Constructs - New Publication

Final approval for publishing my recent article, titled: "Team Shared Cognitive Constructs: A Meta-Analysis Exploring the Effects of Shared Cognitive Constructs on Team Performance" has just been received.

This has been a long process, from conference proceedings introducing meta-analysis techniques, to enduring the peer review process for, ultimately, final approval to publish. 

This article will be published by the flagship publication of the International Society of Performance Improvement (ISPI),  Performance Improvement Quarterly (PIQ). The reference/bibliographical information is provided below (no volume, issue, or page numbers provided at this time):

Turner, J. R., Chen, Q., & Danks, S. (2014). Team shared cognitive constructs: A meta-analysis exploring the effects of shared cognitive constructs on team performance. Performance Improvement Quarterly. Manuscript submitted for publication.

These new emerging shared cognition constructs are beginning to be identified as being critical to the success of team and small group performance and problem solving efforts. More study is needed in these areas which was identified in the article.

In a previous post I presented the conference proceedings introducing the meta-analysis techniques used.

Information Sharing Cognition Constructs - Meta-Analysis

This post also introduced the presentation slides that were used during the conference:

Team Shared Cognitive Constructs

The original presentation was designed for two purposes: 1) to introduce the emerging constructs of team shared cognition, and 2) to present the steps required to conduct a comparative meta-analysis study. In summary, the team shared cognition constructs that were prepared are provided in the table below, titled 'Shared Cognitive Constructs'.

In conclusion, the results from the meta-analysis are provided in the slide below, titled 'Conclusion'.

Limitations: 

As identified in the manuscript the sample size for this meta-analysis was small.  Having a small sample size prevented the possibility of making any type of inference(s) from the results. However, the main purpose of this study was to 1) identify the different constructs that were currently being studied in various disciplines, and 2) to run a comparison of these constructs to shed some light on which constructs resulted in better performance outcomes. With these shared cognition constructs being emerging constructs, meaning that they are new developing constructs, there is not a lot of research available to begin with. Thus, a secondary purpose of this research study was to call to researchers to contribute further to the research of these emerging constructs - beginning with those that were identified in this meta-analysis as being potentially better predictors of performance: information sharing, cognitive consensus, and shared metal memory.

Why Theories Are Important

Theories are needed: 

"to satisfy a very human 'need' to order the experienced world. The only instrument employed in the ordering process is the human mind and the 'magic' of human perception and thought" (Dubin, 1978, p. 7)

A theory purpose is to either predict or explain the phenomenon being studied (Dubin, 1978; Creswell, 2014). Theories are conceptual models identifying the relationships between concepts, constructs, variables, and events, structured around a predefined set of boundaries (limitations). Jaccard and Jacoby (2010) reflect this in their definition of a theory: "an explanation of relationships among concepts or events within a set of boundary conditions" (p. 112). 

A theory remains a conceptual model up to the point that the researcher tests the theoretical model, at this point the theoretical model becomes a scientific model (Dubin, 1978). It is through testing theoretical models that the model is either accepted or rejected. Theoretical models are accepted when theories have been subjected to empirical testing and have been shown to be useful (Jaccard & Jacoby, 2010). Likewise, theoretical models are not accepted when theories have been subjected to empirical testing and have not been shown to be useful. A theoretical model is deemed as being valid through empirical testing, and is deemed as being useful or not useful (utility) by your peers in academia and by those in practice (consensual evaluation; Jaccard & Jacoby, 2010). To be considered scientific, Jaccard and Jacoby (2010) identified that theoretical models must achieve empirical verification or falsification. This is done through testing the theoretical model. 

Additionally, empirical research requires theoretical or conceptual models to identify the connections and relatedness of the variables being tested. The theoretical model provides the foundation for the hypothesis that are being tested in empirical research. The theoretical model also makes it easier for other scientist to replicate a study.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

Dubin, R. (1978). Theory building (Revised ed.). New York, NY: The Free Press.

Jaccard, J., & Jacoby, J. (2010). Theory construction and model-building skills: A practical guide for social scientists [Kindle]. Retrieved from Amazon.com

New Article Acceptance: Multiagent Systems as a Team Member

I have received notice that my article titled Multiagent Systems as a Team Member will be published by Common Ground Publishing in their journal: The International Journal of Technology, Knowledge, and Society. The web page for the journal follows: http://ijt.cgpublisher.com

No date as to when the article will be published but it should be this fall. Listed below is the abstract for the journal article to give those interested an indication of what the article is about.

Abstract

With the increasing complex business environment that organizations have to operate in today, teams are being utilized to complete complex tasks.  Teams are capable of completing complex tasks that no one individual can achieve.  Effective team decision-making requires team members to discuss new information (unshared knowledge) and to consider this new information along with existing information (shared knowledge).  Research has shown that shared knowledge is favored over unshared knowledge during team discussion, this is known as the unshared knowledge barrier in the literature.  One method of transferring unshared knowledge to shared knowledge is to take advantage of new multiagent systems (MAS) that are designed to support teams.  Multiagent systems are capable of filtering information without any bias toward shared information over unshared information.  This conceptual model incorporates individual intelligent agents and MAS that monitor and actively interact with team members as an effort to address the unshared knowledge barrier, resulting in better team decision-making and problem solving outcomes. 

I presented this article during the conference that CGPublishers held this past January in Vancouver, Canada. The presentation slides from this presentation have already been made available on my blogpost MAS as Team Member - Presentation Slides. All of my presentation slides are also available at my slideshare page: www.slideshare.net/JohnTurner5/

Information Sharing Cognition Constructs - Meta-Analysis

I just completed my presentation at the SLOANet4 International Conference in Las Vegas - Planet Hollywood.

Attached below are the presentation slides from slideshare.  This meta-analysis reviews six shared cognition constructs from various disciplines:Team Mental Model (TMM), Shard Mental Model (SMM), Transactive Memory Systems (TMS), Information Sharing (IS), Cognitive Congruence (CC), and Group Learning (GL).

Information Sharing Cognition Constructs: Slideshare Link

Blogging - Dissertation - Theory to Practice

One main benefit blogging can provide young researchers is that it can act as an outlet to test new theories or associations between differing constructs. Coverdale (2012) highlights on slide #12, in his slideshare presentation on Social media practices: Benefits and risks for doctoral researchers, the cyclical relationship between one's blog and one's thesis. Blogging can provide the reflective tool to organize your thoughts for your dissertation.

People who follow your blog and comment on your thoughts and ideas are identified as being part of your Personal Learning Network (PLN). Those who make up your PLN help to keep you on track and provide the peer pressure to help you finish your dissertation (Pasquini, 2013). As Larive (2013) highlighted in her blog post How Blogging Helped Me Write my Dissertation: "blogging has allowed me to face my ghosts, build up a network of contacts, and advance faster through the process of writing my dissertation" (p. 1). 

From the vast variety of content available on academic associated blogs I tend to try to keep my content on the following few items. The items that I list here are just a few of the content items that Cloverdale (2012) identified, my list is a modified/condensed version of his original list:

• Reports on academic events, including workshops, seminars, and conferences.

• Book and article reviews.

• Research methods and methodologies, and academic writing.

• Using research tools and software.

• Development of theoretical and conceptual ideas.

• Professional development (p. 2).

One item that I would add to this list, and one item that I try to focus on whenever I submit a blog, is to connect 'theory to practice'. I try to identify the So-What Factor, how research and theory relates to, or can be applied to, everyday work and/or life issues.

Blogging can help make new connections as well as provide a medium to challenge your ideas and your research. Cloverdale (2012) posed: "Academics are increasingly under pressure to engage with wider academic (and non-academic) audiences and articulate the relevance of their research in relation to wider societal issues and prescribed 'real-world' problems" (p. 4). Academic blogs can help: 

1) close the gap between academia and the practitioner world, 

2) shed light on societal issues, and

3) provide a means of personal development for the blogger and those associated to the blogger's PLN.

References:

Cloverdale, A. (Dec. 2012): The benefits of social media for doctoral researchers. Paper presented at the conference of the Society for Research into Higher Education (SRHE), Wales, UK.

Cloverdale, A. (Dec. 2012). Social media practices: Benefits and risks for doctoral researchers. #srheconf12 Presentation. Retrieved from http://www.slideshare.net/andycoverdale/social-media-practices-benefits-and-risks-for-doctoral-researchers

Larive, M. (Jan. 29, 2013). How blogging helped me write my dissertation. The Chronicle of Higher Education. Retrieved from http://www.chronicle.com/article/How-Blogging-Helped-Me-Write/136893/

Pasquini, L. (Feb. 19, 2013). Re: How-Blogging-Helped-Me-Write [Web log comment]. 

MAS as Team Member - Presentation Slides

I just recently presented my paper Multiagent Systems as a Team Member at the 9th International Technology, Knowledge and Society Conference on January 14, 2013, in Vancouver, Canada; presented by Common Ground Publishing, USA. 

As mentioned in a previous post, 9th International TKS Conference, this paper is in the process of being peer-reviewed and will hopefully be published by this spring. Earlier I discussed that I would provide my presentation slides once they were finalized, "Once the presentation slides are put together, edited, and finalized I will have them posted on Slideshare. I will post the link to the presentation slides once I have them completed". 

The complete presentation slides can be found at the following Slideshare address: http://www.slideshare.net/JohnTurner5/mas-teams-slidesfinal

Innovation at the Intersections: Many-To-Many Connections

Timothy Chester identified in his blog The Accidental CIO that the nature of collaboration has shifted, it has shifted from one-to-many exchanges to many-to-many exchanges (Chester, MOOCs, 2013). The examples used for one-to-many was the traditional classroom setting in which the teacher presents knowledge to students. In the corporate setting the same could be said of certain hierarchical levels in which the upper levels dictate to the lower levels.

Competing in this complex environment requires new knowledge, innovative ideas, and exchanges that take place within the organization as well as outside of an organization. In their blog on Social Capital, TNT - The Network Thinkers discussed this same idea in the framework of social capital: “Creating competitive context requires social capital - the ability to find, utilize and combine the skills, knowledge and experiences of others, inside and outside of your organization” (Social Capital). Having the ability to utilize and find knowledge requires many-to-many exchanges.

As explained by TNT - The Network Thinkers, “Innovation happens at the intersections” (Social Capital). The intersections refer to the numerous connections made between the many, as opposed to the intersections in a one-to-many connection. You can see that more connections are possible in a many-to-many connection compared to a one-to-many connection. As these connections increase the number of intersections increase, and the potential for new knowledge and innovative ideas grow exponentially.

Expand your current network so that you are taking full advantage of the many-to-many connections rather than utilizing the one-to-many connections. To do this TNT - The Network Thinkers offer some steps to take in their article Community Networks. First, identify your current structure. Know where there are gaps, bridges, linchpins, and identify who is the core of the network and who is in the periphery (Community Networks). Secondly, begin closing the gaps by inviting and including all members within the network to contribute.

References:

Chester, T. (January 18, 2013). Why MOOCs are like Farmville. The Accidental CIO. Retrieved from www.accidentalcio.com/2013/01/why-moocs-are-like-farmville-html

Community Networks (October, 12, 2012). TNT-The Network Thinkers. Retrieved from www.thenetworkthinkers.com/2012/10/community-networks.html

Social Capital: the key to success in the connected age (July, 04, 2012). TNT-The Network Thinkers. Retrieved from www.thenetworkthinkers.com/2012/07/social-capital-key-to-success-in.html

Competencies for Today's Workforce - Critical Thinking

In a recent survey conducted by the American Society for Training & Development (ASTD) a significant skills gap was identified in addition to a lack of the following critical soft skills: communication, creativity, collaboration, and critical thinking.  Respondents also reported that leadership or executive-level skills were the number one skills gap in their organization (Pace, 2012).

Pace (2012) identified the following competencies that employees look for in their young workforce. These competencies are provided as they were reported 20 years ago, today, along with predictions for the future workforce (see table below). 

Top 5 competencies employees look for in youths entering workforce
20 Yrs. AGO	TODAY	FUTURE
Technical Mastery	Self-motivation and Discipline	Adaptability
Self-motivation and Discipline	Effective Communication	Effective Communication
Confidence	Learning Agility	Learning Agility
Effective Communication	Self-Awareness	Multicultural Awareness
Resourcefulness	Adaptability	Self-motivation and Discipline
		Collaboration

There are two discrepancies from the previous information. The first is that even though respondents identified the following skills gap lacking in their workplace (communication, creativity, collaboration, and critical thinking), they also reported that leadership training was the number one skill gap in their organization. Additionally, the top five competencies for the entering workforce do not include those missing skills that were already identified, with the exception of communication and collaboration, although collaboration was only listed in the future category. This leaves out creativity and critical thinking skills.

With nearly $12 billion dollars dedicated to leadership training in the US alone (Peters, Stephens, & Baum, 2012), there is a disproportion of training leadership related skills compared to training the required critical skills needed to complete everyday tasks. Pace (2012) highlighted that collaboration between higher education and businesses was essential to developing these skills and to assist in closing these gaps. However, examples provided were for leadership skills collaboration programs. I agree with Pace, there must be collaboration between higher education and businesses (private & public) so that the workforce employed today will have appropriate competencies and skills. Unfortunately, concentrating on leadership training will not resolve the skills gap.

Concentrating on the competencies identified above would be the best place to begin closing the skills gap between what today’s businesses are requiring and what new employees are offering. Essential in closing this skills gap is to concentrate on providing today’s students with critical thinking skills. If students are not able to think on their feet and solve complex problems how can you expect them to lead these efforts from others (through leadership training?). If businesses concentrated on demanding and hiring employees that were able to demonstrate true critical thinking skills, then, their workforce would be able to be trained to be future leaders. Higher education should be able to train their students to practice critical thinking skills through workshops and groups assignments in classrooms (group projects add the collaboration to the mix, with collaboration being one of the competencies). Teaching critical thinking skills can be practiced and taught through the following three criteria:

• Reflective scepticism.
• Identifying and challenging assumptions.
• Imagining and exploring alternatives (Martin, 1995, p. 5).

Additionally, teaching students to become self-directed learners could be easily accomplished through the following four stages presented by Grow (1991), the student’s stage is listed first followed by the teacher’s role listed second:

• Dependent - Authority Coach
• Interested - Motivator, Guide
• Involved - Facilitator
• Self-Directed - Consultant, Delegator

By addressing the competencies, rather than trying to train leaders, businesses and higher education will be able to address the skills gap more effectively. Resulting, ultimately, in a better qualified workforce for the future.

References:

Grow, G. O. (1991). Teaching learners to be self-directed. Adult Education Quarterly, 41, 125-149. doi: 10.1177/0001848191041003001

Martin, G. W. (1995). An approach to the facilitation and assessment of critical thinking in nurse education. Nurse Education Today, 16, 3-9.

Pace, A. (December, 2012). Preparing Today’s Youths for Tomorrow’s Workplace. Training + Development, 42-46.

Peters, L., Stephens, G. K., & Baum, J. (December, 2012). When developing leaders, don’t blame training! Training + Development, 59-62.