Analytics in Practice: Developing an Engagement Survey, Part 1

In this episode, co-hosts Jennifer Miller and Ron Landis start their conversation about developing a measure of employee engagement. They identify and discuss several steps/phases that should be present when we develop measures of any kind. 

In this episode, we had conversations around:  

• The importance of clearly, accurately, and comprehensively defining what it is that we are interested in measuring (e.g., engagement). 
• As part of the definition process, the importance of thinking about whether our survey is measuring one big factor or is comprised of several subdimensions. 
• The importance of specificity in defining what you want to measure. 
• The importance of considering how engagement scores will ultimately be used. Specifically, are we interested in using overall scores or are dimension scores more helpful for decision making of building models (such as flight risk)? 
• The definition as the foundation for creating a test blueprint. The blueprint helps ensure that we measure exactly what we want to measure (nothing more or nothing less) 
• Aspects of item writing were also discussed. In particular, we talked about making sure that we carefully consider how items will be read and interpreted by people who will respond to them. 

Key Takeaways:  

• When creating an engagement measure (or any measure), a critical step is to first carefully define what it is you are interested in measuring. Incompleteness, ambiguity or inaccuracy at the outset will lead to a poor measure. 
• Carefully consider the dimensionality of what it is you are measuring. The dimensionality will serve as a foundation for creating the measure and for evaluating its validity. 
• Every measure you use should always have a test blueprint that serves as a framework for guiding item writing 
• When you write items, keep in mind that they are the only way you have of communicating with your respondents. Write items that are clear and to the point and presented in a manner that facilitates ease of understanding.