Analytics in Practice: Developing an Engagement Survey, Part 2

In this episode, co-hosts Jennifer Miller and Ron Landis continue their conversation about developing a measure of employee engagement. They identify and discuss several steps/phases that should be present when we develop measures of any kind.  

In this episode, we had conversations around these topics:  

• Aspects of item writing were discussed. In particular, we talked about making sure that we carefully consider how items will be read and interpreted by people who will respond to them. 
• We talked about approaches to writing items (for example, do we do it as a group?, do we ask stakeholders to comment?, do we pilot test?). In particular, we talked about balancing the need to get feedback from different stakeholders while being as efficient as possible. 
• We talked about the negative outcomes that may occur if we don't carefully consider whether individuals will interpret the items in the same way. 
• We spent time talking about the number of scale points that we might include on our engagement. We emphasized the importance of balancing giving people options so that we get variability with a need to ensure our measures are reliable. 
• We also talked about the importance of using the best anchors with the goal of ensuring that we collect the most accurate information. 
• We emphasized the importance of a clear communication plan for launching and administering the survey. 

Key Takeaways:  

• When creating an engagement measure, be sure that your items are clear and fit within the test blueprint. You should also ensure that the items are interpreted consistently by respondents. This can be facilitated through engaging various stakeholders during the item writing process. 
• Think carefully about the number of scale points that you will use and the anchors that will accompany each of those scale points. Good items can be undercut if the anchors and/or scale points create confusion or frustration for the respondents. 
• Make sure that whenever a survey is launched that there is a clear and comprehensive communication plan in place to ensure that people understand what is being measured, how the data are being used, and why it is important to respond.