Valid vs. Available: The Data Trap Most Orgs Fall Into

Most organizations are confident they have data. Loads of it. But in this episode of the Kirkpatrick Podcast, Vanessa sits down with psychometrician and data-analytics professor Dr. Anna Lissitz to ask the uncomfortable question: is it the right data — and is it telling you the truth?

Anna, who wrote the triangulation chapter in Building a Culture of Evaluation, makes the case that a single metric — measured one way, at one moment, from one perspective — simply cannot tell the whole story. Worse, the data leaders feel most confident about is often the data that just confirms what they already believed. Together they unpack why completion rates and satisfaction scores feel safe but hide poor performance, what separates valid data from merely available data, and how triangulation turns shaky numbers into decisions you can stand behind.

In this episode:

• Why "we have lots of data" and "we have good data" are not the same thing

• The difference between valid data and available data — and why it matters

• How triangulation protects you from false confidence and confirmation bias

• Why a low survey response rate is itself a finding, not a dead end

• Where AI helps with evaluation — and where it can quietly mislead you

Ready to stop studying evaluation and start building it? The Kirkpatrick® Builder Certification is a hands-on, live-online program where you design a complete set of evaluation instruments across all four levels — for your own initiative. The founding cohort runs Sept 28–Oct 2, 2026. Learn more and register at kirkpatrickpartners.com.

Think your organization is measuring impact? Most aren't.

The EPI Maturity Assessment helps you quickly identify where you stand—and what's needed to better connect learning to performance and results.

👉 Take the assessment and get clarity on your next step.

Order the book behind the Culture of Evaluation framework and updated Kirkpatrick Model. The book comes with a Companion Guide to help support you on the journey.

👉 Order the book now

Kirkpatrick Learning and Performance Center of Excellence Certification builds capability. Accreditation builds systems. The Kirkpatrick Certified Organization accreditation recognizes organizations that have embedded the Kirkpatrick Four Levels® into how they actually operate — through governance, SOPs, feedback loops, and cross-functional accountability. If your evaluation capability would disappear when a key person leaves, this is the credential designed to fix that. Founding Organization status is now open — first year of annual maintenance waived for inaugural cohort members.

👉 Learn more and apply

Not ready to apply yet because you need to build the systems? Grab the SOPs. We're building a library of Kirkpatrick-standard SOPs — governance frameworks, evaluation procedures for all four levels, planning templates, and implementation guides — so you can build your infrastructure before you apply.

👉 Learn more

If you're ready to move beyond activity and start driving real performance, the Kirkpatrick Collective brings together leaders focused on applying evaluation in the real world.

Get practical tools, shared insight, and the structure to make better, evidence-based decisions.

👉 Join the Collective and start building evaluation as a capability—not just an activity.

Learn more about the Kirkpatrick Model

Watch the Show on YouTube!

Submit your Questions/Stories: Do you have a question you would like us to answer on the Kirkpatrick Podcast? Have a story about how the Kirkpatrick Podcast or other Kirkpatrick events/programs have positively impacted your career? We would love to hear from you! Follow this link to submit your questions and/or stories, and we may just share them on the next episode of the Kirkpatrick Podcast.

#KirkpatrickPodcast #LearningImpact #PerformanceImprovement #TrainingEvaluation #CultureOfEvaluation #KirkpatrickModel