Techniques and Best Practices for Measuring Training Effectiveness

Many still view the measurement of training effectiveness as a black art. But in five years of research, I have identified key philosophies, techniques and best practices that are practical and really work.

The goal of training measurement is to deliver useful information for decision-making. You must collect consistent, repeatable data so you can make meaningful comparisons of programs, audiences, instructors, delivery methods and other factors. The most practical approach is through standardized, end-of-program assessments for learners and their managers. At Eaton University, KPMG and AT&T, learners must complete standard, end-of-course assessments that include satisfaction and effectiveness measures. The organization can then plot trends over time to identify well-designed programs and problem areas.

KMPG Canada and Randstadt each ask their learners standard questions such as, “How much did this training program help you increase your job performance?” Although they’re subjective, responses are clear indicators of relative effectiveness. Another technique is to send two assessments: one at the end of the program and another 90 days later. Although the second assessment will typically have a lower response rate, it provides a better understanding of job impact.

Use Job-Impact Indicators
One of the most important measures of a well-designed program is direct impact on job performance (called Kirkpatrick Level 3). A simple and powerful approach to measuring job impact is to use a series of standard assessments such as: How much impact did this training program have on your job? (standard numeric ranges); how well can you apply the skills and knowledge you received in this program on the job? (standard numeric ranges); what are the biggest obstacles that will prevent you from using these skills and knowledge on the job? (multiple choice answers).

These indicators will give you significant understanding of how the program is used and applied. As KPMG Canada discovered, many learning programs rated high for learning impact do not score nearly as well on job impact.

As you apply standardized assessments, you might find one program with a slightly lower satisfaction or effectiveness rating than another. Don’t jump to conclusions based on a single score. You must do additional analysis to understand which programs succeed, which fail, and why.