When trainers want to evaluate the effectiveness of their courses, they usually give participants a reaction form asking for their thoughts on the program. If you have ever used a form like this, you may wonder how helpful it is, especially when most come back with a positive rating but give you no clue on whether or not the training was effective.
There are four basic levels you can use to evaluate training:
- Level 1 – Learner Reaction to Training
- Level 2 – Knowledge and Skill Mastery
- Level 3 – Behavior and Job Performance Change
- Level 4 – Business Results/Needs Met
Level 1 – Learner Reaction to Training
A level 1 evaluation is normally conducted with the reaction form. Many times, these forms come back filled with check boxes and notes that simply tell you the class was wonderful and the participants had a good time. Personally, I wonder if these forms serve a purpose; without critical feedback, how am I supposed to know what I can improve upon?
The key to making these forms effective is in how you word the questions and the options you provide for answers. Remember, the goal of training is to create change in how learners perform. Craft questions that ask about:
- Usefulness of the information to the participants’ jobs
- Whether they can incorporate the information into their daily practices
- How supportive their work environment is of new information.
The answers to questions like these can give you good insight into what you are teaching and how to follow up. After creating the questions, use a scale with five or seven gradients (for example from “Not Likely” to “Highly Likely”) to make the question easy to answer. Questions like these can be assigned point values (on a seven-gradient scale give one point for “Not Likely” and seven points for “Highly Likely”) and then tallied and tracked to see if certain areas score higher or lower than others.
Level 2 – Knowledge and Skill Mastery
A level 2 evaluation should verify whether the learner has acquired the skills they need from the training. There are multiple ways to accomplish this verification including tests, presentations, oral exams, and demonstrations of the skills taught.
Conduct a level 2 evaluation very shortly after the training; everything should be fresh and in the forefront of the learners’ minds. An evaluation at this level tells you not only if the learners are receiving the information they need, but also if you communicated the information effectively. If learner results are consistently poor at this level, it is more than likely due to the design of the course or shortcomings of the teacher.
Level 3 – Behavior and Performance Change
This level truly represents the aim of our training: to drive change in on-the-job performance. It evaluates how learners incorporate new skills in their daily tasks.
Allow some time between the class and this evaluation, typically 60 to 90 days, depending on the training and evaluation needs. Some of the methods to conduct this evaluation include offering the same test the learner took in level 2 and observing them as they perform the newly learned skills.
One of the best ways to evaluate at this level is to observe the learners in their work environment to determine if they are using their new skills. Create a checklist of skills the learners should be demonstrating before you conduct the observation.
Level 4 – Business Results/Needs Met
At this level you are evaluating whether the training program impacted the overall results of the business. The best way to do this is to compare key performance indicators taken before the training to those taken after the training at specific intervals. While improved job performance and positive change are the aims of training, meeting business needs should really be the heart of what you do.
If you are looking at those four levels of evaluation and feeling a little overwhelmed by the thought of conducting all of them, you should know that your business needs will determine what evaluation level is appropriate for your training program. Remember that every training program may require a different level of evaluation. Level 1 evaluations are easy to conduct, while level 4 evaluations are more challenging as they require detailed analysis. Choose the level that will give you the data you need to evaluate your training effectively.
In conclusion, remember the four steps to create effective training for adults:
- Analyze and clearly define the training need. (See CMM’s October 2017 Tips from the Trainers department.)
- Create specific and measurable learning objectives that will drive the program forward. (See CMM’s January/February 2018 Tips from the Trainers department.)
- Engage adult learners by reducing presentation time and increasing activity and interaction. (See CMM’s March 2018 Tips from the Trainers department.)
- Evaluate your results at a level that make sense for your organization and learning needs.
