As a training consultant, one of the most frequent questions I get asked is, “What is the most effective way to measure the success of my training?” to which I reply, “How much time do you have?” with a smile and a laugh, but being completely serious. Then in a more serious tone, I’ll ask a series of questions to truly understand what they need:
- What are you measuring?
- What is the purpose of your evaluation?
- How do you define a successful learning program?
- With whom are you sharing the results?
- How you are currently evaluating the impact of your learning programs?
The list goes on. Unfortunately, there is no straightforward answer to this seemingly simple question. When it comes to evaluating learning programs, there is no one-size-fits-all approach that works best. There are many variables (and obstacles) training professionals need to consider when determining how to evaluate learning programs, including the organization’s goals and culture, stakeholders and audience’s needs, needs assessments, performance analysis, resources, time, budgets, data collection methods and tools, and even the different types of knowledge and skills we are trying to impart.
Many organizations are not taking the time to effectively evaluate their training. When they do evaluate learning programs, most companies focus on participants’ reactions and learning, not the impact it has on the business results. According to Evaluating Learning: Getting to Measurements That Matter, only “35 percent of 199 talent development professionals surveyed reported their organizations evaluated the business results of learning programs to any extent.” In fact, many training professionals I speak to fear the evaluation process for one reason or another.
I will admit that early in my career, I didn’t take the time to effectively evaluate my training programs either. With a total of five people in the training department serving more than 10 departments and 1,000 employees, we had limited time and resources. We also found it difficult to quantify many of the skills we were training, especially the soft skills. However, once we started evaluating our training, we began seeing how they affected business results and contributed to the organization’s growth.
We learned what worked and what didn’t. It allowed us to refocus our priorities and resources and make necessary changes to programs that weren’t effective. We also began getting leadership’s buy-in for other training programs because they were seeing quantifiable measurements on productivity, metrics, customer satisfaction, employee engagement, and turnover. Lastly, we were more invested in what we were doing and how we were doing it. The evaluations helped validate that our learning programs were making a positive difference on the overall organization and its people. The more we evaluated our training, the easier it was to measure. The more we planned the evaluations ahead of time, the less time-consuming they were at the end. The evaluation ended up becoming one of our favorite parts of the process because we could see the fruits of our labor.
Our evaluation process was a very “holistic,” fluid one that connected many of the learning program elements and business needs. Once we found what worked, we continued using the same model with slight variations depending on the behaviors we were trying to shape or metrics we wanted to improve. Below are a few things to consider when evaluating your learning programs that we found helpful.
1. Planning the Evaluation and Needs Assessment
When it comes to evaluations, we can’t treat them as an afterthought. Effective evaluations always begin with careful planning and purpose. That means thinking about:
- what you want to learn and measure from the evaluations
- the data collection methods and tools you will utilize
- whom you are getting the information from
- the timing of collecting and analyzing the data
- whom you are reporting the information to and how. (Know your stakeholders! Meet with them prior to delivering the completed evaluation to ensure what and how you are presenting is valuable to them.)
The list goes on. The point is to be purposeful in what, why, how, when, and whom you are evaluating, as well as to whom you are reporting before you even begin to design and develop your learning program.
The needs assessment phase serves as the basis for so many things in the learning program. Particularly for evaluation planning, a needs assessment, along with performance analysis, helps us identify stakeholders, current performance, desired performance, gaps, metrics, baselines, and other important data that drives the evaluation. As you collect data, you can even begin to formulate the quantifiable measurements of the evaluation. Needs assessments can provide baseline information to track learning, behavior changes, and increased business results that signify a successful training.
2. What Are You Measuring?
The first thing you need to be clear on is what you are measuring. For instance, do you want to know if someone liked your program or transferred a new skill back on the job, if the training yielded increased business results, or the program’s return on investment (ROI)? These questions should be asked as part of your needs assessment, even before the design phase, because they help shape the overall learning program. The evaluation phase of a learning program doesn’t happen at the end of the program; it happens throughout the entire process.
Kirkpatrick’s Four Levels of Evaluation and the Phillips ROI Methodology help identify specific measurements for learning programs broken down into different levels. The higher the level, the more valuable the information and significance it has to the overall organization.
By anticipating and quantifying your expectations of the evaluations, you will be able to see if your learning programs are on target. If the actual numbers are different from your projected ones, that is an opportunity to learn what happened, why, and how to improve it for next time. Conducting the needs assessment as discussed above helps you project these targets. These targets also help to shape the learning goals of program, another critical element in effectively evaluating learning programs.
3. Learning Goals
Effective programs always start with learning goals that tie back to the department and overall organization’s goals. By asking, “What outcome do we want as a result of this learning program?” you will not only isolate the program’s learning goals, but also identify the results.
Examples of desired results could include higher customer satisfaction ratings, increased sales, more productivity, less overtime, fewer defects, more employee engagement, and less turnover. Whatever the desired outcome, learning goals need to be specific, measurable, achievable, relevant, time-based, and tie back to the department or organization’s goals.
Ultimately, once the learning goals are achieved, you will know your training was successful. You will have also contributed in helping your organization reach its business goals. One tip: Be sure to get leadership’s agreement on these goals at the very beginning before designing the learning program. Getting everyone on the same page at the beginning saves you a lot of time and energy in the end.
4. Learning Objectives
Once you know the specific learning goals or outcomes of the program, the next step is to create specific learning objectives, which tie back to the learning goals, which connect to organizational goals. Do you see a pattern here?
Learning objectives provide another way to measure the effectiveness of your training that is more detailed in the knowledge, skills, and behavior the participants are acquiring and applying on the job.
The ABCD format of Learning Objectives provides a four-component structure to writing effective, measurable objectives:
- audience: who is being trained
- behavior: what will the audience be doing or saying; this may include knowledge, skills, or attitudes
- condition: resources, tools, or equipment used to complete the desired behavior
- degree: how well or how often the audience will be able to demonstrate the behavior.
If the learning objectives are done correctly, you could measure the success of your training once the audience has met these objectives either right after the program or a few months back on the job.
Some of you may be wondering, “Shouldn’t I determine the learning objectives and goals before deciding on what I am going to measure?” It depends. Evaluating the effectiveness of our learning programs is much more of a dance than a linear, step-by-step process. As stated in Real World Training Evaluation, “Evaluation is both a science and an art.” Although it may seem very analytical, there is so much more that goes into evaluations. We have to be able to effectively plan, work well with all stakeholders, ask the right questions, identify and communicate actual needs and performance gaps, design and develop relevant learning programs, and make future recommendations based on the evaluations to seamlessly measure the success of our learning programs.
So back to the question, “What is the most effective way to measure the success of my trainings?” The answer is “all of the above.” If you only have time or resources to do a couple of things, evaluating how the learning program contributed to the business results will probably be the most significant (depending on your organization).
No matter how you choose to evaluate your learning programs, one thing is for sure: As the landscape of talent development continues to evolve, evaluation is a necessary task for learning and development professionals. Effective evaluation is key to ensuring more learning programs in the future.
Evaluation is just one element of an effective training function. Join me for ATD’s Training Certificate to learn how to evaluate program impact at different levels using a variety of methods.
