So your learners completed a course and probably learned some stuff. That is great news. So what? Consumption does not always equal digestion, and so how do you know that the learning was effective? News flash – if the only metrics your learning organization is reporting on are time spent in a course, completion rates and grades, you are missing half the equation.
Because learning is rapidly changing to a modern pace of short form, bite-sized snippets of content that can be consumed and accessed anywhere, it is time to change how you think about measuring your learning effectiveness. As the title of the blog suggests, learning is not the outcome.
How Do You Measure?
The best way to measure your learning effectiveness is to step into a modern learning frame of mind. Take a step outside of the workplace and just think about how you measure really anything. Movies – they get thumbs up or thumbs down. Restaurants and hotels – they get 1-5 stars. So turn this around into learning, and ask all your learners to tell you what they like about the content and if it was relevant to them.
What Do You Measure?
Brandon Hall Group says that about two-thirds of organizations at any given time are thinking well below the idea that learning is driving performance. So if your learning organization is still measuring learning effectiveness as the number of people who took a course, well you are not alone. But it is imperative to change the way you think about this. The modern learner wants to consume small, bite-size pieces of content because they can easily digest it and then apply that knowledge at the moment of need. You are behind the times if you are still thinking in terms of courses, and how many learners completed them. Then you are simply measuring the learning, and not the change in behavior.
Connecting business performance to learning is still pretty hard, we get it. BUT behavior change is the lead indicator to performance so this is where your focus should be.
Here is a quick example of how behavior change lead indicators work:
Let’s say you just launched a new product and in order to meet the quarterly revenue goals, you must sell 10 of these new products. You have 10 sales people and roughly 12 weeks to accomplish this business goal. The first two weeks are spent training the sales team on the new product features, benefits and use cases.
What are the lead indicators here?
- New product training assessment score – per sales rep
- # of times the new product shows up in an opportunity – per sales rep
- # of times the new product video is given to prospect – per sales rep
- # of new product sales weekly during the quarter – per sales rep
All BUT the first indicate actual sales rep behavior change. If you just measure weekly sales without the other two lead indicators, you will only find out too late that the behavior change was unsuccessful.
According to Bersin by Deloitte, less than half of the L&D organizations they survey feel they are perceived by their stakeholder leadership as a strategic business partner. Part of the issue here is that many L&D departments are simply measuring learning, and not showing how the learning impacts overall business performance. As in the example above, maybe L&D does not have access to the data that the sales manager has access to? Partnership with the business is based on common goals, information sharing, and putting corrective measures in action together as needed to ensure that the learning “sticks.”
Here are 6 things you could be measuring learning against that are more strategic:
- Employee retention
- Employee engagement
- Increasing sales performance
- Reducing turnover
- Reducing time to performance
- Increasing customer satisfaction
You see the trend here. All of the above metrics are business performance metrics, and that is what you should focus on as the outcome of your learning content. Work closely with HR and Business Line managers to identify what those lead indicators are.
Who Do You Measure? And How?
Now that we know it is a change in behavior that leads to a change in performance, think about the learning as an experience. Is this piece of training content going to help someone do their job better? Will it help them advance their career? If the answer is no, then why not?
Through the Experience API (formerly known as Tin Can) and other mechanisms, we can start to track who downloaded what content, who responded to that content, and who commented on that content. And perhaps more importantly, we can look deep within a course, at the lesson, topic, activity or media level. What particular sections seemed to resonate most with people that keep going back to? What parts did they skip? How do those parts relate to our lead indicators to behavior change?
By measuring things at the granular level you can track what elements are impactful and use that information to improve the learning content you provide. It helps to really understand your learners and where they are coming from or what is stopping them from succeeding in their behavior change. And what is really exciting here is then you can better target your learning budgets to your various audiences or learning “personas”.
What is a learning persona? It is taking a page out of the marketing playbook. Think about who your learners are and what their needs are, because every learner is different. Some are at a desk, some are out in the field. Some don’t have access to Wi-Fi, some are always plugged in. Understanding more about your learner segments allows you to be more efficient and effective with your learning content.
At Xyleme, we believe that in order to build great organizations, you have to unleash and empower the people who make it all happen. With Xyleme as your partner, you can transform the way you author, publish, deliver, and analyze learning content to drive business performance. Contact us to learn more.
