Reporting - What is it good for?
You might think that asking what reporting is good for is an odd question. Reports are one of those things, especially in the world of LMS, which we almost take for granted. In a typical LMS procurement process, reporting will always be one of the top requirements - although it’s usually one of the least clearly defined.
Running reports is just one of those things we do. There seems to be a notion that measurement (and therefore reports) is intrinsically good - after all, don’t they say what get’s measured gets done?) so we just carry on doing it.
Most organisations generate a ton of data, but rarely take the time to think about why. They should be asking themselves: what do we do with all that data?
Why do we do what we do?
Some people may say that we have to do lots of reporting because we need to measure learning. That’s a nice idea, but we’re not measuring learning (at least not in any meaningful way). Our reports tell us:
- On this date person X attended this workshop, on this subject.
- Or person Y spent 30 minutes looking at this elearning module on that subject.
At best we may measure training, but probably all we actually do is measure activity.
We do this because it’s easy. Measuring training would be hard and measuring learning would be nearly impossible. We may be able to very crudely measure that someone knows more after a learning or training intervention, but what does that tell us?
The problem is that we often set out to measure what is actually important, but because that’s difficult to do we end up treating what we can actually measure as important.
As a result we end up with lots of data, but we don’t know what to do with it. In fact it’s quite likely that there isn’t anything at all we can do with some of this data.
The problem is twofold.
- All we have is data when what we really need is information. The difference being that information is data that has been processed to give it context and meaning
- The information that we do have is rarely actionable.
So are reports a waste of time? Most definitely not. Reports can be an incredibly useful tool, but you need to think carefully about what you need to know and why.
All data should be processed into information in the context of the actions that you will take once you have seen that information. Indeed, before any reports are developed or run you should be able to state very clearly what questions that report will help you to answer. What decisions will it support (or not) - what action will you take if it gives you answer A or answer B?
For example, if you find yourself thinking something like “it’s been a month since we launched that product knowledge course about the new widget, I should run a report to see how many people have done it” - ask yourself “why?”
If the answer is “it will be interesting to see how many people have completed it” (or a variation of this) what you are about to produce is not actionable.
If the answer is more like “by this date we expected to reach 20% of our staff, if the figure is lower than that I need to investigate why” then that is most definitely actionable. You have a target and if it hasn’t been achieved there is an associated action.
Planning a report
Here’s a simple structure for planning a report.
What is the action?
What is the decision that I need to make? What is the action associated with that decision? If there’s no decision or action, then no information (or report) is needed.
What information do I need?
In order to make that decision what information will I need? You may find that not all of it resides in the LMS.
What context does the data need?
Each piece of data will need some context that is additional data that will help to make sense of it. That might include when something was done, who did it and what else was going on.
Large scale reporting on activity alone adds little value. Indeed it may even be damaging because it can reinforce the impression that L&D is something separate to the work - it’s just an activity, something that has to happen.
On the other hand small scale, actionable reports can be a powerful tool in helping you make the right decisions for your learners and your organisation.