Big data

15th October

Big Data has been much talked about over the past few years and recently people have begun to take an interest in how it can be used to improve learning.

In April 2014 the Virtual College, in its report ‘On-line learning - Threat or salvation for further education?’, noted that the government had highlighted Big Data as one of the factors ‘playing important roles in delivering this newly-focused employer-led training provision’.1

However, just because people are talking about it doesn't mean it’s actually happening. In their 2013-14 Benchmark Study, Towards Maturity noted that only 19% of the respondents used their LMS for Big Data or learning analytics.2

What is Big Data?

What actually is Big Data? This is where it starts to get a little confusing, because there is often a sizeable difference between the true meaning of Big Data and what people really seem to be referring to when they use the term (particularly in the world of L&D).

A widely accepted definition of Big Data was first described by Gartner in 2001 and then updated in 2012. Essentially it says that Big Data has volume (there is a lot of it), variety (there is a range of sources and types of data) and velocity (the data is generated and analysed quickly). Furthermore, Big Data is usually considered to include data sets so large that it is beyond the ability of commonly used systems to even be able to manage and process that data.

If we take Big Data at face value, it’s hardly surprising that there is little or no activity around its use in L&D; even in large organisations the data sets lack the volume, variety and velocity to be considered Big Data. Most L&D departments, or organisations, wouldn't have access to the tools needed to process these large data sets even if they had them.

So, does that mean that Big Data isn't relevant to us? Not at all. What we need to consider is the type of analysis that Big Data is capable of and then apply that thinking to the smaller data sets we do have access to.

Big Data in the real world

Here are a couple of examples of Big Data usage from outside of the world of L&D.

Inrix uses data collected from over 100 million drivers around the world, as well as roadside sensors and other sources, to predict commuting times. Estate agents have begun to licence this data to help their customers in choosing a home. An extra five minutes each way on a daily commute adds around 40 hours per year!3

Netflix uses Big data analysis to predict network traffic and the types of devices customers are using to access their service. This helps them to scale their infrastructure to meet peaks in demand - such as the release of a new movie or TV series.4

Adding value to L&D with Big Data

We can apply this thinking to L&D, but first we have to shift our scope well beyond activity completion.

For example, how do most organisations use data to plan their future spending and resources? I feel pretty confident in saying that generally, the only data used in these decisions is historical reporting of activity.

  • We ran x number of workshop y this year, so we’ll need to run another x of them next year.

  • This e-learning module was completed by x number of people, so we should probably build some more like it.

Not very helpful. Even if we’re working with a relatively small data set, let’s see how we can apply Big Data thinking to this.

Volume. We can start by looking at all of the training data we have. Sure, we should be looking at the high volume activities but we may need to think differently about what the data is telling us.

If we ran that workshop last year, and the year before what does that tell us? If it’s something that does need to be run regularly (e.g. as part of induction) can we reduce the cost of doing so by using technology. On the other hand, if this is a workshop on something like customer service we should be asking why it is we need to keep running it. If customer service hasn’t improved after two years then there’s a fair chance that the workshop isn't working - perhaps training isn't the problem.

Variety. The problem with training data is that it is almost exclusively historical. If you want to predict training needs you need to look at a more diverse data set.

  • You could analyse complaints data and identify common problems which might be addressed by training.

  • What about HR data on new starters - if you have a high proportion of people leaving soon after recruitment there may be an issue with induction training, or perhaps better recruitment skills are required.

  • Perhaps too much overtime being recorded, in which case some up-skilling on rota management might help.

  • Are your company car drivers having too many accidents? What are the circumstances of those accidents? If drivers are regularly speeding to get to where they should be, perhaps the issue is time management not driving skills?

Velocity. Given the volume and the variety of the data it goes without saying that this analysis shouldn't be a once a year activity intended to predict budget needs.

Data needs to be gathered quickly, analysed quickly and solutions offered quickly. It’s a constant process of monitoring and correction.

What about you?

Are you using, or planning to use, Big Data to improve learning in your organisation?

1) Online learning - Threat or salvation for further education? - David Patterson, April 2014. 
2) New Learning Agenda, 2013-14 Towards Maturity Benchmark Study. Towards Maturity, November 2013. 
3) http://bits.blogs.nytimes.com/2013/03/12/predicting-commutes-more-accurately-for-would-be-home-buyers/
4) http://blogs.wsj.com/cio/2012/10/26/netflix-uses-big-data-to-improve-streaming-video/

About the author
Barry Samson

Barry Sampson is a consultant focusing on the use of technology to improve workplace performance. In 2009 he co-founded Onlignment, a consultancy specialising in organisational communication and learning. Previously he worked in a range of delivery and management roles in HR and Learning & Development before becoming Learning Technology Manager at B&Q where he led a number of award-winning elearning and blended learning programmes.

Want to find out more?

Tweet Barry now!