• What do you do with the data?

Do you have your finger on the pulse? (visibility & tracking)

Hopefully you spend a reasonable amount on training and development.  You may run courses, action learning sets and webinars.  You may (I hope) have created some e-learning programmes which learners will complete either standalone or as part of a blended solution where one activity (such as completing an online module) will be a pre-requisite to some other training intervention such as a course or an on the job activity.

If you have taken the e-learning plunge and either created or purchased access to some online learning tools, the chances are they are SCORM compliant.  The Shareable Content Object Reference Model (SCORM for short – please feel free to immediately forget what the actual letters stand for) is just a way that Learning Management Systems (LMS) record who does what, when and how well.

SCORM is the software protocol which enables you to see just how many of your learners have still not completed that really important health and safety course.

So if you have e-learning which is accessed through an LMS, someone somewhere has the ability to generate a report about which learners have completed which module and – assuming there’s an assessment worthy of the name and the server space – a way of finding out whether Frank in Accounts has passed his anti-money laundering online module.  In sophisticated systems, with a little administration, you may even have a complete training record for everyone in your organisation – face-to-face courses completed; online modules started, in progress and passed; certificates achieved; work-based projects downloaded, etc etc.

You may have judged from the tone of this piece that I’m a little cynical about all this tracking stuff.

Why you may ask (assuming you give a monkey’s uncle)?

Let’s be clear.  Tracking = good thing.  No problem with gathering data about what people have done or what they still need to do, or even how well they got on.

What do you do with the data?

This is where my problem starts.  When I was a training manager routinely I would report on the number of courses delivered and the number of delegates.  The measurement (such as it was) was exclusively focused on inputs rather than outputs.

My first issue with LMS data is that we rarely look at outputs even now – we look at completions  – how many people went through the e-learning rather than how much they learned.  Of course the assessment is important – did they pass?  At least it would be if the assessments were particularly taxing.  I’ve rarely seen an assessment in an e-learning programme I couldn’t pass without using the learning first.  Often this isn’t because I know a lot about everything – quite the contrary – but because the way the questions have been written gives you a huge steer to the correct answer.  Of course, you could set the pass mark at 100%, but then you get a question which is ambiguously worded (or just nonsense) and all hell breaks loose when no one can pass the assessment.

I once showed an assessment to a very experienced subject matter expert who had actually written the policy on which the programme was based.  The person with whom we were working had assured us that the questions he had supplied were the minimum standard despite everyone in our office really struggling to work out what the answers were.  He also insisted on a 100% pass rate and was unmoved by our contrary advice.  When the guy who wrote the policy couldn’t get more than 60% the questions were soon changed!

So checking assessment scores is not always the reliable measure of outcomes that you would wish.  And in any case is learning just about passing some kind of exam?  Isn’t it about going and doing something with what you have learned?

Imagine the situation:

• 100% of learners within a department complete a training programme.
• 100% of learners pass the assessment.  Pretty good, huh?  You’d claim success and probably win an award, wouldn’t you?
• What if the course was flower arranging and the learners worked on an oil rig?  Still good?

I’m exaggerating of course, but sometimes I see training programmes which might as well have been as divorced from the business need as this example.  Even if the course and the work to be done are harmonised, the assessment measure doesn’t tell you whether anything changed or not.  At best, in an environment requiring proven compliance it makes sure that the organisation can (perhaps) avoid being sued when something non-compliant happens and at least they have a record to show that they had told that person not to do it and they had shown that they knew that  they shouldn’t have done it.  Let’s hope the non-compliant action doesn’t result in major financial losses or even global economic meltdown.  Covering your back then may look a little inadequate.

So, LMS – what are they good for?

Everyone of a certain generation is now singing along with “absolutely nothing” but that would be far from true and my cynicism doesn’t extend to all tracking and the visibility which an LMS brings.  I merely point out that used stupidly, you get stupid results.

So what would good be like?

In an advanced LMS, one which – like our own Involve system uses Web 2.0 and collaborative technologies in an intelligent way – the LMS gathers data on a learner’s activity and uses it to build a profile of them.  This is then used to generate recommendations for follow-up learning and to generate emails to line managers and designated coaches who can then reinforce these recommendations.  Those recommendations are based on three things:  Role profiles mapped to competences; patterns of use by other learners and recommendations from other users.

Sound familiar?  You may have come across something similar called Amazon or eBay.  We call this transactional networking – an LMS which not only tracks learning but provides dynamic visibility of other learning resources available to that individual.  When we build blended programmes we can not only promote the next stage in the learning journey – workshop, work-based task, coaching session etc. – we can ensure that the L&D team know to enrol the person on the next stage and prohibit attendance by those who have not completed the necessary pre-requisites.

A good LMS also generates reports about users and may be able to group them by function or grade.  How about grouping them – and reporting – by line manager?  Want your line managers to encourage learning and to play an active part in developing his or her people?  Well, now you can look at patterns and identify outliers – those who are really encouraging learning and providing time and support to people to develop their skills and knowledge, and those who apparently have a team who require no further learning to perform marvellously.  A quick check of whether there is a correlation between those who train and those who get results or retain staff, might be an instructive process.

Using web 2.0

If you have set up the LMS to speak to the learner and act as an aide to learning management rather than a policeman of learning diktat, then you may have other functionality available.  How about the chance for people studying a subject or learning a new skill to ask a question of your in house expert?  How about making that conversation available to all learners, globally or in a specific function or geography?  You’ve now started to build new content and information in the system, and if you also enable people to post their own information then you may provide access to hints and tips and experiences.

Now this is not to say that such growing content is easy to generate or moderate.  Nor is it to build the idea that somehow this is necessarily learning – of course what people post up there may be of use to no one else and often it may just be an unusable fragment of what’s needed.  In an environment requiring compliance, it may also be downright wrong and the presence of such material on your intranet may not make happy reading during the dawn raid from Her Majesty’s Inspectorate of naughty things which companies do!

However, if you have got the facility to see what people are doing and sharing and you can place a governance process in place which doesn’t discourage the sharing of information you would want, then you have some more useful data.  Interestingly, not what people have posted, but the fact that these people have posted and shown themselves sufficiently motivated, engaged and interested to do so.  Having trouble getting your managers enthusiastic about learning?  Swap them for the people who are sharing their knowledge in this transactionally networked world.