One of my goals at mLearnCon was to figure out “this xAPI thing.” More importantly, whether I needed to worry about it right now.
My first impression – SCORM replacement. SCORM, to me, is the selection in the drop-down when I publish my tutorials if I want them to report scores and such. I then pray it works when I dump the tutorial into my LMS after, hopefully, choosing the correct SCORM in the drop-down during loading.
More drop-down options are OK – as long as those options make my job easier. The quick, surface take-away for me was “I don’t need to worry about this yet.”
However… during Aaron Silvers’ xAPI hallway session, I saw something…
One of my ongoing struggles is with analytics. Being able to show that my training initiatives and deliverables have some sort of impact on the business.
The LMS tracks stuff that educators have always tracked – usage (or “butts in seats”) and scores. The real meaty information, however, is in the enterprise systems.
Many enterprise systems have large, complicated databases on the back end. These databases have various field names and often use multiple fields for the same information. Pulling the information out of these systems often requires database management knowledge, Crystal Reporting expertise, and more than a little magical hoodoo. Some systems might have some moderately useful canned reports – but these often require further manipulation with Excel.
Running reports from my LMS and making up numbers is fine. But my Holy Grail is the ability to attach the stuff coming out of the enterprise systems with the stuff from the LMS without having to go through the following process:
- Define some reports with a very rudimentary knowledge of what I can actually get
- Talk to / bribe the Data Guru into taking on my project
- Wait for the Data Guru team to program the reports (if they take on the project)
- Pray I got the requirements right while I wait
- Hope they pulled the right information in their programming
- See whether I got what I thought I asked for
- If no, then repeat the process until either the Data Guru team gets angry, I get frustrated, or we actually get a useful report.
What if ALL enterprise systems output in the same language?
What if I could create my own reports on the fly?
I’d like to give an example from a conversation I had in mid-July with one of our subject matter experts.
Our organization is in the process of updating our expense reporting process.
I asked the subject matter expert what the team wanted to get out of this project.
“We want to reduce the amount of time it takes to do an expense report.”
After shaking off my amazement that I got a real, measurable business objective without having to use various forms of torture, I saw a real opportunity to possibly capture whether the project, and the training, worked.
Problem is… we don’t have a great system to capture the information we need.
Right now, if we want to determine whether the new expense report process worked – we would need to do a mess of surveys before and after the implementation. These surveys (hopefully) would have accurate records of how long it takes someone to create and submit an expense report, honest evaluation of each system (old and new) and a large enough feedback pool to consider it useful data. Then we would need to tack on anything we captured in the LMS. Then, somehow, we would need to collect information from folks who didn’t go through the training, but use the system anyway. All of this would then wind up in an Excel spreadsheet to show to the executives.
But what if I had xAPI compatible applications?
| Name | Verb | Direct Object | Activity | Timecode | Data From |
|---|---|---|---|---|---|
| John | Started | Expense | Report | 6/17/12 1205 | Oldsystem |
| John | Finished | Expense | Report | 6/17/12 1308 | Oldsystem |
| John | Attended | Expense | Training | Timecode | LMS |
| John | Read | Expense | QR | Timecode | Web Page Analytics |
| John | Started | Expense | Report | 7/17/12 1214 | NewSystem |
| John | Finished | Expense | Report | 7/17/12 1244 | NewSystem |
| Time used Start to Finish | 63 min | ||||
| Time used Start to Finish | 30 min | ||||
| Difference | -33 min |
This is really basic – I would probably want to take multiple iterations of the Start and Finish of both the old and new processes and get an average. But this is at least a start.
It would be really cool to then do a comparison of EVERYONE using the Old System, then the New System whether they took training or read the reference materials or not. Then take a look at those who just looked at the Quick Reference and didn’t attend training. That would let me know what is really working.
Essentially – I saw that xAPI potentially solves a problem. A big one.
Now all we need to do is get everyone to play nice with each other.
