On 8/27/14, I hosted a webinar alongside Larla Bogle and James Merrill from AT&T entitled “How AT&T is Using xAPI to Enhance Compliance Training”. The attendees had more questions that we could answer during the live webinar, so we’ve posted the questions and the answers here.
Q: Why did you need xAPI instead of SCORM 1.2 or SCORM 2004?
Q: What did xAPI allow you to do that SCORM 1.2 or 2004 couldn’t?
xAPI enables us to collect data from many different systems (not just traditional e-learning experiences in an LMS). In addition to the Nexlearn simulation, this project also tracked data from a video platform, an intranet site, surveys and assessment tools.
There are some aspects of this project that SCORM could have helped with, however using SCORM would have provided a very limited set of tracking data. SCORM doesn’t have the rich vocabulary necessary to capture the learner interactions that occur across many different training modalities.
It sounds trivial, but simply being able to launch the learning experience without the overhead of entering a LMS was a big win that xAPI facilitated.
Larla alluded to a delayed phase of this project that includes directly collecting data from the compliance incident tracking system to judge the ultimate effectiveness of the training and its ability to change behaviors.
From James Merrill:
“We are limited on the types of SCORM statements we can make, report on, and analyze due to the massive volume of traffic we have on our LMS. It simply can’t handle the extra statements we would have to send and we would risk performance issues on the LMS that would impact the user experience. Our goal was to also go mobile and our traditional LMS does not play well with mobile devices due to the potential loss of connection.”
Q: That sounds great, but can’t it also be achieved without xAPI?
xAPI doesn’t create data, it simply makes it easier to capture it. So, yes, anything that can be done with xAPI can also be done without xAPI, it’s simply a matter of how much work it would take.
Theoretically, one could go into each of these systems and capture all of the required data in a CSV. One could then masage that data in Excel and glean the same insights. The key difference that xAPI makes is the seamless, real-time nature of the data flow.
From AT&T’s perspective, if this project was just about collecting this finite set of data and analyzing it, then today, it might have actually been less effort to do the project without xAPI. Their choice to use xAPI was an investment in the future.
As xAPI continues to gain adoption and become the de facto communication protocol for learning events, projects like this one will be a simple matter of configuring an LRS. It is this future state, the time when all of this data is freely and easily accessible, that is really exciting.
From James Merrill:
“We could have captured the data in a database but there is something to say about having an industry communication standard that is as simple and beautiful as plugging in your iPhone charger and knowing it will work in any any electrical outlet.
Q: What types of activities did they track from the SimLearn simulation – I saw which major activities – but didn’t see what activities within were tracked and what was helpful
Q: I didn’t see how the Experience API was used in this
Q: Guys..respectfully could you tell us how you are using xAPI
From James Merrill:
“For clarification, the tool is called SimWriter. Each time the learner was presented with a decision, we tracked what decision was selected. We also tracked the selection of the help button, the “rewind” options such as the time machine, and all of the resource links. Pretty much everything that required user interaction we tracked it using API statements. I apologize if you wanted to see code but I didn’t think that would be of interest. I used standard get/send and State API statements.“
We intentionally stayed away from many of the technical details on this webinar. This presentation was targeted to a slightly different audience than many of our previous, more technical webinars. But we’re geeks at heart and happy to dive into the technical details. Give us a shout if you’re interested in the nitty gritty.
At the very highest level, in case it wasn’t clear, xAPI was the way we collected and aggregated data from several different applications (none of which were in an LMS and only one of which was even a traditional “learning” application).
We tracked every choice they made in the simulation. We tracked all the resources links that were clicked, and rewind and replay buttons. This was beneficial because we can measure engagement and interest, not just through survey responses but through looking at the learners’ actual behavior.
Q: What advantages specifically did the Record Store give that the LMS could not?
Q: So is AT&T keeping their LMS or has the LRS replaced it? What are the differences? How do you pull reports?
Q: How extensive, if any, is the dependence on an LMS for this? Is it all self contained and recorded in the Watershed LRS?
Q: I didn’t understand if AT&T was eliminating their LMS and how they were incorporating the completions from LRS with LMS. Can you eliminate need for LMS if you have LRS and xAPI?
AT&T is not eliminating their LMS any time soon. It is still a massive and mission critical application. This project was simply a pilot of how xAPI could enable a new model of compliance training delivery.
I often get asked whether the LRS is going to displace the LMS and my answer is “maybe, but not for a long time”. LMSs are still useful tools that serve an important role in organizations.
However, if the question is “can you eliminate the LMS”, then the answer is “it depends on what your needs are.” If you simply need to track results, then, yes, an LRS can do that just fine. That is often what an LMS is used for, but LMSs also contain a LOT of other functionality that an LRS does not.
When we originally coined the term “LRS”, our conceptualization of it was “the part of the LMS that tracks results and implements the Experience API to store and retrieve results data”. When thinking about whether an LRS can replace an LMS, it can be helpful to keep that definition in mind.
This particular project was completed without touching the LMS. All of the training is delivered from an easily accessible portal that does not required logging into the LMS. That is an important capability of xAPI; learning can live anywhere. Training resources can be easily accessible for performance support wherever they would be most convenient for the learners.
Watershed LRS allows you to collect and analyze a much broader set of data than a typical LMS. By and large, LMSs are designed to report on formal training activity that happens within the LMS system. They are not typically equipped to analyze data from many different sources and to present that data in ways that are helpful to validate a hypothesis.
From James Merrill:
“xAPI and the LRS will be a key player in helping us capture and analyze data that we can’t capture and analyze today. When formal completions on content delivered outside of the LMS we will integrate a method to send completions from the LRS to our training history database.”
Q: When you say donation…are you saying that associates actually took the course after promising to give to a charity?
Q: I like the idea of linking the compliance with community donation so staff can feel empowered and make a positive contribution.
AT&T made a charitable donation for each learner that participated in the Code Challenge.
Q: This is compelling that NexLearn and Rustici supported each other to support AT&T. Are OR can you work with LMS company’s so xAPI is supported. Most specifically we are using SkillSoft’s LMS capabilities. Can you work with them?
Yes. For the past 12 years our specialty has been helping LMS vendors to support standards well. If the vendor is willing and cooperative, we can tremendously expedite this process.
Vendors are adopting xAPI on their own every day, but to make the promises of xAPI a reality we need even more adoption. Tell your vendors that you want to see them implement xAPI (and do it well).
Q: As opposed to checkbox “I read and understand” compliance, how would you be able to provide information about learner compliance to some legal authority or auditor?
The learners’ participation was tracked in great detail via xAPI and all of the records are stored in Watershed. It would be very simple to show an auditor just how much training was provided to the each learner.
If we wanted to, we could have passed completions to our learners’ permanent training history, but for this project, we intentionally did not to test the voluntary aspects of the proof-of-concept.
Q: Once learners are aware of xAPI and input tracking in general do you think it will make this entire process superficial?
Good question. I personally think it comes down to presentation and messaging. If we present a situation in which “failure” is the expected norm (as it is in branching simulations or games), the fact that those “attempts” are tracked probably won’t be perceived as threatening.
From Larla Bogle:
“To make it clear, for The Code Challenge, we intentionally tracked the interactions but made the outcome reporting anonymous. This was a part of our initial communication and did not seem to hinder participation. As with anything, we will need to decide based on the learning situation which approach is best for each learning opportunity.
From an educator’s perspective, I think the tracking will do the opposite of making it superficial. It is human nature to perform at a higher level when we are being observed. This has been one of the benefits of classroom training we had to sacrificed for the efficiency of Web-based training. xAPI brings back this accountability. If learners know that their interactions are tracked, there is the hope that this might motivate a higher level of participation. Instead of just knowing that a learner reached the end of a course or passed a test, we can now know if the learner really did access the pop-up, hyperlinks, etc. that are in our online courses. Clicking the link doesn’t guarantee learning, but it tells us that the learning opportunity was taken.
I am most excited about what this information will do for us analytically to help us design better and more effective training programs. We can identify top performers by job titles and trace their learning backwards. What optional learning opportunities did they complete? Did they interact differently with content than lower performers who took the same courses? Equipped with the knowledge we can make our training more effective and efficient. xAPI and the LRS will make Level 3 evaluations much easier and more realistic to all of us.”
Q: Hi — these are great visuals! Could you slow down a bit so we can see them a bit better?
Sorry, we had a lot to cover. All of the slides are available at ../webinar.
Q: Have there been similar success cases from any customers using xAPI & Watershed with training delivery through a traditional LMS?
Yes! Stay tuned for the next exciting episodes of “Mike Talking About xAPI”. 🙂
