Thanks.

Check it out and watch for updates here. I expect a few more QTI experts and pundits from other specification bodies to have some observations soon. However, an unnamed organization with a litigious nature may be unwittingly suppressing discussion.

What is SCORM Cheatlet?
I coined “cheatlet” as a portmanteau of cheat and bookmarklet (itself a portmanteau of bookmark and applet). A bookmarklet is a browser bookmark that uses the javascript: protocol prefix instead of the typical http: prefix used for web pages. This allows one to create a bookmark that runs JavaScript code in the browser.

I dove into bookmarklets when developing my iPastelet utility for iPhone in the summer of 2008. It immediately occurred to me that this technique could be an interesting way to hack/cheat the SCORM JavaScript API. Thus was born my implementation of the cheatlet. It worked easily and nearly immediately. By clicking a bookmark, I could send a score to an LMS. I tuned it to send a time, a status, and in a sinister turn, close and then nullify the API object handle to prevent any legitimate data from overwriting the hacked score.

Cheatlet Concept goes Public
In late August 2008, with a working demo of this code in hand, I sent it to major players in the SCORM world, including a major ADL contract agency, major LMS vendors, SCORM code suppliers of various types, and tool vendors. Many vendors responded that it was simply an instantiation of a hack to a known issue. One indicated they were well aware of this and raised the issue early on to discourage the AICC from deprecating HACP for the JavaScript API, that vendor was Questionmark (Disclosure, I started work for Questionmark in January 2009).

I seemed to be more concerned than others. In late August, I submitted a paper on the issue to the LETSI SCORM 2.0 Workshop, without disclosing precisely how to implement or code the exploit. The paper, Security Before Features was discussed online and at the Pensacola meeting in October 2008, but little seemed to happen as a result. Work continued on SCORM 2004 4th Edition without any API changes.

Cheatlet Example/Running Code Goes Public
Flash forward to a few weeks ago. Working completely independently, Phillip Hutchison had a similar moment of inspiration and crafts his own SCORM “cheatlet” bookmark. The big difference was this included a working cheat as a link right in the blog post (chealet linked since removed, but code is still available by request to Phillip). Soon this issue received broader attention. Reaction and opinions flew about the internet. Plateau proactively sent a letter to its US government LMS customers about this issue to help calms their fears. Ironically, I think this also drew further attention and discussion.

Defense Approaches, Work-arounds, Opinions
Soon after the example code hit the internet, opinions flew between experts on twitter, email and blogs. I posted a overview of the issues on the Questionmark blog, along with a follow-up post on general defensive strategies, including some specific solutions to support those approaches.

• The Importance of Security and Integrity of Performance Data
• Defense in Depth: Security for SCORM and Beyond

The ADL posted workarounds for some aspects of the vulnerability. Both are more clearly aimed and HTML/JavaScript coders, but that may be exactly who read this blog.

• SCORM Content Vulnerability Workarounds by Jonathan Poltrack
• Securing Your Assessments, Excerpt from Carnegie Mellon Best Practices Guide for the Design and Development of SCORM Assessments (means to make “View source” more challenging)

Some feel quite strongly that SCORM has never been suited for more than the lowest-stakes elearning events. I feel that stakes must always be viewed as a continuum. Furthermore, I believe it is far easier to consider the high stakes end of the spectrum first and back-down as necessary. In any environment, is incredibly difficult to start with little or no security and patch your way up to a secure system.

Others may see it differently. I’m sure there will be plenty of opinions. I’d say my views on this issue tend to align with Phillip Hutchison (whom I’ve never met, but respect) more than Mike Rustici (whom I’ve known & respected for many years). I’ll come back and update this post as they emerge. Other opinions include:

• SCORM security (two kinds of SCORM people), Phillip Hutchison, Pipwerks
• SCORM Security – Some Perspective, Mike Rustici, Rustici Software

Read it. Think about it. Who are the real-world, radical change-observing “pragmatists” and who are the in denial status-quo with incremental-change “revolutionaries” in your organization?

Are the people who say that the now and the future is in informal learning, collaboration, mobile and social networking the revolutionaries,or the pragmatists? Are the experts those wizened experienced people who say learning & training have been and always will be structured, pre-defined and centralized, (and they often add or else it is wasteful and inefficient).

Look outside your windows (or preferably Mac) there is a whole world (-wide web 2.0 ) happening. What the heck, check it out on your phone or Xbox or …

See past the matrix illusion of the Central Committee’s integrated-firewalled-siloed starts-and-stops-at-your-enterprise LCMS-LMS-authoring-tool including Centralized Succession Planning, now with connect-to-your-actual-cubemate-Social-Networking™.

Got it? Good. Now go read two Jay Cross posts, New Roles for Former Trainers and then Agile Instructional Design. For bonus points tonight or tomorrow, twitter (+2), text (+1) or email (+.05) a few colleagues and collaborate on how you can apply scrum techniques on your next training or elearning effort.

Feedback? Like this kind of post? Let me know, I’ve got a few more cans of elearning willy pete in the armory.

versus

Fortunately my typical vanity search did find me. Even more interesting is that Cuil quickly “learned” about SCORM in just an hour or so. Click either image to see more current results.

I’m always ready to throw a few stones at learning and training, yet also ready to come to the aid of an industry that has comfortably paid the bills for me. [I'm a bit conflicted myself, but it is an industry that has provided cost-effective, mission critical training to sales associates, fighter pilots, jet mechanics, commercial aviation mechanics, construction managers, accountants, law enforcement personnel, healthcare professionals, and even software developers.]

PONG- So I wrote a comment to Brooks’ post, feebly defending the industry that is my patron. [Dang, I want to drive a Boxster again.] Seriously though, if the point is to raise awareness to improve things– I am all for it. Allow me a brief aside on criticism.

People have found fault with elearning, computer-based training and its precursors since the days of TICCIT and PLATO. It might just be endemic to any form of compulsory knowledge transfer; few text books or training films ever become a NY Times Bestseller or a box office smash.

All along the way the criticism has generally raised the capabilities, quality and effectiveness while lowering the costs. Expectations rise too. Things change. Cutting edge and high quality always has a price, but those expectations are a moving target. The green screen training that had text-based role plays, probably took as many development hours as a similar Flash-based piece with a digital avatar today. BUT it only played on the corporate or campus mainframe, and you were quite lucky if it did more than show text and beep.

So, Hail to the critics, they have challenged the industry and industry has responded. Likewise, Condemnation to shameful designers, they besmirch our trade with discouraging and unimaginative content. While I’ve been bored during presentations anchored with snazzy multimedia PowerPoint, I’ve been wrapt with fascination by compelling speakers with simple Kodachrome slides.

Thus my point, great content transcends technology. Great technology enables. Clever designers focus on the content first, and make good use what the technology enables. Was Shakespeare held back by the lack of Microsoft Office for Windows Vista or empowered by a simple quill? [Personally I think he would have used a Mac though.]

As I heard Dr. Michael Allen say earlier this year, “It is a poor craftsman who blames his tool.” And I might add, it is a poor industry that never improves its tools.

PING- Brooks posts again, Why Elearning Is Dead. And I respond here, to the problems he cites. With a veritable volley to each point. Hang on.

PONG- First point from Brooks.

• Reusable content, the raison d’être of SCORM / AICCC [sic]…

PING- Actually, the AICC exists to…

• Promote the economic and effective implementation of computer-based training (CBT) media.
• Develop guidelines to enable interoperability.
• Provide an open forum for the discussion of CBT (and other) training technologies

Quoted from the AICC FAQ page. As I recall, driving factors TWENTY YEARS AGO when the AICC formed, were economics and interoperability issues. Issues were things like the fact that there wasn’t a widely adopted digital audio file format (WAV didn’t exist). The CMI (Computer Managed Instruction, aka Learning Management System) specification work started a few years later, and focused on interoperability. At that time the desired level of re-use was the LMS itself. Believe me, it was NOT better when each set of training materials came with its own proprietary LMS silo capable only of running the corresponding proprietary content developed in that vendor’s proprietary authoring tool.

As for SCORM, a few of us remember when the “R” stood for Repository. I don’t know the back-story of why it was changed or by who. I like to speculate that it was partially a marketing maneuver to secure political support and funding.

PONG- Later in that point, he continues.

…reuse just falls flat on its face. I’ve found it to be near impossible to achieve reuse across departments within a single organization

PING- I’ll generally agree. I recall having conversations with Phillip Dodds about my desire for a disposable content object model. Meaning content object wrappers so cheap and easy, they became to consistent quality learning what the disposable cup is to the Starbucks latte.

PONG- Next point.

• Testing (SCORM + LMS) has been a failure. Despite all the fancy API features you still can’t reliably certify results. Physical environments and instructors are still required for anything needing mission critical result certification. We might as well be using simple survey tools rather than bloated standards.

PING- At first, I thought he meant the LMS certification test. I’d consider this point a “FOL” as I’ve seen in called some bug bases– a Fact Of Life, not a criticism of SCORM or AICC. Unattended remote testing for high stakes certifications (lives or livelihoods at risk) generally doesn’t make sense. If you must do medium or high stakes testing to do electronically there is just one way to go in my opinion, Questionmark. A great product, a company filled with people of great integrity, and they can more than manage low stakes testing, assessments and surveys too.

PONG- Next point on costs [or salesmanship].

• The cost of developing lean forward elearning experiences is at least an order of magnitude greater than its pitched at. In fact elearning is pitched as a cost saver when in reality its usually a net loss. Most elearning is PPT based because the cost of creating a compelling experience from an SME’s physical course is so high (at least that’s been my experience).

PING- Almost too easy to rebute. We’ve all seen things oversold, maybe even been reluctant participants in some way. As a developer/consultant, I had to backpedal on sales promises made at more than one previous employer.

I’d feel sad and try to avoid projects where my work is (un)recognized as a net loss. As for PPT versus costs– not every piece of elearning replaces a SME’s course. And in many cases a great blended design might shorten the overall length of a classroom course, and allow the instructor to convey better/richer material. In such cases the elearning serves the role of individualized instructor allowing each student to slow down only when they need to do so. This as opposed to an instructor slowing down a whole class of 25 for the one student challenged at the current moment. This works great for classes where students may speak different languages. Likewise, a shorter footprint for classes can be real savings when you have high volumes of students to train or training must span the globe. Sending students or instructors across oceans isn’t cheap, and you’ll need classrooms and hotel rooms too.

PONG- Next point on failures [or bad situations].

• Every LMS / LCMS vendor I’ve worked with gets a FAIL. They’re bloated, difficult to administer and use, and often require organizations to wrap their infrastructure around them (which just doesn’t happen too much). Again these tools are pitched as cost savers, but typically require full-time administrators and the large vendors have notoriously bad service track records.

PING- Sigh. More sadness. I know it happens. I guess I’ve been fortunate, working with some wonderful customers and vendors. I’ve been tremendously impressed with Accenture, American Airlines, Boeing, Herman Miller and others. They all faced challenges with partners and vendors, and both sides dealt with it well.

PONG- On to more failures, such as discoverability.

• Distributed content / repositories reign supreme whether on the Web or across organizations. Again the LMS / LCMS get a FAIL and SCORM SCOs have had little tangible value.

PING- Remember when the “R” stood for repository. Now I sigh for myself. I thought CORDRA was supposed to move things forward on this. It has not. Time for some good thoughts to be shared and popularized to solve this. [See original call for LETSI white papers]

As far as SCO’s having little tangible value, I think a few million Korean parents might disagree regarding the SCORM-based elearning their children receive. I believe Chrysler also has some hard numbers on savings they achieved with SCO’s, you can find it via this Google search.

PONG- Home stretch now, second last point is on community and standards.

• A real infrastructure and community never really developed, at least not on the scale we should reasonably expect. Actually you could say the Web raced ahead and that search (GOOGLE), Wikipedia, Creative Commons, etc. form the backbone of real elearning. Adding community features doesn’t mean your going to build a great community and standardization here might hurt more than it helps.

PING- I’m personally amazed that the little presentations I saw in 1996 and 1997 led to something the size, diversity and adoption level of SCORM. I don’t know what scale Brooks was expecting. It’s bigger than I ever expected when I helped form a tiny company to build an early standards-based CMI/LMS in 1993. Back then we had to explain what learning management was, it was all just content. The typical training management decision was build-versus-buy. The tools skills an CBT/elearning designer, developer or consultant had with one system had very little applicability to another

PING- Last point, on a failure to change the classroom paradigm (I’ll add, that is something that I never thought AICC or SCORM set out to do).

• The elearning industry failed to fundamentally improve the old classroom led paradigm. Big institutions still employ SMEs (Subject Matter Experts) as course developers and instructors. The elearning movement as we know it has largely failed to create tools that can effectively allow SMEs to create elearning courseware. This meant the introduction of a new class employees–IDs (Instructional Designers) and Courseware Developers. In most cases we’re talking about new hires under different managers and even departments. There’s a huge level of distrust between these groups based on paranoia, ego and organizational allegiance. All of this results in increased operational overhead (financial and development).

PING- In my experiences, when big institutions change it is either almost imperceptible slowly or relatively quickly due to major disruption or catastrophe. The good news is that for big institutions, elearning has not been a major disruption or catastrophe. Lots of big organizations get lots of content out quick, almost too quick. I’d now argue for more filters, and shorter content, and less content, as much as better content (which I think such filters would also bring).

Even better, for small and medium institutions, elearning been a huge improvement. It allows them to time-shift training with self-paced e-learning, span geography with virtual classrooms, and keep training far more current than classroom approaches and scheduling would ever allow. It makes it worthwhile to send out 5, 10 or 30 minutes of training. Far below the threshold of the duration we might expect for a class (hours or days) to take.

We shake hands after a game well played. Finally, the denouement.

All this said, there are some really fantastic people in the elearning world–maybe they’re going to kick some ass and surprise me with SCORM 2.0. :-P

I hope so Brooks. I hope you’re surprised, and I hope you’re one of those fantastic people.

I think this is important, as we could be at the cusp of a make-or-break situation for evolution (or revolution) of learning and training infrastructure. Much of the current e-learning and LMS infrastructure is grounded in the learning and training approaches of the 1990s (’80s? ’70s??). By comparison, today’s technical and learning environment is much more “read-write”, collaborative, social and nomadic– all while being more personal and individualized.

Excuse me while I meander, ramble and eventually get to the point of why it is important.

Recently, I realized I have been unwittingly (and somewhat weakly) channelling the thoughts of Dr. David Wiley regarding the isolated “read-only” static nature of LMS-centric training, by mentioning this in conversation and scattered bullet-points in presentations over the last year. I discovered this thanks to Brian Lamb, who I have never met, but who I remotely and greatly appreciate via his blog abject learning.

Brian in a passing credit mentioned that David provided a lot of meat for many of his own presentations last year, and then lead me straight to Dr. Wiley’s 2007 presentation Openness, Localization and the Future of Learning Objects. If you can’t take the time to watch/listen to the whole presentation, I’d strongly encourage you to advance the slides and time marker to the following points:

• 30:37 Engineer invasion and the next slide, Technical standards soup (Tom adds, Mea Culpa– and I’m not even an engineer)
• 15:27 Education vs Everday (a more cogent expression of some ideas I discussed at AICC in Hamburg last month)
• 13:27 Innovative in 1995 and next slide, Education v Everyday (revisited)

If that doesn’t inspire you to respond to the call for papers, perhaps it might get the attention of Dr. Wiley or Brian Lamb.

E-learning isn’t completely broken, but current specifications and infrastructure don’t match how we live, learn and work.

Interoperability specifications can’t completely fix that, but maybe, maybe, the specification efforts can be oriented to enable and facilitate more effective and more congruent approaches. Too often they seem resistant and brittle towards innovation.

I’m certain that LETSI looks forward to learning from and sharing your white paper ideas.

Elearnity Research – Vendor Navigator

A combination of that chart and the Clark Aldrich Chart of Consolidations might make for quite an interesting discussion of the future of elearning. Or at least make someone look like a genius for talking through both of these in their next presentation.

Are there other charts us genius-wannabe’s should be looking at? Let me know.