SCORM 2004 interaction tracking: learner_response and correct_responses rejected by Moodle (error 406)

Legacy Evolve General Help
1

Has anyone else run Evolve’s SCORM 2004 interaction tracking on Moodle and checked what actually gets stored?

We publish SCORM 2004 4th Edition with Use Advanced Tracking enabled. Interaction tracking works, but two fields are rejected by the LMS on every MCQ and GMCQ submission. Tested on both Moodle 3.11 and Moodle 5.1, same result on each.

MCQ:

setValue: cmi.interactions.4.learner_response=1%3ATrue
Course could not set cmi.interactions.4.learner_response to 1%3ATrue Error Info: Data Model Element Type Mismatch Additional LMS Error Info: 406
setValue: cmi.interactions.4.correct_responses.0.pattern=2%3AFalse
Course could not set cmi.interactions.4.correct_responses.0.pattern to 2%3AFalse Error Info: Data Model Element Type Mismatch Additional LMS Error Info: 406

GMCQ (question text redacted):

setValue: cmi.interactions.0.learner_response=1%3ADepressive%20disorders
Course could not set cmi.interactions.0.learner_response to 1%3ADepressive%20disorders Error Info: Data Model Element Type Mismatch Additional LMS Error Info: 406
setValue: cmi.interactions.0.correct_responses.0.pattern=1%3ADepressive%20disorders[,]3%3AAnxiety%20disorders[,]4%3ASubstance%20use%20disorders
Course could not set cmi.interactions.0.correct_responses.0.pattern to (as above) Error Info: Data Model Element Type Mismatch Additional LMS Error Info: 406

Everything else lands fine — id, type, description (full question text, which is great), result and timestamp are all stored, and reattempts are journalled as separate records.

The cause looks like the values being URL-encoded, and carrying the full option text rather than an identifier (1%3ADepressive%20disorders being “1:Depressive disorders” encoded). For choice-type interactions the spec restricts these fields to short identifiers, so percent signs and colons aren’t valid. Moodle validates strictly and refuses them. The [,] delimiter between multiple responses looks correct, so it seems to be only the identifier content that’s the problem. SCORM Cloud appears to be lenient about this, which may be why it hasn’t come up before.

Practical effect: we can see that a learner got a question wrong, but not which option they picked. On a four-option MCQ that rules out any distractor analysis, which was the main reason we turned interaction tracking on.

A couple of questions:

  1. Has anyone else hit this, on Moodle or another spec-strict LMS?
  2. @Matt_Leathes — is emitting spec-conformant short identifiers here something that could be looked at for Legacy? Happy to send debug logs or a sample package, and we can test a fix on our LMS.

Also raised with support so there’s a ticket on record.

Thanks,
Shaun

2

Hi @ShaunP - this is actually an issue I am aware of with Evolve SCORM 2004 and Moodle. However, the issue actually lies with Moodle because it is checking the format for this value against the format that was specified in SCORM 1.2 - not the updated format that SCORM 2004 uses.

In the SCORM 2004 docs, the learner/correct response for a ‘choice’ interaction is defined as:

An ISO-10646-1 characterstring representing a set of short_identifier_types, where each element of the set is separated by a special reserved token (“[,]”). The LMS shall support characterstrings that include at least 36 (the required SPM) short_identifier_types, separated by the reserved “[,]” token.

And the short_identifier_type is defined as:

The short_identifier_type shall be a characterstring that conforms to the syntax defined for URI, refer to RFC 3986 […] The short_identifier_type value shall be implemented with an SPM of 250 characters.

So ‘the syntax defined for URI’ is why the string is URL-encoded. If we were to send an un-encoded answer string, that would also be rejected by Moodle (and every other SCORM 2004 LMS) due to the use of the colon and whitespace in values like ‘4:Substance abuse disorders’

As to why we don’t send an ‘identifier’ instead of the URL-encoded answer string - the only real way of doing this that would allow the answer to be reliably identified would be to use the unique item id - which would be something like 6a731b9d0a86f00087d092d1 - which is not really going to be very useful to our customers. Even sending something more friendly like ‘option1’ would still require you to have the course to hand in order to look up what ‘option1’ actually is.

We’ve tested Evolve’s interaction tracking extensively in SCORM Cloud and it validates fine there - see screenshot below of the debug logs from an example course I just created & tested that uses the same answers format as you have provided above.

More importantly: Moodle doesn’t actually support SCORM 2004 and states that ‘If you require a fully certified SCORM 2004 Player in Moodle, Rustici Software have a Moodle plugin which connects to their commercial SCORM Cloud service turning Moodle into a fully compliant SCORM 2004 LMS.’

Finally just to add that in the two years this functionality has been available, we’ve only ever had this issue reported with Moodle… although I suspect it might exist in Totara as well since that is based on Moodle. Interestingly, I vaguely recall them announcing that recently did decide to support SCORM 2004 - so I don’t know if reporting the bug to Totara might eventually get the fix pushed into the Moodle source code?

SCORM Cloud debug logs
SCORM Cloud debug logs2212×1038 251 KB

3

Thanks Matt, appreciate the detailed response — that clears it up.

You’re right that the encoding is valid URI syntax under the 2004 spec, and your SCORM Cloud validation shows the values passing ValidShortIdentifier checks, so this lands on Moodle’s validation being stricter than the spec rather than an Evolve issue. Good to have the mechanism confirmed either way, and useful context that it’s only ever surfaced on Moodle.

Given Moodle have stopped development on their 2004 support, we’ll treat it as a known limitation of the platform combination — everything else in the interaction tracking is working well for us (result, question text in the description field, timestamps, and reattempts journalling separately), so we still get solid item-level analysis from it.

Thanks again for taking the time to dig into it!

Shaun

4

@ShaunP I think the check it performs is fairly simple so if this is an instance of Moodle you control, removing/relaxing the check should be quite straightforward?

5

Thanks @Matt_Leathes — unfortunately not an option here. It’s a managed hosting setup so no core code access.
Happy to leave it as a known platform limitation — the rest of the interaction data is serving us well. Thanks again for the detail on this.