Maths with QTI – Response Processing

From my experience, the responseProcessing section of a QTI XML file is the most complicated. It is the place where you check whether the user has given the correct response, deal with incorrect responses, and display the appropriate feedback. In theory it could be very simple, if you are just checking whether an answer is correct, showing the ‘correct’ feedback if it is and showing ‘incorrect’ feedback if it is not. However, it is possible to give much more tailored feedback than this, pointing users in the right direction if they have made a common mistake.

Another common use of the responseProcessing is to show a hint and/or a solution if the user requests it. For a generic structure for doing this, please see the Solution and Hint post.

responseProcessing Start and End

<responseProcessing>
<!-- Response processing goes here -->
</responseProcessing>

responseConditions

General Structure

This is the general structure for if…else if…else statements for dealing with responses. responseCondition statements can be nested within each other to create complex hierarchies of conditions.

<responseCondition>
   <responseIf>
   </responseIf>
   <responseElseIf>
   </responseElseIf>
   <responseElse>
   </responseElse>
</responseCondition>

Simple Example

In this example, the RESPONSE is tested for being NULL, in which case a score of 0 is given, and no feedback shown, so the user can just have another go. Next, RESPONSE is tested for being equal to ANSWER (i.e. the response is correct), in which case the CORRECT feedback is shown and the score is set to 1. Otherwise, the INCORRECT feedback is shown and the score set to 0.

<responseCondition>
   <!-- Is the response NULL? -->
   <responseIf>
      <isNull>
         <variable identifier="RESPONSE"/>
      </isNull>
      <setOutcomeValue identifier="SCORE">
         <baseValue baseType="float">0.0</baseValue>
      </setOutcomeValue>
   </responseIf>
   <responseElseIf>
      <!-- Is the response Correct? -->
      <equal toleranceMode="exact">
         <variable identifier="iAnswer"/>
         <variable identifier="RESPONSE"/>
      </equal>
      <setOutcomeValue identifier="FEEDBACK">
         <multiple>
            <baseValue baseType="identifier">CORRECT</baseValue>
         </multiple>
      <setOutcomeValue identifier="SCORE">
         <baseValue baseType="float">1.0</baseValue>
      </setOutcomeValue>
   </responseElseIf>
   <responseElse>
      <!-- Otherwise the response must be incorrect -->
      <setOutcomeValue identifier="FEEDBACK">
         <multiple>
            <baseValue baseType="identifier">INCORRECT</baseValue>
         </multiple>
      </setOutcomeValue>
      <setOutcomeValue identifier="SCORE">
         <baseValue baseType="float">0.0</baseValue>
      </setOutcomeValue>
   </responseElse>
</setOutcomeValue>

Feedback

Empty the Feedback Variable

This empties the feedback ‘multiple’ variable, so any previously shown feedback (from previous response) is removed before we process the current response. The EMPTY variable must have been declared in the Variable Declarations.

<setOutcomeValue identifier="FEEDBACK">
   <multiple>
      <variable identifier="EMPTY"/>
   </multiple>
</setOutcomeValue>

Add Feedback to the Feedback Variable

This defines the feedback sections that will be shown to the user. Multiple feedback sections can be shown by adding multiple identifiers.

<setOutcomeValue identifier="FEEDBACK">
   <multiple>
      <baseValue baseType="identifier">INCORRECT</baseValue>
      <baseValue baseType="identifier">NOTSIMP</baseValue>
   </multiple>
</setOutcomeValue>

Append Feedback to the Feedback Variable

If some feedback has already been defined and you want to add further feedback, you can append another feedback identifier as shown.

<setOutcomeValue identifier="FEEDBACK">
   <multiple>
      <variable identifier="FEEDBACK"/>
      <baseValue baseType="identifier">SEEN-HINT</baseValue>
   </multiple>
</setOutcomeValue>

Comparisons

Equal

This simply compares whether two numerical values are equal. If ‘toleranceMode’ is set to exact, the values must be exactly equal. toleranceMode can also be set to absolute or relative, in which case the ‘tolerance’ must be specified. See the specification for more information.

<equal toleranceMode="exact">
   <variable identifier="iAnswer"/>
   <variable identifier="RESPONSE"/>
</equal>

EqualRounded

This compares whether two values are equal, to a specified level of rounding. The roundingMode can either be decimalPlaces or significantFigures.

<equalRounded roundingMode="decimalPlaces" figures="0">
   <variable identifier="RESPONSE"/>
   <variable identifier="iAnswer"/>
</equalRounded>

Substring contains

<substring caseSensitive="false">
   <baseValue baseType="string">+</baseValue>
   <variable identifier="printMath"/>
</substring>

isNull

This tests whether a value is null.

<isNull>
   <variable identifier="RESPONSE"/>
</isNull>

Custom Operator

Custom operators can be used to carry out more complicated tests, e.g. comparing mathematical responses using the MathAssess engine. The ‘action’ can take one of three values:

  • equal: algebraic equivalence (e.g. factorised and expanded forms are equivalent)
  • syntequal: syntactical equality (e.g. factorised and expanded forms are not syntactically equal)
  • code: allows custom comparison code to be provided

As an example of the difference between equal and syntequal, 1/2 and 2/4 are equal, but not syntequal. 2(x+1) and 2x+2 are equal, but not syntequal.

<customOperator class="org.qtitools.mathassess.CasCompare" ma:action="equal" ma:syntax="text/x-maxima">
   <variable identifier="mAnswer"/>
   <variable identifier="RESPONSE"/>
</customOperator>

The following example of a ‘code’ action tests whether the sum of the two response and answer is 0, i.e. the response is correct, but has the wrong sign.

<customOperator class="org.qtitools.mathassess.CasCompare" ma:action="code" ma:code="is(equal($1+$2,0));" ma:syntax="text/x-maxima">
   <variable identifier="RESPONSE"/>
   <variable identifier="mAnswer"/>
</customOperator>

Operators

And Or

As you would expected, placing two comparisons within an ‘and’ operator gives a true result only if both comparisons evaluate to true. Placing two comparisons within an ‘or’ gives a true result if either (or both) of the comparisons evaluate to true. For or, just use the following snippet and change the <and>/</and> tags to <or>/</or>

<and>
   <equal toleranceMode="exact">
      <variable identifier="iAnswer1"/>
      <variable identifier="RESPONSE1"/>
   </equal>
   <equal toleranceMode="exact">
      <variable identifier="iAnswer2"/>
      <variable identifier="RESPONSE2"/>
   </equal>
</and>

Not

The ‘not’ operators gives the ‘logical negative’ (i.e. opposite, e.g. true => false) of the subexpression. The following snippet will return true if the value of RESPONSE is not null.

<not>
   <isNull>
      <variable identifier="RESPONSE"/>
   </isNull>
</not>

 

Maths with QTI – Solution and Hint

Use the following chunks of code to add “Show Hint” and “Show Solution” buttons to the question, define the hint and solution that are shown and do all the necessary processing

Variable Declarations

Request variables set to true when user asks for Hint or Solution

<responseDeclaration identifier="HINTREQUEST" baseType="boolean" cardinality="single"/>
<responseDeclaration identifier="SOLREQUEST" baseType="boolean" cardinality="single"/>

Outcome variables for tracking whether the user has viewed the Hint or Solution

<outcomeDeclaration identifier="seenHint" baseType="boolean" cardinality="single">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration identifier="seenSolution" baseType="boolean" cardinality="single">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>

Outcome variables for tracking whether the user has viewed the Hint or Solution

<outcomeDeclaration identifier="ASKHINT" baseType="identifier" cardinality="single">
   <defaultValue>
      <value>askhint</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration identifier="ASKSOLUTION" baseType="identifier" cardinality="single">
   <defaultValue>
      <value>asksolution</value>
   </defaultValue>
</outcomeDeclaration>

ItemBody

Hint and Solution feedback blocks

<feedbackBlock identifier="SOLUTION" outcomeIdentifier="FEEDBACK" showHide="show"> Solution goes here </feedbackBlock>
<feedbackBlock identifier="HINT" outcomeIdentifier="FEEDBACK" showHide="show"> Hint goes here </feedbackBlock>

“Show Hint” and “Show Solution” buttons

<div class="">
   <feedbackBlock showHide="show" identifier="askhint" outcomeIdentifier="ASKHINT">
      <p>
         <endAttemptInteraction responseIdentifier="HINTREQUEST" title="Show Hint"/>
      </p>
   </feedbackBlock>
   <feedbackBlock showHide="show" identifier="asksolution" outcomeIdentifier="ASKSOLUTION">
      <p>
         <endAttemptInteraction responseIdentifier="SOLREQUEST" title="Show Solution"/>
      </p>
   </feedbackBlock>
</div>

responseProcessing

The code below can be used as the framework for the responseProcessing section. The responseProcessing expressions that, for example, idenitfy whether the question has been answered correctly should be inserted at the “***Response Processing goes here***” comment.

<responseProcessing>
   <responseCondition>
      <responseIf>
         <!-- If user has asked for the Hint -->
         <variable identifier="HINTREQUEST"/>
         <!-- Show the Hint -->
         <setOutcomeValue identifier="FEEDBACK">
            <multiple>
               <baseValue baseType="identifier">HINT</baseValue>
            </multiple>
         </setOutcomeValue>
         <!-- Set seenHint to true -->
         <setOutcomeValue identifier="seenHint">
            <baseValue baseType="boolean">true</baseValue>
         </setOutcomeValue>
      </responseIf>
      <responseElseIf>
         <!-- If user has asked for the Solution -->
         <variable identifier="SOLREQUEST"/>
         <!-- Show the Solution -->
         <setOutcomeValue identifier="FEEDBACK">
            <multiple>
               <baseValue baseType="identifier">SOLUTION</baseValue>
            </multiple>
         </setOutcomeValue>
         <setOutcomeValue identifier="seenSolution">
            <baseValue baseType="boolean">true</baseValue>
         </setOutcomeValue>
         <setOutcomeValue identifier="completionStatus">
            <baseValue baseType="identifier">completed</baseValue>
         </setOutcomeValue>
         <setOutcomeValue identifier="ASKHINT">
            <baseValue baseType="identifier">null</baseValue>
         </setOutcomeValue>
         <setOutcomeValue identifier="ASKSOLUTION">
            <baseValue baseType="identifier">null</baseValue>
         </setOutcomeValue>
      </responseElseIf>
      <responseElse> 
         <!-- ***Response Processing goes here*** -->
         <responseCondition>
            <responseIf>
               <!-- If the user has seen the solution, set the score to 0 -->
               <variable identifier="seenSolution"/>
               <setOutcomeValue identifier="FEEDBACK">
                  <multiple>
                     <variable identifier="FEEDBACK"/>
                     <baseValue baseType="identifier">SEEN-SOLUTION</baseValue>
                  </multiple>
               </setOutcomeValue>
               <setOutcomeValue identifier="SCORE">
                  <baseValue baseType="float">0.0</baseValue>
               </setOutcomeValue>
            </responseIf>
            <responseElseIf>
               <!-- If the user has seen the hint, divide the score by 2 -->
               <variable identifier="seenHint"/>
               <setOutcomeValue identifier="FEEDBACK">
                  <multiple>
                     <variable identifier="FEEDBACK"/>
                     <baseValue baseType="identifier">SEEN-HINT</baseValue>
                  </multiple>
               </setOutcomeValue>
               <setOutcomeValue identifier="SCORE">
                  <divide>
                     <variable identifier="SCORE"/>
                     <baseValue baseType="float">2.0</baseValue>
                  </divide>
               </setOutcomeValue>
            </responseElseIf>
         </responseCondition>
      </responseElse>
   </responseCondition>
</responseProcessing>

Maths with QTI – Modifying assessment.xml (for Assessments created in Uniqurate)

To date, I have only created assessments in Uniqurate, so the items in this post just refer to a couple of things I have needed to modify in the assessment.xml (which exists at the top level of the assessment zip file) file to get the assessments working properly.

itemSessionControl

Control whether feedback is shown when reviewing (showFeedback), and whether the user can view the solution (showSolution). The solution is defined by using setCorrectResponse in the templateProcessing. The itemSessionControl tag can be contained within testPart, assessmentSection or assessmentItemRef (individual assessment item) blocks.

<itemSessionControl showFeedback="true" allowReview="true" showSolution="true" />

Test Feedback

For assessments created in Uniqurate, the total (i.e max score) for an assessment seems to be calculated by summing the max scores for each of the individual questions in the assessment, even if it is set up so that questions are repeated. Therefore, as a very simple example, if an assessment has a single section containing a single question, but “Number of questions to choose from section” is set to 2 and “Repeat questions” is switched on, the total/max score for the assessment is set to 1, when it should in fact be 2. This can be fixed by editing the testFeedback section of the xml. Just change the value in the last set of span tags to adjust the maximum score:

<testFeedback identifier="TEST_FB" showHide="show" outcomeIdentifier="TEST_FEEDBACK" access="atEnd">
     <div>You have reached the end of the test. Your total score was <printedVariable format="%.1f" identifier="TEST_total"/> out of <span class="uqfeedbackvar total">2</span>. </div>
</testFeedback>

Note that this example is taken from the default Uniqurate output, you can of course define your own feedback, displaying any of the variables defined in the outcomeProcessing section of the assessment.xml file.

Maths with QTI – Explanation of Help Guide for Maths Entry Questions

Here’s the link to the Help Guide for Maths Entry Questions, if you’ve come here looking for that. This post explains why we’ve created that guide.

We’re starting to deliver Maths questions online using QTIWorks, and are putting together a help guide for users, as it is not always easy to know how to correctly represent mathematical expressions in a maths answer box.

The “Input Hints” provided in QTIWorks are of some help, but are perhaps not the most user friendly and could be improved by the addition of examples, and perhaps some reordering to place the simpler and more commonly used inputs nearer the top of the list.

QTIWorks Input Hints
QTIWorks Input Hints (click to view full size)

To supplement (/replace) this hints box, I’ve started to write a (hopefully) simple guide to using the maths input box, with some examples. This is a work in progress, so will evolve, but hopefully is a reasonable starting point. Any comments are very welcome.

Maths with QTI – Things I wish I’d known before I started

Writing maths questions in QTI has been a bit of a learning curve, and I took my (/everyone’s?) usual approach of diving straight in and trying things out, without spending too much time reading the specification. As a result, there have been some things that I didn’t really understand when I started, but have realised along the way, and really wish that I had  taken the time to find out before I started. As with pretty much all of my QTI-related posts, this is a work in progress and will probably be updated as and when more things that I should have known crop up!

The meaning of adaptive (for an assessmentItem)

I didn’t really look too much at the assessmentItem properties when creating new questions, I just copied an old question, changed the identifier and title, and added extra namespace definitions if I found I hadn’t defined one that I was using (e.g. MathML or MathAssess) when I tested the question. I didn’t really pay attention to the ‘adaptive’ property, and didn’t notice that on some questions it was true and some it was false.

It was not until I put tests together and tried them out that I found out that items for which adaptive was set to false only allowed a single attempt. Since these are formative questions, I wanted students to be able to have multiple attempts, see a hint, see the solution, etc, which means setting adaptive to true. Therefore, had to go back through lots of questions, change adaptive to true and then re-add them to my tests.

What setCorrectResponse does

The name of the setCorrectReponse class gives a pretty good hint as to what it does, but I hadn’t fully understood it. As well as setting the correct response for a response variable (explained below), it is also used for displaying the model answer when a student is reviewing a test. If the test is set up to allow solutions to be shown when reviewing (using <itemSessionControl showSolution="true" /> in the testPart in the assessment.xml file for the test), the user can then choose the show the model solution by clicking the “Show Solution” button. This replaces their answer in the answer box with the model solution. The can switch back to the answer they gave by clicking the “Hide Solution” button.

To summarise the setting/getting of a correct response in an asssessment item, there are two ways to assign a correct response to a response variable:

  1. When declaring the response variable:
    <responseDeclaration identifier="RESPONSE" cardinality="single" baseType="float" >
       <correctResponse>
          <value>5.2</value>
       </correctResponse>
    </responseDeclaration>
  2. Using setCorrectResponse in templateProcessing. This is often more useful as the correct response can be assigned after template variables have been set and/or calculated.
    <setCorrectResponse identifier="RESPONSE">
       <variable identifier="fAns"/>
    </setCorrectResponse>

It is then possible to get the correct response for a response variable, in the responseProcessing, as follows:

<correct identifier="RESPONSE"/>

This can be used for comparing a given response to the correct response:

<equalRounded roundingMode="decimalPlaces" figures="0">
   <correct identifier="RESPONSE"/>
   <variable identifier="RESPONSE"/>
</equalRounded>

 

Writing Tests in Uniqurate – Tips and Tricks

We are using the QTI specification to create maths questions that can be delivered online. In order to do this we have used Uniqurate, partly as a tool for writing questions (although the vast majority of questions have been ‘hand-coded’ in QTI XML, due to their complexity), but mainly as a tool for bringing questions together into  tests.

I thought I would share a few things that have made my life easier when using Uniqurate, in case they are useful to anyone else:

  • Use ‘Enter’ (carefully!) to confirm actions – Uniqurate often asks for confirmation, e.g. when deleting questions, using a pop up box. This can make deleting lots of questions an annoying process if you use the mouse to click the OK button in the confirm box. However, if you just click the delete button (rubbish bin) and then hit the enter key, this will confirm, so you can delete lots of questions quickly. Needless to say be very careful when doing this!
  • Go back to the main menu if it starts misbehaving – Uniqurate sometimes starts misbehaving, e.g. not adding questions properly. If this happens, just go back to the main menu, then click the “Edit current test” button to return to the edit screen.
  • Click on the Section name to edit it – it’s not immediately obvious, but you can edit section names by clicking on/selecting the current name of a section and then typing in the new name.
  • To move a question, click on white space, not the question name – You can drag and drop questions within a section (but unfortunately not from one section to another). To do this, you need to click on the white space to the right of the question name, not on the question name itself (doing so just selects the text).
  • “Add question to section” link only works when the mouse cursor is an arrow – Depending on your browser (I’m talking about IE10 on Windows), the pointer will be one of an arrow, a pointer (hand with pointed figure) or a text cursor when you hover over the “Add question to section” link, depending on which part of the link you are over. It only seems to actually work as a link when it is the arrow, so move around until you get an arrow before clicking!
  • Check the total score when you have repeated questions – see the post about modifying assessment.xml for more information

I will add any further things as and when they crop up.

Maths with QTI – Item Body

The itemBody section is where the question text, interactions, buttons and feedback are defined. Questions, solutions, etc will vary hugely depending on the question being asked, but I have included some standard sections that tend to be used regularly.

Question Text

Question text displaying an equation using variables defined using Maxima in templateProcessing:

<div class="QTEXT">
   <p>Solve the equation
      <m:math display="block">
         <m:mrow>
            <m:mi>mLHS1</m:mi>
            <m:mo>=</m:mo>
            <m:mi>mRHS1</m:mi>
         </m:mrow>
      </m:math>
      giving your answer as an integer or as a fraction in its lowest terms.
   </p>
</div>

Displaying Variables

Maths variable: <printedVariable identifier="mVar"/>
Integer: <printedVariable identifier="oQansVars" format="%.0f"/>
Number (to 2 decimal places – change the number before the f in the format attribute to change the number of decimal places): <printedVariable identifier="oQansVars" format="%.2f"/>

Interactions

Text/numerical entry interaction

<textEntryInteraction responseIdentifier="RESPONSE" expectedLength="2" />

Mathematical expression entry interaction

<customInteraction id="customInteraction1" ma:syntax="text/x-maxima" ma:printIdentifier="printMath" responseIdentifier="RESPONSE"/>

Show Hint button

<feedbackBlock showHide="show" identifier="askhint" outcomeIdentifier="ASKHINT">
   <p>
      <endAttemptInteraction responseIdentifier="HINTREQUEST" title="Show Hint"/>
   </p>
</feedbackBlock>

Show Solution button

<feedbackBlock showHide="show" identifier="asksolution" outcomeIdentifier="ASKSOLUTION">
   <p>
      <endAttemptInteraction responseIdentifier="SOLREQUEST" title="Show Solution"/>
   </p>
</feedbackBlock>

Feedback

Correct

<feedbackBlock outcomeIdentifier="FEEDBACK" identifier="RESPONSE-CORRECT" showHide="show">
   <div>Correct</div>
</feedbackBlock>

Incorrect

<feedbackBlock outcomeIdentifier="FEEDBACK" identifier="RESPONSE-INCORRECT" showHide="show">
   <div>Incorrect, try again. <br /><br /> </div>
</feedbackBlock>

No answer entered

<feedbackBlock outcomeIdentifier="FEEDBACK" identifier="RESPONSE-NULL" showHide="show">
   <div>Please enter an answer<br /><br /></div>
</feedbackBlock>

Hint

<feedbackBlock identifier="HINT" outcomeIdentifier="FEEDBACK" showHide="show">
   <div>Solution goes here<br /><br /></div>
</feedbackBlock>

Solution

<feedbackBlock identifier="SOLUTION" outcomeIdentifier="FEEDBACK" showHide="show">
   <div>Solution goes here<br /><br /></div>
</feedbackBlock>

MathML

I won’t cover MathML in any detail here, but the University of Edinburgh’s LaTeX to MathML converter is a very useful tool to help with this. This section contains a few tips for achieving some things that aren’t immediately obvious in MathML.

Greek Letters in MathML

These must be represented by the HTML decimal (e.g. &#913;) or hex codes (e.g. &#x391;), not the HTML entity codes (e.g. &Alpha;). Here’s a full list of codes.

Operators in MathML

Times symbol: <m:mo>&#xd7;</m:mo>
Space: <m:mi>&#160;</m:mi>

Maths with QTI – Template Processing

Please note that this is a work in progress! I have added blocks that I just happen to have been using since I started writing this post, so they are not necessarily being added ‘simplest’ first. I will continue to add more as and when I use them.

After declaring all of the variables in an assessment item in a QTI XML file, the next step is to do the template processing. This is where variables are set or randomised, answers are calculated etc. There are too many possibilities for this section to get anywhere near specifying everything that is possible, and I won’t cover what can be done using Maxima in this post, but hopefully some of the blocks of code here can be used as they are, or will act as good starting points for doing other things.

templateProcessing Start and End

<templateProcessing>
   <!-- Template processing goes here -->
</templateProcessing>

Set Fixed values

Setting a Fixed Integer value

<setTemplateValue identifier="iA">
   <baseValue baseType="integer">2</baseValue>
</setTemplateValue>

Setting a Fixed Float value

<setTemplateValue identifier="fFloat">
   <baseValue baseType="float">3.14159</baseValue>
</setTemplateValue>

Set Randomised values

Set a positive Randomised Integer value from a range

<setTemplateValue identifier="iA">
   <randomInteger min="2" max="9"/>
</setTemplateValue>

Set a positive or negative Randomised Integer value from a range

<setTemplateValue identifier="iC">
   <product>
      <randomInteger min="1" max="10"/>
      <randomInteger min="-1" max="1" step="2"/>
   </product>
</setTemplateValue>

Note that the step attribute means that the second randomInteger will be selected only every other number in the specified range. In this case, that means it can only be -1 or 1, and not 0, so this defines whether the value is positive or negative.

Set an integer to a random value from within a range

<setTemplateValue identifier="iB">
   <random>
      <multiple>
         <baseValue baseType="integer">2</baseValue>
         <baseValue baseType="integer">3</baseValue>
         <baseValue baseType="integer">5</baseValue>
         <baseValue baseType="integer">8</baseValue>
         <baseValue baseType="integer">13</baseValue>
         <baseValue baseType="integer">21</baseValue>
      </multiple>
   </random>
</setTemplateValue>

Rounding

Round to a number of significant figures

<setTemplateValue identifier="fFloatRounded">
   <roundTo roundingMode="significantFigures" figures="3">
      <variable identifier="fFloat"/>
   </roundTo>
</setTemplateValue>

Round to a number of decimal places

<setTemplateValue identifier="fFloatRounded">
   <roundTo roundingMode="decimalPlaces" figures="2">
      <variable identifier="fFloat"/>
   </roundTo>
</setTemplateValue>

 

Constraints

Set a value to not equal a fixed value

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <variable identifier="iA"/>
         <baseValue baseType="integer">7</baseValue>
      </equal>
   </not>
</templateConstraint>

Ensure Greatest Common Denomiator of 2 values is 1 (i.e. the values share no common factors other than 1)

<templateConstraint>
   <equal toleranceMode="exact">
      <gcd>
         <variable identifier="iA"/>
         <variable identifier="iB"/>
      </gcd>
      <baseValue baseType="integer">1</baseValue>
   </equal>
</templateConstraint>

Ensure 2 values are not equal

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <variable identifier="iA"/>
         <variable identifier="iB"/>
      </equal>
   </not>
</templateConstraint>

Ensure one value is larger than another

<templateConstraint>
   <gt>
      <variable identifier="iB"/>
      <variable identifier="iC"/>
   </gt>
</templateConstraint>

Constraint to ensure a – (b + c) != 0, i.e. a != (b + c)

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <subtract>
            <variable identifier="iA"/>
            <sum>
               <variable identifier="iB"/>
               <variable identifier="iC"/>
            </sum>
         </subtract>
         <baseValue baseType="integer">0</baseValue>
      </equal>
   </not>
</templateConstraint>

Conditions

Set iB to equal 6, 8 or 10 if iA < 5, otherwise iB equal to 2, 4 or 6

<templateCondition>
   <templateIf>
      <lt>
         <variable identifier="iA"/>
         <baseValue baseType="integer">5</baseValue>
      </lt>
      <setTemplateValue identifier="iB">
         <randomInteger min="6" max="10" step="2"/>
      </setTemplateValue>
   </templateIf>
   <templateElse>
      <setTemplateValue identifier="iB">
         <randomInteger min="2" max="6" step="2"/>
      </setTemplateValue>
   </templateElse>
</templateCondition>

Constants

<mathConstant name="pi"/>

Maxima Script Dummy Variable

I won’t cover what is possible with Maxima and hence what goes in this variable, but this is the code used for setting the dummy variable:

<setTemplateValue identifier="tDummy">
   <customOperator class="org.qtitools.mathassess.ScriptRule" ma:syntax="text/x-maxima">
      <baseValue baseType="string"><![CDATA[
         mAns : ev(iA*x^2iB,simp);
      ]]></baseValue>
   </customOperator>
</setTemplateValue>

Maths with QTI – Issues

Please note that this is a work in progress, and we will add issues as we find them, or remove them as they are fixed (or we find out that they aren’t really issues).

As discussed in other posts, we’re using QTI (Question and Test Interoperability) to create and deliver maths questions. The majority of the questions are being written by modifying (directly in the XML) questions from Dr Sue Milne’s collection. Uniqurate is being used to put questions together into tests, and QTIWorks is being used to deliver the questions.

We have found all the parts of the system – Uniqurate, QTIWorks and the QTI specification in general – very useful, usable and flexible. However, as is inevitable with such a complex and extensive specification and combination of creation and delivery systems, we have come across some issues. None of these are major problems that we cannot work with or around, but I thought I would continue my recent theme of writing “work in progress” blogs, and write one containing all of the issues that we have with each part of the system/process.

QTI (inc. Maxima, MathAssess engine)

  • Rounding floats doesn’t seem to work properly. After rounding, there seems to often be an arbitrary (small) value added to or subtracted from the rounded value, giving a value to >10 decimal places. For example, rounding 41.28947 to 2 decimal places, which should give 41.29, might give 41.29000014723.
  • There can be issues with recognising 2 equally valid correct answers as such. For example, if 1/x^3 is the correct answer to a question, x^-3 might be considered not fully simplified, even though it is at least as simple and arguably simpler. Another example would be requiring -(3x^2)/y, when (-3x^2)/y or -3x^2/y, or even -3x^2y^-1 would be equally correct. It might be possible to fix this within the XML, but would ideally be built into the Maths comparison engine.

Uniqurate

  • Questions are sometimes added to the wrong section of a test, or are not added at all (the latter seems to happen after a question has just been deleted from the test section). Generally returning to the Main Menu and then editing the test again gets rid of the issue temporarily.
  • It would be great to be able to add multiple questions to a test section in one go, rather than having to add them one at a time.
  • It is possible (and not that difficult) to create invalid combinations of of the “Number of questions to choose from section” and “Show questions only once/Repeat Questions” settings. If you tell it to show more questions that you have added, but leave it set to use each question only once, the test is invalid. It would be good if Uniqurate stopped you from doing this. To me, it seems as though there is significant overlap between these settings, at least when you tell it to show more question instances that there are questions in the section.
  • Can only specify the total number of questions in a section, not the number of each question.
  • The “Add question to section” link only works when the mouse cursor is an arrow. Depending on the browser (I’m talking about IE10 on Windows), the pointer will be one of an arrow, a pointer (hand with pointed figure) or a text cursor when you hover over the “Add question to section” link, depending on which part of the link you are over. It only seems to actually work as a link when it is the arrow.
  • Related to the above point, if you choose to show more questions than there are in the section, and hence set it to repeat questions, you are not guaranteed one of each question. E.g. if you have 2 question, and choose to show 3 questions, you could get 3 of one question and none of the other.
  • IDs given to sections are random strings, which makes them hard to identify when looking at results in QTIWorks. It would be good to be able to specify the ID for a section.
  • If questions are repeated in a section, the total score for the test is calculated as if each question was to be shown only once.

QTIWorks

  • The maths input help box could do with some improvements – see my post about a Help Guide for Maths Entry Questions.
  • Would like to be able to display multiple questions on a single page, rather than one question per page. This would be possible to set up by creating multiple questions in a single XML file, but it would be much nicer to be able to do it through the delivery settings.
  • Right clicking on a MathML block, then going to Settings > Math Renderer > MathML sometimes shows a simplified version of what is displayed, which for some questions (e.g. division of fractions) might actually be the answer. While it is unlikely someone would realise and do this, and it is only for certain questions where it would give the answer, it is something that could be looked at.
  • When a single part question is answered, it shows as “Answered” (in blue). However, when a multi-part question is answered, it is shown as “Finished” (in grey).
  • Similarly, if you answer a question (so it is marked as “Answered” and in blue), but then look at the solution, it is then marked as finished. This makes sense if the answer given was incorrect, but if the answer was correct and then the student looked at the solution, it would be good if the status remained as “Answered”.

Fixed/Non Issues

  • Would like scores, feedback and/or solutions to be shown when reviewing a test – Feedback and model solutions can be shown when reviewing by adding the following to the testPart section in the assessment.xml file for a test:
    <itemSessionControl showFeedback="true" showSolution="true" allowReview="true" />

    showFeedback means that the last feedback that the user was shown when attempting the question is shown when they review it. showSolution provides a “Show Solution” button, which puts the correct response (as defined in the response declaration, or by using setCorrectResponse in templateProcessing) in the answer box. However, it seems that if a maths variable is set as the correct response to a question, this will not  be displayed properly as the solution when reviewing.

Maths with QTI – Variable Declarations

Please note that this is a work in progress!

The first thing that you need to do in an assessment item, in a QTI XML file, is to declare your variables. These declarations come in 3 types – responseDeclarationoutcomeDeclaration and templateDeclaration.

responseDeclarations

This is where you declare variables used for responses and response processing. The following should be useful building blocks:

User Response

These are variables used to store the user’s response to a question.

<responseDeclaration identifier="RESPONSE_INTEGER" cardinality="single" baseType="integer"/><!-- integer response -->
<responseDeclaration identifier="RESPONSE_FLOAT" cardinality="single" baseType="float"/><!-- float response -->
<responseDeclaration identifier="RESPONSE_STRING" cardinality="single" baseType="string"/><!-- string response -->
<responseDeclaration identifier="RESPONSE_MATHS" cardinality="record"/><!-- maths response -->

printMath

This is a variable into which Presentation MathML form of a RESPONSE expression is copied.

<responseDeclaration identifier="printMath" cardinality="single" baseType="string"/>

Hint and Solution Request

These can be set to true when the hint or solution is requested, and should be tested at the start of the responseProcessing.

<responseDeclaration identifier="HINTREQUEST" cardinality="single" baseType="boolean"/>
<responseDeclaration identifier="SOLREQUEST" cardinality="single" baseType="boolean"/>

outcomeDeclarations

These are used for identifying and recording the outcomes from a user’s interactions, e.g. score and feedback.

Correct Response

These can be used to store the correct response to a question, for comparison with the user’s response.

<outcomeDeclaration identifier="iAnswer" cardinality="single" baseType="integer"/>
<outcomeDeclaration identifier="mAnswer" cardinality="record"/>

Score and Feedback

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="float" normalMaximum="2">
   <defaultValue>
      <value>0</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration identifier="FEEDBACK" cardinality="multiple" baseType="identifier"/><!-- multiple cardinality - container for multiple values. For feedback, this means we can show multiple feedback sections at once -->
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="EMPTY"/><!-- Empty value, for emptying FEEDBACK container -->

Hint and Solution Related

<outcomeDeclaration baseType="boolean" cardinality="single" identifier="seenSolution">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="boolean" cardinality="single" identifier="seenHint">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="ASKHINT">
   <defaultValue>
      <value>askhint</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="ASKSOLUTION">
   <defaultValue>
      <value>asksolution</value>
   </defaultValue>
</outcomeDeclaration>

templateDeclarations

These are used for defining and randomising variables used in the question itself.

Template Variables

<templateDeclaration identifier="iInt" cardinality="single" baseType="integer" mathVariable="true"/><!-- Integer -->
<templateDeclaration identifier="fFloat" cardinality="single" baseType="float" mathVariable="true"/><!-- Float -->
<templateDeclaration identifier="sString" cardinality="single" baseType="string" mathVariable="true"/><!-- String -->
<templateDeclaration identifier="mX" cardinality="record" mathVariable="true"/><!-- Maths -->

Use mathVariable=”true” to enable the variable to be used in MathML expressions.

Maxima Script Dummy Variable

<templateDeclaration identifier="tDummy" cardinality="single" baseType="boolean"/>