Computer Marked Maths now available

MSD Learning Technologies is now able to offer computer marked maths.

We are looking for departments with the University to make use of our project – for formative assessment at this stage. We particularly want to gauge enthusiasm and, ideally, to see examples of the questions which you’d find useful.

Highlights of the maths assessment system:

  1. interacts with Maxima which gives it full algebraic evaluation for simple or complex maths.
  2. allows use of random variables in questions
  3. can calculate and show worked answers
  4. can include control flow functions (if, while, etc, to modify or calculate variables)
  5. supports multiple tests on an answer (eg is it simplified? is it algebraically correct?)
  6. supports multi-part questions
  7. supports SI units
  8. supports questions requiring significant figures and decimal places

We have integrated with WebLearn, so you can offer assessments via your existing WebLearn sites. Assessments can also be provided as a standalone service using SSO if that is required.

The sponsorship of this project is through the IT Innovation Fund so we can offer this to any part of the University.

Maths with QTI – Explanation of Help Guide for Maths Entry Questions

Here’s the link to the Help Guide for Maths Entry Questions, if you’ve come here looking for that. This post explains why we’ve created that guide.

We’re starting to deliver Maths questions online using QTIWorks, and are putting together a help guide for users, as it is not always easy to know how to correctly represent mathematical expressions in a maths answer box.

The “Input Hints” provided in QTIWorks are of some help, but are perhaps not the most user friendly and could be improved by the addition of examples, and perhaps some reordering to place the simpler and more commonly used inputs nearer the top of the list.

QTIWorks Input Hints
QTIWorks Input Hints (click to view full size)

To supplement (/replace) this hints box, I’ve started to write a (hopefully) simple guide to using the maths input box, with some examples. This is a work in progress, so will evolve, but hopefully is a reasonable starting point. Any comments are very welcome.

Maths with QTI – Template Processing

Please note that this is a work in progress! I have added blocks that I just happen to have been using since I started writing this post, so they are not necessarily being added ‘simplest’ first. I will continue to add more as and when I use them.

After declaring all of the variables in an assessment item in a QTI XML file, the next step is to do the template processing. This is where variables are set or randomised, answers are calculated etc. There are too many possibilities for this section to get anywhere near specifying everything that is possible, and I won’t cover what can be done using Maxima in this post, but hopefully some of the blocks of code here can be used as they are, or will act as good starting points for doing other things.

templateProcessing Start and End

<templateProcessing>
   <!-- Template processing goes here -->
</templateProcessing>

Set Fixed values

Setting a Fixed Integer value

<setTemplateValue identifier="iA">
   <baseValue baseType="integer">2</baseValue>
</setTemplateValue>

Setting a Fixed Float value

<setTemplateValue identifier="fFloat">
   <baseValue baseType="float">3.14159</baseValue>
</setTemplateValue>

Set Randomised values

Set a positive Randomised Integer value from a range

<setTemplateValue identifier="iA">
   <randomInteger min="2" max="9"/>
</setTemplateValue>

Set a positive or negative Randomised Integer value from a range

<setTemplateValue identifier="iC">
   <product>
      <randomInteger min="1" max="10"/>
      <randomInteger min="-1" max="1" step="2"/>
   </product>
</setTemplateValue>

Note that the step attribute means that the second randomInteger will be selected only every other number in the specified range. In this case, that means it can only be -1 or 1, and not 0, so this defines whether the value is positive or negative.

Set an integer to a random value from within a range

<setTemplateValue identifier="iB">
   <random>
      <multiple>
         <baseValue baseType="integer">2</baseValue>
         <baseValue baseType="integer">3</baseValue>
         <baseValue baseType="integer">5</baseValue>
         <baseValue baseType="integer">8</baseValue>
         <baseValue baseType="integer">13</baseValue>
         <baseValue baseType="integer">21</baseValue>
      </multiple>
   </random>
</setTemplateValue>

Rounding

Round to a number of significant figures

<setTemplateValue identifier="fFloatRounded">
   <roundTo roundingMode="significantFigures" figures="3">
      <variable identifier="fFloat"/>
   </roundTo>
</setTemplateValue>

Round to a number of decimal places

<setTemplateValue identifier="fFloatRounded">
   <roundTo roundingMode="decimalPlaces" figures="2">
      <variable identifier="fFloat"/>
   </roundTo>
</setTemplateValue>

 

Constraints

Set a value to not equal a fixed value

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <variable identifier="iA"/>
         <baseValue baseType="integer">7</baseValue>
      </equal>
   </not>
</templateConstraint>

Ensure Greatest Common Denomiator of 2 values is 1 (i.e. the values share no common factors other than 1)

<templateConstraint>
   <equal toleranceMode="exact">
      <gcd>
         <variable identifier="iA"/>
         <variable identifier="iB"/>
      </gcd>
      <baseValue baseType="integer">1</baseValue>
   </equal>
</templateConstraint>

Ensure 2 values are not equal

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <variable identifier="iA"/>
         <variable identifier="iB"/>
      </equal>
   </not>
</templateConstraint>

Ensure one value is larger than another

<templateConstraint>
   <gt>
      <variable identifier="iB"/>
      <variable identifier="iC"/>
   </gt>
</templateConstraint>

Constraint to ensure a – (b + c) != 0, i.e. a != (b + c)

<templateConstraint>
   <not>
      <equal toleranceMode="exact">
         <subtract>
            <variable identifier="iA"/>
            <sum>
               <variable identifier="iB"/>
               <variable identifier="iC"/>
            </sum>
         </subtract>
         <baseValue baseType="integer">0</baseValue>
      </equal>
   </not>
</templateConstraint>

Conditions

Set iB to equal 6, 8 or 10 if iA < 5, otherwise iB equal to 2, 4 or 6

<templateCondition>
   <templateIf>
      <lt>
         <variable identifier="iA"/>
         <baseValue baseType="integer">5</baseValue>
      </lt>
      <setTemplateValue identifier="iB">
         <randomInteger min="6" max="10" step="2"/>
      </setTemplateValue>
   </templateIf>
   <templateElse>
      <setTemplateValue identifier="iB">
         <randomInteger min="2" max="6" step="2"/>
      </setTemplateValue>
   </templateElse>
</templateCondition>

Constants

<mathConstant name="pi"/>

Maxima Script Dummy Variable

I won’t cover what is possible with Maxima and hence what goes in this variable, but this is the code used for setting the dummy variable:

<setTemplateValue identifier="tDummy">
   <customOperator class="org.qtitools.mathassess.ScriptRule" ma:syntax="text/x-maxima">
      <baseValue baseType="string"><![CDATA[
         mAns : ev(iA*x^2iB,simp);
      ]]></baseValue>
   </customOperator>
</setTemplateValue>

Maths with QTI – Variable Declarations

Please note that this is a work in progress!

The first thing that you need to do in an assessment item, in a QTI XML file, is to declare your variables. These declarations come in 3 types – responseDeclarationoutcomeDeclaration and templateDeclaration.

responseDeclarations

This is where you declare variables used for responses and response processing. The following should be useful building blocks:

User Response

These are variables used to store the user’s response to a question.

<responseDeclaration identifier="RESPONSE_INTEGER" cardinality="single" baseType="integer"/><!-- integer response -->
<responseDeclaration identifier="RESPONSE_FLOAT" cardinality="single" baseType="float"/><!-- float response -->
<responseDeclaration identifier="RESPONSE_STRING" cardinality="single" baseType="string"/><!-- string response -->
<responseDeclaration identifier="RESPONSE_MATHS" cardinality="record"/><!-- maths response -->

printMath

This is a variable into which Presentation MathML form of a RESPONSE expression is copied.

<responseDeclaration identifier="printMath" cardinality="single" baseType="string"/>

Hint and Solution Request

These can be set to true when the hint or solution is requested, and should be tested at the start of the responseProcessing.

<responseDeclaration identifier="HINTREQUEST" cardinality="single" baseType="boolean"/>
<responseDeclaration identifier="SOLREQUEST" cardinality="single" baseType="boolean"/>

outcomeDeclarations

These are used for identifying and recording the outcomes from a user’s interactions, e.g. score and feedback.

Correct Response

These can be used to store the correct response to a question, for comparison with the user’s response.

<outcomeDeclaration identifier="iAnswer" cardinality="single" baseType="integer"/>
<outcomeDeclaration identifier="mAnswer" cardinality="record"/>

Score and Feedback

<outcomeDeclaration identifier="SCORE" cardinality="single" baseType="float" normalMaximum="2">
   <defaultValue>
      <value>0</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration identifier="FEEDBACK" cardinality="multiple" baseType="identifier"/><!-- multiple cardinality - container for multiple values. For feedback, this means we can show multiple feedback sections at once -->
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="EMPTY"/><!-- Empty value, for emptying FEEDBACK container -->

Hint and Solution Related

<outcomeDeclaration baseType="boolean" cardinality="single" identifier="seenSolution">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="boolean" cardinality="single" identifier="seenHint">
   <defaultValue>
      <value>false</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="ASKHINT">
   <defaultValue>
      <value>askhint</value>
   </defaultValue>
</outcomeDeclaration>
<outcomeDeclaration baseType="identifier" cardinality="single" identifier="ASKSOLUTION">
   <defaultValue>
      <value>asksolution</value>
   </defaultValue>
</outcomeDeclaration>

templateDeclarations

These are used for defining and randomising variables used in the question itself.

Template Variables

<templateDeclaration identifier="iInt" cardinality="single" baseType="integer" mathVariable="true"/><!-- Integer -->
<templateDeclaration identifier="fFloat" cardinality="single" baseType="float" mathVariable="true"/><!-- Float -->
<templateDeclaration identifier="sString" cardinality="single" baseType="string" mathVariable="true"/><!-- String -->
<templateDeclaration identifier="mX" cardinality="record" mathVariable="true"/><!-- Maths -->

Use mathVariable=”true” to enable the variable to be used in MathML expressions.

Maxima Script Dummy Variable

<templateDeclaration identifier="tDummy" cardinality="single" baseType="boolean"/>

Maths with QTI – QTI XML Structure and Building Blocks Resource

We have recently started using QTI, specifically Uniqurate and QTIWorks, to write maths questions for delivery to students. Uniqurate can be used to write basic questions, including maths questions with numerical answers and simple randomisation of values. However, to incorporate more complex randomisation, e.g. randomising variable letters, to allow mathematical expressions to be given as answers and to do various other more complex things with maths questions, it is necessary to write (or, preferably, modify someone else’s!) QTI XML. This can get quite complicated and I have found some aspects of it difficult to get stuck into. I find that I know that something is possible, but it can take some time to pin down exactly how to do it, and I usually copy code from other questions. (Many thanks to Dr Sue Milne from ELandWeb for providing extensive help, both by giving access to her questions and answering many QTI-related queries)

One thing that I keep wishing for is a resource containing reusable blocks of QTI code that could be used to build a question. I believe that such a resource does not currently exist, so I have decided to start to write one, the links to which can be found below.

At this point I should make the disclaimer that I am still early on in my QTI journey, so I am no expert, and can by no means guarantee that this resource will be correct, but I will try to improve and extend it as I get to know the QTI specification better. The posts based on the XML snippets that I have been using, and therefore they are far from being a complete resourse. However, I hope they are useful to anyone getting started with QTI. Please feel free to suggest additions and corrections.

  1. QTI XML Basic Structure
  2. Variable Declarations
  3. Template Processing
  4. Item Body
  5. Response Processing
  6. Solution and Hint
  7. Modifying assessment.xml (for Assessments created in Uniqurate)

Maths with QTI – Help Guide for Maths Entry Questions

This is intended as a simple guide to entering mathematical expressions as answers to questions delivered using QTIWorks. Any comments about its usefulness/helpfulness would be most welcome.

Note that you can also view an input hints box by clicking on the blue question mark symbol help_questionmark next to the maths input box. You can also watch a brief video showing you how to enter mathematical answers.

This guide contains the following sections:

The Maths Input box ^ Top

When a mathematical expression is required as an answer to a question, a text box within a larger grey box is shown. When text is typed into the text box, it is interpreted mathematically, and the resulting mathematical expression is shown:

mathsinputbox_mod

If you click on the blue question mark symbol help_questionmark, an input hints box will pop up.

Basic Operators (+ – * / =) ^ Top

The basic mathematical operators can be represented by words or symbols, as shown in the table below.

Operation Symbol Word
Addition +
Subtraction
Multiplication * times
Division / divide
Equals =

note that multiplication is implied between a coefficient and a variable, e.g. 2x = 2*x, and between consecutive variables, i.e. letters, unless the letters make up a special character, e.g. pa = p*a, but pi = π, not p*i.

Other Operators (± > ≥ < ≤ ≠) ^ Top

Other operatores, e.g. comparison operators, can be represented as shown in the table below.

Operation Input Interpretation
Plus or minus +- ±
Greater than > >
Greater than or equal to >=
Less than < <
Less than or equal to <=
Not equal !=

Precedence (order of operations) ^ Top

The order of precedence, i.e. the order in which operations are performed/interpreted, follows the usual rules:

  1. brackets
  2. roots and exponentials
  3. multiplication and division
  4. addition and subtraction
Input Interpretation Notes
2+3*4 precedence_nobrackets equals 14
2+(3*4) precedence_unnecessarybrackets equals 14, brackets are unnecessary
(2+3)*4 precedence_brackets equals 20, brackets are required

Special Symbols and Functions ^ Top

Generally, letters just represent variables, and multiplication between consecutive letters is implied (see Basic Operations above). However, there are certain letters or words that have special meaning, as follows:

Letter/Word Meaning
e e, the natural exponential, 2.71828…
i the imaginary number, √-1
alpha…lambda…pi…etc Greek letters, α, λ, π etc
log logarithm function
ln natural logarithm function
sin, cos, tan trigonometric functions
cap, cup, in, not, notin, subset, subseteq, to,
vee, wedge…and probably others
Logic and Set Theory operators

Fractions (a/b) ^ Top

Use the divide symbol,  ‘/’, between the numerator (top) denominator (bottom) of a fraction. Brackets may be needed to give the correct numerator and denominator.

Input Interpretation
1/2 half
2x+1/3x+2 fraction_no_brackets
(2x+1)/(3x+2) fraction_brackets

Brackets (a(b+c)) ^ Top

Brackets can be used as they normally would when writing an expression.

Input Interpretation Notes
sin(2x+30)  brackets_sin  brackets not needed for sin of single value, e.g. sinx gives sinx
5(x+1)(2x-3)  brackets_2
3(2x-(2-y))  brackets_nested equivalent to 3(2x-2+y)

Powers (x^a) ^ Top

Use ^ (hat/caret symbol) to represent powers:

Input Interpretation Notes
x^2 powers_xsquared    
x^(1/3) powers_xtothird fractional powers need brackets
x^-1 powers_xtominus1 negative powers do not need brackets
e^(x(2x+1)) powers_expbrackets
e^x(2x+1) powers_expnobrackets

Roots (sqrt(x) or x^(a/b)) ^ Top

Only a square root can be represented with the root symbol. All other roots must be represented as fractional powers:

Root Input Interpretation
Square root (√x) sqrtx or sqrt(x) roots_sqrtx
Cube root (∛x) x^(1/3) roots_cubertx

Subscripts (x_a) ^ Top

Use _ (underscore) to give a subscript (similar to using ^ for powers/superscripts)

Input Interpretation
x_1 subscript_x_1
y_a subscript_y_a

“Sorry, I could not make sense of your input” ^ Top

You will see this message in a number of situations (including, but not limited to):

  • When you have an operator (e.g. + – * / ^ _ =), that does not have a value/variable before and/or after it, e.g. ‘1+’, ‘*3’, ‘x/’, ‘y^’
  • When you have two consecutive operators, e.g. ‘+*’
  • When you have opened a bracket and not (yet) closed it, e.g. ‘2(x+3’

If you see this message when you believe you have entered a complete expression, make sure you check your expression carefully, in particular that you have closed all brackets.

Getting excited by maths handling in MathAssessEngine

Just been trying out some sample QTIv2.1 questions sent to me by Dr Sue Milne from ELandWeb Ltd which address some issues with maths in online assessment that we have been struggling with for some time:

  • How do we allow students to choose the units in which they answer a numeric question?
  • How do we allow students to answer an algebraic question with an algebraic answer?
  • How can we randomise the values presented whilst testing the same mathematical concept from student to student and from attempt to attempt (to allow students to retest themselves)?
The answer appears to be that what we need is MathAssessEngine from the University of Edinburgh and QTIv2.1 .
Answering questions with algebra
Answering a question with algebra in MathAssessEngine
We haven’t yet had time to look at this is any great detail but these two examples demonstrate that there is a whole new world of assessments out there waiting to be explored.
qtiv21_examples contains two examples:
Example 1  (SineRule-002-mathOpWithSol.xml): Demonstrates a question that ‘understands’ units and the impact they will have on the expected answer
Example 2 (mela012252.xml): Demonstrates answering a question with algebra – in this case ‘sqrt’.  As the participant types, the answer the computer will mark is displayed in MathML.
Both questions randomise the variables shown each time you retake the question. To give them a go yourself, unzip the file then visit MathAssessEngine. ‘Choose file’ under ‘Upload Assessment Items and Tests’ and click ‘Go’.