assessment – MSDLT Blog

May 31, 2012

Accessibility – Rogo vs Perception

When considering and comparing Rogo and Questionmark Perception, one thing we looked at was the accessibility options available in both. As would be expected both give consideration to accessibility requirements, but the approaches and options vary.

Questionmark Perception (V5)

Provides 3 text size and 2 contrast (normal or yellow on blue) options within delivery – these can be adjusted by the user while they are taking an assessment.
Coloured backgrounds, font size and colour and other style-related requirements can be set using templates.
Extra time can be set (i.e. default assessment time limit can be overwritten) when scheduling an assessment.
Accessibility conformance statements:

Rogo

In Rogo, accessibility settings are set for an individual, rather than for an assessment/schedule. Once specified, these settings are used for every assessment that the user sits. The following settings are available:

Extra Time (doesn’t actually do anything as assessments are not timed)
Font Size
Typeface
Background Colour
Foreground Colour
Marks Colour
Heading/Theme Colour
Labels Colour

User’s cannot adjust any settings for themselves within exam delivery, but this is unlikely to be a problem, as accessibility requirements are almost always known about prior to an assessment. Furthermore, they can easily be changed ‘on the fly’.

April 18, 2012April 18, 2012

A minor change to the assessment delivery page

For EMQs in Perception, we ‘manually’ (using our Question Maker) number the question stems with Roman numerals – (i), (ii) etc – and letter the answer options – A, B, C etc. However, in Rogo, the stems are automatically lettered by default. Therefore, having imported questions from Perception into Rogo, the stems had both letters and Roman numerals, e.g. A. (i) The first stem.

To fix this, the options were to change all of our questions when we imported them, or to remove the automatic stem lettering in Rogo. The former would be a prohibitively large amount of work, so thankfully the latter is merely a case of changing one CSS property in the paper/start.php file, line 370. The “list-style-type:upper-alpha” property of “.extmatch li” was changed to “list-style-type:none” to remove the item-marker. There are many possible values for this property – see http://www.w3.org/TR/CSS21/generate.html#list-style.

April 18, 2012May 1, 2012

Thoughts on Rogo Delivery

The Rogo delivery system seems to be robust and, unlike the admin interface, free from bugs. It also seems fast, and initial load-testing has shown it to be capable of putting up with a reasonable number of students starting or finishing an assessment at the same time, although we have not tested this in a live environment. This is despite us running the system on a single, reasonably-spec’d, but not especially new, server.

One handy feature in the Rogo delivery system, that Perception does not have, is the Fire Alarm button – see https://learntech.medsci.ox.ac.uk/wordpress-blog/?p=15.

However, there are a number of things that are missing, compared with Perception: Auto-save, timed assessments, secure browser.

Auto-save

Rogo is not able to auto-save assessments, e.g. every 5 minutes. It does save when a user moves to another screen (and when the fire alarm button is clicked) but since we prefer to run our exams as a single, scrollable screen containing all of the questions, this is not particularly helpful. However, since the functions are there for saving the current status of an exam while the exam is continuing, it would proably not be a big step to write an AJAX-based auto-save function.

Timed Assessments

Assessments in Rogo are never timed and so do not autosubmit when the time is up. The assessments we have run in Perception have traditionally been timed, so that each user’s time only begins when they enter the assessment page, and the assessment automatically submits when their time runs out. There is also a timer on the page, so the student can see exactly how much time they have left, as this would not be the same for all students. However, we have recently been leaning increasingly towards running untimed assessments, mainly due to the problems of disaster recovery with a timed assessment. For example, if a fire alarm were to go off, it would not be possible to pause the timer and allow students to carry on from where they left off. Therefore, the lack of timed assessments is unlikely to be a major issue, as we were heading in this direction anyway. Nonetheless, it would be good to have the option!

Secure Browser

Unlike Perception, Rogo does not have built in secure browser support. However, there are free secure browsers out there that can be used instead, e.g. Safe Exam Browser – http://safeexambrowser.org. We might cover this more fully at a later date when we’ve done more testing with it. Update: Now covered in this post