A minor change to the assessment delivery page

For EMQs in Perception, we ‘manually’ (using our Question Maker) number the question stems with Roman numerals – (i), (ii) etc – and letter the answer options – A, B, C etc. However, in Rogo, the stems are automatically lettered by default. Therefore, having imported questions from Perception into Rogo, the stems had both letters and Roman numerals, e.g. A. (i) The first stem.

To fix this, the options were to change all of our questions when we imported them, or to remove the automatic stem lettering in Rogo. The former would be a prohibitively large amount of work, so thankfully the latter is merely a case of changing one CSS property in the paper/start.php file, line 370. The “list-style-type:upper-alpha” property of “.extmatch li” was changed to “list-style-type:none” to remove the item-marker. There are many possible values for this property – see http://www.w3.org/TR/CSS21/generate.html#list-style.

Thoughts on Rogo Delivery

The Rogo delivery system seems to be robust and, unlike the admin interface, free from bugs. It also seems fast, and initial load-testing has shown it to be capable of putting up with a reasonable number of students starting or finishing an assessment at the same time, although we have not tested this in a live environment. This is despite us running the system on a single, reasonably-spec’d, but not especially new, server.

One handy feature in the Rogo delivery system, that Perception does not have, is the Fire Alarm button – see https://learntech.medsci.ox.ac.uk/wordpress-blog/?p=15.

However, there are a number of things that are missing, compared with Perception: Auto-save, timed assessments, secure browser.

Auto-save

Rogo is not able to auto-save assessments, e.g. every 5 minutes. It does save when a user moves to another screen (and when the fire alarm button is clicked) but since we prefer to run our exams as a single, scrollable screen containing all of the questions, this is not particularly helpful. However, since the functions are there for saving the current status of an exam while the exam is continuing, it would proably not be a big step to write an AJAX-based auto-save function.

Timed Assessments

Assessments in Rogo are never timed and so do not autosubmit when the time is up. The assessments we have run in Perception have traditionally been timed, so that each user’s time only begins when they enter the assessment page, and the assessment automatically submits when their time runs out. There is also a timer on the page, so the student can see exactly how much time they have left, as this would not be the same for all students. However, we have recently been leaning increasingly towards running untimed assessments, mainly due to the problems of disaster recovery with a timed assessment. For example, if a fire alarm were to go off, it would not be possible to pause the timer and allow students to carry on from where they left off. Therefore, the lack of timed assessments is unlikely to be a major issue, as we were heading in this direction anyway. Nonetheless, it would be good to have the option!

Secure Browser

Unlike Perception, Rogo does not have built in secure browser support. However, there are free secure browsers out there that can be used instead, e.g. Safe Exam Browser – http://safeexambrowser.org. We might cover this more fully at a later date when we’ve done more testing with it. Update: Now covered in this post

Importing Multiple Users in Rogo

Rogo enables the import of multiple users from a CSV file.

The are a number of required fields, although the order of the fields is flexible. The import does not worry about the case of the field headings, and can accept alternatives for each, as shown below:

  • Student ID: student_id, id
  • Forename(s): first names, forenames
  • Surname: surname, family name
  • Title: title
  • Course Code: course code, course
  • Year of Study: year of study, year of course (1, 2, 3 etc, not actual year, which is held in session)
  • Email Address: email, local email

It is also possible to specify a username for each student. If no username is given, the system will take everything before the @ in the email as the student’s username. A Module and Session (e.g. 2011/12) can also be specified – both are needed for this to be added. Module needs to be the module ID, e.g. OrgBod.

The Student ID does not refer to the student’s username or their database ID (an incremental digit), but a student ID held in a separate table in the database, which does not have to be unique.

Importing a user with the same username as a user that is already in the database will result in that user being overwritten.

There is no way of specifying a password for a user. All passwords are generated randomly and then hashed. This means there is also no way of finding out a user’s password. The only way to define a password for a user is to click “Reset” from the User File, which sends the user an email that they can then click on to set their password. This makes sense from a security point of view, but it is important for us to be able to see user’s passwords (something that is possible in Perception), so that we can log them in if they have forgotten their password, and can test multiple users with defined passwords.

In order to enable defining of passwords in user import, the following was added to import_users.inc:

Line 117:

    //JHM 2012-04-13: Enable passwords to be imported by csv
if (isset($header['password']) && $fields[$header['password']] != "") {	//Check that we have a password header, and the password is not blank for this user
    $password = $fields[$header['password']];
} else {
    $password = PasswordUtils::gen_password();	//No password was provided, so generate a random one
}

Line 137, comment out:

//$password = PasswordUtils::gen_password();	//JHM 2012-04-13: Password now obtained from csv or generated above (line 118)

From looking in the database, this was changing the value of the password for a user, but still did not allow the user to login using the given password. It emerged that this was because the passwords were being doubly encrypted, first in import_users.inc and then in UserUtils::createUser (userutils.class.php line 50). Therefore, I changed lines 137-139 of import_users.inc to:

//$password = PasswordUtils::gen_password();	//JHM 2012-04-13: Password now obtained from csv or generated above (line 118)
//$encpw_password = PasswordUtils::encpw($username, $password);	//JHM 2012-04-13: Password encrypted in createUser below
$encpw_password = $password;	//JHM 2012-04-13: Password encrypted in createUser below, so use plain password

This then worked fine for new user imports, but not for importing users over the top of old users. This required changes to lines 153 onwards in import_users.inc:

$encpw_password = PasswordUtils::encpw($username, $password);	//JHM 2012-04-13: Encrypt password for updating

//JHM 2012-04-13: Added password - 'ssssss' refers to var types, so add extra s to get it to work
$result = $mysqli->prepare("UPDATE users SET yearofstudy=?, title=?, first_names=?, surname=?, grade=?, password=? WHERE username=?");
$result->bind_param('sssssss', $year, $title, $forname, $surname, $course, $encpw_password, $username);

This now seems to work. This issue has been submitted to the Trac system – ticket #701, https://suivarro.nottingham.ac.uk/trac/rogo/ticket/701

We have also made a request for the import to flag up potential problems with header names, and/or enabling users to tell Rogo which headers it should expect, so it can report back if it failed to find any of these. Trac ticket #700 – https://suivarro.nottingham.ac.uk/trac/rogo/ticket/700

A few things that Rogo does better than Perception

Having been tied to Perception for many years, it is refreshing to be able to test out Rogo, which, although still in it’s infancy, has the potential to be a viable alternative. While we have come across some issues, there are also a good number of things that it does better than Perception (and I expect there are plenty more to come):

  1. Quicker, simpler, less clunky, more intuitive – Rogo just feels like a nicer, less 90’s system to work with
  2. Version history for questions – it will be extremely useful to be able to look back at how a question has changed and developed. In Perception, we use a complex system of question naming to keep track of major versions of a question, but this is far from ideal.
  3. Drag and drop question ordering
  4. Easy duplication of exams, with options to copy the questions or just link to the same set of questions
  5. Fire Alarm/Exit button in exam mode – students can click this button to if there is a fire alarm or other problem during an exam and it basically just shows a blank screen, so that students can be stopped and started together, without anyone seeing anyone else’s screen while they leave the room, or benefiting from seeing the exam for an extra amount of time. Assessments are not timed, and do not auto-submit, so this is not used to stop the timer.
  6. Flexible feedback settings

Perhaps the most important benefit of Rogo though is that it is OSS (Open Source Software) and written in PHP, which is where our skills primarily lie. Therefore, when there are things that don’t work as we would like, we can change them. Whereas with Perception, we generally just have to live with them and pay for the pleasure!

Experimenting with Rogo

My usual role in working with assessment software is inputting questions and creating assessments, so my thoughts are going to reflect this.

Obviously, with any new piece of software there’s a few howls of “Why won’t it do this???” “How on earth do I do that?”  I just have to say to myself … this too will pass.  And Rogo seems to be – so far – a very intuitive system.

What I’m more concerned about is things that will slow my workflow, and I’ve hit a couple – one which resolved itself, and one which hasn’t yet …

1) The automatic numbering system works against the one that we have established over the last five years.

Rogo has the pattern of numbering question stems A, B, C, D etc and numbering options 1, 2, 3, 4 etc.

Our existing system numbers stems (i), (ii), (iii) etc and options A, B, C, D etc. Numerals are used on diagrams (as in “What is the shaded area at 3?”)

I’m not saying that our system is perfect but … it works. And more importantly, we have upwards of five thousand questions configured in that way.

It seems that we CAN turn off the numbers on options, but we haven’t yet found out if we can take off letters on stems.

2) In the past, to support students who need to take the exam in a paper-based form, and as a back-up in case the system goes down, we have evolved a template to do this.  It’s a one click, load the template process; it automatically adds page breaks and replaces dropdown lists with paper-friendly boxes.  Are we going to be able to do this in Rogo?  Or will I go back to the pre-template system of making the changes to the html manually in Dreamweaver?

A third problem resolved itself beautifully … I regularly create a crib sheet with the correct answers for examiners to use to check the paper.  In Perception, you need something in the answer boxes before it will share the answers with you (because of the way we have set up the system.  I therefore have a little tool in Firefox that automatically fills in forms, and I was most disappointed that it didn’t work in Rogo – until Jon showed me that simply submitting the paper gave me a full set of answers – which is MUCH easier.

I should add that by the time 8it reaches the students, they will not be in this happy position …

 

Rogo Install – directory permissions

For Rogo install, we had to give the Apache user permissions to write to various directories (https://suivarro.nottingham.ac.uk/trac/rogo/wiki/Permission):

  • /help/staff/images
  • /help/student/images
  • /media
  • /temp
  • /qti/imports (not in Wiki at the time)
  • /qti/exports (not in Wiki at the time)
  • /config (only for install to allow config.inc.php to be written. Not on Wiki at the time)

To find out the user/user ID and group/group ID for the Apache user, I did the following:

  • In Putty, “ps aux | grep apache” to find out username of Apache user (wwwrun)
  • Looked in etc/passwd to find out user ID (30) and group ID (8) – wwwrun:x:30:8:WWW daemon apache:/var/lib/wwwrun:/bin/false
  • Checked group/group ID in etc/group
  • In WinSCP, changed permissions, giving the above user/group info: user = wwwrun [30], group = www [8]. Gave this user permission over the whole help directory.

Rogo install problems

On trying to install Rogo 4.2 for the first time, we came across the following issues.

On going to /install/index.php, all we saw was two errors, 103 and 104, with no explanation. On looking in the code we saw that these were caused by the /qti/imports and /qti/exports directories not being writeable by the Apache user. These weren’t mentioned in the File System Permissions page on the wiki (https://suivarro.nottingham.ac.uk/trac/rogo/wiki/Permission), but are now.

Having made these writeable, the install page then displayed input boxes, but without any labels. We realised this (and the lack of text for the errors above) was probably caused by the language file not being loaded properly. After a bit of debugging we discovered that in line 53 of classes/lang.class.php, the str_replace function was replacing the slashes in $_SERVER[‘PHP_SELF’], as $cfg_root_path is just “/” (as it would be for any installation in the webroot directory). I would guess the purpose of this str_replace is to remove the top level directory from $_SERVER[‘PHP_SELF’] for installations that are in a subdirectory of the webroot? Removing the str_replace function works for a webroot installation, but I’ve suggested a more complete solution below (though the if condition might need to be more rigorous). Another thing to note is that $cfg_web_root already has a “/” after it (as defined in install/index.php, line 45), and another slash is added in classes/lang.class.php, line 53 – I’ve changed that too below

Modified code (classes/lang.class.php, line 53):

if($cfg_root_path !== "/") {
     $self_path = str_replace($cfg_root_path, '', $_SERVER['PHP_SELF']);
}
else {
     $self_path = $_SERVER['PHP_SELF'];
}
$lang_path = $cfg_web_root . "lang/$language" . $self_path;

Once this was fixed, it was possible to complete the form with the necessary data for installation. However, the install still failed as it was not able to write the /config/config.inc.php file, as the /config file was not writeable by the Apache user. Having changed this (and deleted the database tables which had already been created), the install was attempted again and worked successfully.

Added to Trac, Ticket #680 – https://suivarro.nottingham.ac.uk/trac/rogo/ticket/680