Copyright © 2008 MELS - Multimedia & E-Learning Services, University of Zuerich, Switzerland
Table of Contents
List of Tables
This documentation is intended for OLAT developers. It explains the test concept and framework used for testing the OLAT system both in standalone and clustered mode.
Here's a quick guide on how to get started with Selenium IDE
Get Selenium IDE from here
Install Selenium IDE as plugin in Firefox
Restart Firefox
Start Selenium IDE: under Tools there's now a new entry called Selenium IDE: opens a new window
As a result you should get an empty Selenium IDE window. Notice that there is a list of test cases (with one entry 'Untitled') in this case) to the left of the table. You can make it appear by moving the resize indicator in the left.
Here's what it should look like

An empty Selenium IDE
We'll go into details about how the Olat Map file works later. Here's how to configure it in Selenium IDE
Go to Options>Options...

Setting for the Olat Map file in Selenium IDE
Configure the user-extensions.js file with one of the following paths:
Copy the nightly generated user-extensions.js from http://nightly.olat.uzh.ch/nightly/raw/doc/user-extensions.js to a local file and configure that local file. Make sure to repeat this often to catch changes in the olat-ui-map!!!
if you're doing modifications on the map file (see later), build it yourself and configure it, e.g.: C:\eclipse\workspace\seleniumtests\build\classes\user-extensions.js
Enable 'Remmber base URL'
Enable 'Record assertTitle automatically'
Enable 'Record absolute URL'
Enable 'Enable UI-Element' (important!)
Restart Selenium IDE (not the whole browser, just Selenium IDE. Activates the UI-Elements)
This section walks through recording the first test case with Selenium IDE
Make sure the Record button in the right end of the toolbar is switched on
Open https://nightly.olat.unizh.ch/nightly/dmz/
Enter 'administrator' and the corresponding password and click 'Login'
This should result in something like this:

The first test case in Selenium IDE
To rerun the test, click on one of the green Play buttons
A few things Gotchas here
Currently, Selenium IDE sometimes has refresh problems and you don't see the recorded steps until you move the mouse into the command table
Selenium sometimes has troubles catching keyboard input when Firefox does form-autocompletion.
Sometimes Selenium records 'click' instead of 'clickAndWait' and rerunning fails because of this.
Selenium-Recording is not always 100% automated - in fact there is quite a bit of manual effort involved and you can do it all manually in Selenium IDE as well. For example: you can easily type 'clickAndWait' in the command field and enter the target link in the target field yourself - instead of relying on the record mechanism
You can set Breakpoints in Selenium IDE by right clicking on any line and selecting 'Toggle Breakpoint' [B]
You can run single Selenium commands by double-clicking on a line
Given that you are on the right page, you can let Selenium visualize a UI-Element by clicking on the 'Find' button. It will let the element blink green for a while
Make sure to play with the basics of Selenium such as Running a test, Breakpointing, Stepping, Finding a UI-Element before going forward
The start of a normal Selenium test case in OLAT could look something like this (for details refer to the Selenium in-depth chapter):
open http://localhost:8080/nightly/dmz
type <username-locator> administrator
type <password-locator> the_password
clickAndWait <login-locator>
The tricky part lies in the locators: these are generally speaking XPaths into the XHtml DOM tree.
In order to achieve the goal of having tests that survive between releases, and at the same time allow structural, graphical and internal changes that effect only the DOM tree and not complete workflows, there must be a concept of having locator URLs independent from the current XPath - i.e. we need to have a Locator Abstraction Layer
Luckily Selenium already comes with a concept called 'UI-Element'. If you have Selenium IDE installed, check here for the documentation
'UI-Element' allows defining customized locator. This means that we can define an alias locator, use that in the test and hide the implementation in a OLAT-version-specific Olat Map file. This allows tests to remain stable from release to release depending on the stability of workflows! If workflows change from release to release, then obviously tests that tested based on the old workflow need to be adjusted
The Olat Map file contains a hierarchically structured list of UI-Element locators mapped to XPaths
These UI-Element locators are then used by Selenium tests and are the only locators that must be used within an Olat Selenium test (enforced by the base testclass BaseSeleneseTestCase)
The Olat map file is stored here:
$CVS/olat3ext/tests/seleniumtests/ui-map/olat-ui-map.xml
Here is an excerpt from the olat-ui-map.xml file:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE ui-map PUBLIC "olat/ui-map" "olat-ui-map.dtd"> <ui-map> <pageset name="dmz" description="DMZ links"> <xpath-ui-element name="username" xpath="//input[@id='lf_login']" description="username input field"/> <xpath-ui-element name="password" xpath="//input[@name='lf_pass']" description="password input field"/> <xpath-ui-element name="login" xpath="//input[@value='Login']" description="login button"/> </pageset> <pageset name="home" description="The home page"> <pageset name="menu" description="the left hand menu"> <xpath-ui-element name="settings" xpath="//div[contains(@class, 'o_home')]//div[@class='b_tree']//a[text()='Settings']" description="left navigation link 'Settings'"/> </pageset> </pageset> </ui-map>
The above map translates into the following UI-Element locators:
ui=dmz::username() ui=dmz::password() ui=dmz::login() ui=home::menu_settings()
Here is a description of the steps involved in converting the olat-ui-map.xml file into user-extensions.js
/olat3ext/tests/seleniumtests/build.xml triggers the whole process in target 'gen_olat-ui-map.js'
/olat3ext/tests/seleniumtests/ui-map/olat-ui-map.xml is the input - the actual olat ui map file
/olat3ext/tests/seleniumtests/ui-map/gen_olat-ui-map.xsl converts the olat-ui-map.xml into /olat3ext/tests/seleniumtests/build/classes/olat-ui-map.js
the three files ui-element.js, olat-ui-map.js and rollups.js are then concatenated into /olat3ext/tests/seleniumtests/build/classes/user-extensions.js which is the final user-extensions file that Selenium needs to be configured with
This section is about how to record selenium tests for Olat and how they get integrated into the nightly test infrastructure
Virtually all links in Olat are language dependent - therefore the Olat Map file is also language dependent. This means that the test will only succeed, if the current language is correctly set to English. For test recording this is best configured before start. For the nightly test framework, this is handled by the BaseSeleneseTestCase (the base test class)
Therefore: always switch to English before you start testing
Logging into Olat is going to be the first task that every test needs to execute. At the same time it is somewhat complicated since we have to make sure, that we're running in English mode.
To simplify all this, there is framework code that handles the login:
BaseSeleneseTestCase.olatLogin(selenium,url,username,password,language)
Note that by default we should be using 'English' as the test language since the olat-ui-map file only works with English
What this means for test recording in Selenium IDE is, that the input for the automated Selenium test classes only need the part after the login
As described in the Olat Map file section earlier, tests are only maintainable if they don't use XPaths/Id-Links directly, but use the Olat Map file with their UI-Element locator notation (ui=dmz::username()). When recording tests Selenium IDE most of the times finds corresponding mapped locators and automagically records Ui-Element locators instead of XPaths or IDs. If that's the case then everything is fine. You will surely come across non-encoded locators though - or special cases which are UI-Element locators that are parametrized. In these cases you must manually fix the locator
Here's a list of known parametrized UI-Element locators:
ui=learningResources::content_clickCourseEntry(nameOfCourse=Demo course wiki)
ui=tabs::closeCourse(nameOfCourse=Demo course wiki)
ui=course::menu_link(link=Wiki sandbox)
Therefore: always use locators that start with ui= and add it to the olat-ui-map.js if the link is not encoded yet
A Selenium test recorded in Selenium IDE can be stored in a Selenium-own format (HTML) or converted into any programing language.

Supported conversion formats in Selenium IDE
The process of converting a Selenium IDE test into a nighlty olat selenium test is as follows:
Record and store the test in Selenium IDE's own HTML format

The test example
Convert the test into Java

The test example converted into Java. The green box marks the relevant part
Extract the relevant part (marked green above) and put it in a Junit class:
package org.olat.test.tutorial.singlenode.junit;
import org.olat.test.util.selenium.BaseSeleneseTestCase;
public class TutorialTestTest extends BaseSeleneseTestCase {
public void setUp() throws Exception {
selenium = connect("idmels04.unizh.ch", "*chrome", "https://nightly.olat.uzh.ch/nightly/dmz/");
}
public void testNew() throws Exception {
olatLogin(selenium, "https://nightly.olat.uzh.ch/nightly/dmz/", "administrator", "******", "English");
assertEquals("OLAT - Home", selenium.getTitle());
selenium.click("ui=tabs::learningResources()");
selenium.waitForPageToLoad("30000");
selenium.click("ui=learningResources::menu_courses()");
selenium.waitForPageToLoad("30000");
selenium.click("ui=learningResources::content_clickCourseEntry(nameOfCourse=Demo course wiki)");
selenium.waitForPageToLoad("30000");
selenium.click("ui=learningResources::content_showContent()");
selenium.waitForPageToLoad("30000");
selenium.click("ui=tabs::closeCourse(nameOfCourse=Demo course wiki)");
selenium.waitForPageToLoad("30000");
assertEquals("OLAT - Learning resources", selenium.getTitle());
}
}
Selenium IDE is the tool for recording and playing back directly in the browser, while Selenium RC (Remote-Control) is for testing remotely on the one hand and automated browser-launch on the other hand.
Download Selenium RC here
But note: Version 1.0 beta 1 does NOT work with Firefox 3 - therefore download the nightly version or ask the developers for the right version since nightly is very unstable at times
Start Selenium RC as follows:
Open a command shell
Change into selenium-remote-control../selenium-server-.. directory
Make sure you have formerly compiled the user-extensions.js file
Then start Selenium RC with this command:
java -cp ./selenium-server.jar org.openqa.selenium.server.SeleniumServer -userExtensions \eclipse\workspace\seleniumtests\build\classes\user-extensions.js
And you are ready to go!
Use the tutorial examples in $CVS/olat3ext/tests/seleniumtests/src/org/olat/test/tutorial
as starting points.
The main idea is to have your JUnit test class extend BaseSeleneseTestCase
and call the two main features:
connect(seleniumHostname, browserId, baseTestUrl);
olatLogin(seleniumInstance, baseTestUrl, username, password, language);
After a test was successfully made run locally, it needs some tweaking to run nightly as well
First of all, the selenium host is no longer localhost but idmels04.uzh.ch instead.
(at least for the moment)
Second, the baseTestUrl might have to be adjusted, e.g. test against nightly.uzh.ch
Finally any other setup which was dependent on having an olat instance or cluster running locally or anywhere else needs to be migrated to nightly - that would include using the Setup Framework as described later in this document
The declared goal of testing in Olat is to achieve a high degree of automatism
Any guarantee of high code quality that can be achieved in an automated way - for example in a nightly test - is very valuable. Having automated tests allows to both keep test costs low and achieve an improvement of quality over release-cycles rather than a decrease due to higher complexities etc.
This automatism is not feasible on exactly all levels of testing, but the goal is to automate where it makes sense from a effort/gain point of view
Table 2.1. Test Matrix
| Functional Testing | Load/Stress Testing | Performance/Scalability Testing | |
|---|---|---|---|
| Class |
|
| |
| Module |
|
| |
| Node |
|
|
|
| Cluster |
|
|
|