OLAT

IMPORTANT: This "single-click" installation is meant to provide you an easy way to get a hands-on impression of OLAT and it's new features. Please note that the preconfigured setup has not thoroughly been tested; specifically with regard to the embedded HSQLDB (Database) used by this installation. This installation comes pre-packaged with the following software components.

Prepackaged Tomcat5
Prepackaged HSQL Database
Comes preconfigured
Provides uninstaller

If you intend to use OLAT in a production environment, please refer to the source distribution. It will give the necessary configuration options to setup OLAT for your specific environment.

Download and unzip the Demo-Installer:

Windows: Double-click on setup.exe to install the preconfigured OLAT on your harddrive. After the installation finished, double-click the newly created OLAT icon on your desktop or run "runOlat.bat".
MacOSX or Linux: Double-click on OLAT-install.jar to install the preconfigured OLAT on your harddrive. If this does not work, open a shell, change to the directory where you've unzipped the Demo-Installer package and run the setup.sh script to start up OLAT. After the installation you need to start a shell and run the runOlat.sh script. On a Mac you need to type:
```
cd /Applications/OLAT
./runOlat.sh
```
On Linux you can double-click the new created desktop icon or open an shell and type
```
cd $PATH_TO_YOUR_INSTALLATION
./runOlat.sh
```
Now start your webbrowser and launch the following URL:
```
http://localhost:8080/olat/
```

You may now log in to your local OLAT installation. Use any of the predefined accounts listed below (username/password):

administrator / olat
author / test
learner / test
test/ test
test2 / test
test3 / test

Please change the passwords of the predefined user accounts or remove the users completely after your first login.

Chapter 2. Install OLAT in a productive environment

This chapter covers the installation and upgrade processes of an OLAT server.

2.1. Prerequisites

For OLAT to run, you'll need the following software:

Java SDK 1.5.x or newer
Tomcat 5.x or newer
Database with UTF-8 support. Note that for our reference system we are running MySQL in Latin1 (not UTF-8) and set the MySQL driver to translate to UTF-8. Running MySQL with UTF-8 support has shown to give a considerable performance hit (sometimes to a magnitude of 10). To avoid this performance hit, the driver mode conversion is a good alternative, although specific to MySQL.
Ant 1.5 or newer (for configuration)

The following software packages are optional, depending on your setup:

Wildfire Jabber Server for instant messaging integration.
MRTG for memory, running threads and usage statistics
Webalizer for access statistics

OLAT has been tested to run on Linux, MacOSX and Windows systems. Our production systems runs on Linux.

If you're planning on running the instant messaging system, make sure you have reasonable memory resources available. The instant messaging system uses (on average) an additional 3 threads per user with a stack size of 256k per thread, which results in significant memory needs. You should be fine with 1 GB of memory to cover for 100 concurrent users if you run both OLAT and the IM system on the same machine. NOTE: This is a rule of thumb projected based on our experience with our production systems with more than 15000 registered users on a single system and average of 150 (peaks of up to 500) concurrent users. See chapter Section 2.4.3, “Tomcat VM parameters and Operating System issues” Tomcat and VM parameter for memory settings

Our reference implementation uses Java 1.5.0.08, Tomcat 5.0.28, MySQL 4.1.18 and latest Wildfire version. Note that MySQL 4.1.11 is not compatible with the MySQL JDBC driver mysql-connector-java-3.1.12-bin.jar that is bundled with the latest release. Use MySQL 4.1.10 or older instead or add "useServerPrepStmts=false" to your JDBC URL to not use prepared statements which are broken in 4.1.11. It is also fine to use a 4.0.x version, it is just a bit slower.

2.2. New OLAT installation

2.2.1. Install the prerequisites

Quick start installation of the needed tools (advanced users)

Install Java SDK (JRE is not enough, because Tomcat needs the Java compiler) from java.sun.com and set the JAVA_HOME environment variable to the install location like JAVA_HOME=C:\java\jdk1.5.0.
Install Tomcat Servlet Engine ( Tomcat download) and set the CATALINA_HOME environment variable to the location where you installed Tomcat. Start Tomcat by running startup (located in CATALINA_HOME/bin directory) and point your browser to http://localhost:8080/ If you see the Tomcat screen, these steps are done.
Install Ant (Built tool, that will compile and setup your OLAT installation) from jakarta.apache.org/ant and set the ANT_HOME environment variable to the install directory. Add the ANT_HOME to your PATH variable as well, so you can run Ant from everywhere.
Install Mysql Database (http://www.mysql.com) Create a new database 'olat' with a user 'olat' and the password of your choice. If you want to execute the jUnit tests, you also need an aditional test-database. E.g. 'testdb' would be a good database name. Some MySQL versions have problems with too long database names (together with the generated tables), eg. olattestdb is too long on some versions. For most users it is fine to just generate the 'olat' database.
There are two alternatives to set up the database and the database users:
1) Use the tool /olat3/bin/dbAndUserSetup.sh. This will generate the OLAT database and the user to access the database automatically.
2) Manually for MySql 4.x: Go to the mysql/bin directory and create a database with mysql_admin programm.
```
# mysql_admin create olat -u root
Change to the mysql database tablespace.
# mysql -u root -p
# use mysql; 
On the console run the following two SQL-statements:
# mysql> insert into user (Host,User,Password) values("localhost","olat",password("olat"));
# mysql> insert into db values("localhost","olat","olat","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y","Y");
# mysql> flush privileges;
```
2) Manually for MySql 5.x: Go to the mysql/bin directory and create a database with mysql_admin programm.
```
# mysql -u root -p
On the console run the following SQL-statements:
# use mysql; 
# mysql> create database olat;
# mysql> create user 'olat'@'localhost' identified by 'olat';
# mysql> grant all on olat.* to 'olat';
# mysql> flush privileges;
```
Alternative Database: Install of PostgreSQL database. See Section 2.3.2, “Postgres”
Now you have a user 'olat' with password 'olat' in the database 'olat'. Note the database username, databasename and password. You will need this later in the build.properties file.
Now your tools are ready to run OLAT.

2.2.2. Getting the sources.

Get the OLAT sources from our homepage www.olat.org.
Create directory for your OLAT installation. We will refer to this directory by PATH_TO_YOUR_OLAT throughout the rest of these installation instructions. Change into your OLAT installation directory now.
Unpack the actual OLAT-X.X.X.zip file in this directory. (eg 'unzip OLAT-X.X.X.zip').

2.2.3. OLAT configuration

Now that you've set up the infrastructure software, you're just a few steps away from your own OLAT installation. All that's left is configuring OLAT according to your setup. This task is almost completely automated by Ant build files.
Copy the PATH_TO_YOUR_OLAT/olat3/build.properties.default to PATH_TO_YOUR_OLAT/olat3/build.properties
Edit PATH_TO_YOUR_OLAT/olat3/build.properties to match your installation. You'll find instructions with regard to the parameters to set in the build.properties file.
Run 'ant config-all' - This will create the necessary config files.
There are two config files for Tomcat generated by 'ant config-all':
server.xml configures Tomcat to accept HTTP requests directly. All requests to OLAT rebuild code will go to Tomcat directly. For testing go with this file and you don't need to install Apache.
server_jk_contector.xml configures your Tomcat with Apache as main gateway to OLAT. All requests go through Apache first and are forwarded through a Tomcat JK Connector by Apache if necessary. This shields Tomcat from the outside world. We suggest using this connector for security reasons.
Rename either of the two files as server.xml and copy it to TOMCAT_HOME/conf/server.xml

To force Tomcat using the compatible XML-APIs, copy the necessary DOM3-enabled XML JARs (xercesImpl.jar, xml.apis.jar and xalan.jar) into Tomcat's endorsed directory.

# cp PATH_TO_YOUR_OLAT/workspace/olat3/webapp/WEB-INF/lib/xercesImpl.jar xml.apis.jar xalan.jar CATALINA_HOME/common/endorsed
To compile OLAT with your installed JDK you need the XML API's also in your Java enviroement.
The JDK endorsed directory is not created by default.
You'll have to manually create a directory called $JAVA_HOME/jre/lib/endorsed.
# mkdir JAVA_HOME/jre/endorsed
# cp PATH_TO_YOUR_OLAT/workspace/olat3/webapp/WEB-INF/lib/xercesImpl.jar xml.apis.jar xalan.jar JAVA_HOME/jre/endorsed

Now you should be ready to compile OLAT, type:
# ant build

Finally you'll need to setup the necessary database tables for OLAT. 'ant dbsetup' will do the job for you. Be careful with that target on a productive system since you will loose all your data!
You can now start Tomcat (if configured restart Apache)
Access your OLAT installation:
You can reach your OLAT installation at http://your.site.com:[PORT]/CONTEXT_PATH where CONTEXT_PATH is the context path of your application as you've configured it in your build.properties file. The default is "olat" e.g. http://localhost:8080/olat
There are some accounts setup by default:
Login: 'administrator' Password: 'olat'. This account has full administrative rights. We advise you to change the password immediately (see 'Home' -> 'settings' -> 'Password')
Login: 'gast' Password: 'gast'. This account is the default account for 'Learningresources' which grant access to guests. By default, this account does not have access to any content in OLAT.
Note: If you encounter an error like: "Error creating bean with name 'bbfactory' defined in file [/opt/OLAT/webapp/WEB-INF/olat_extensions.xml" make shure you give the right ownership to all files processed by the user that runs tomcat. On linux you can reach this by using the command "chown -R yourtomcatuser:yourgroup PATH_TO_YOUR_INSTALLATION".

2.3. Database and UTF-8 configuration

2.3.1. MySQL

OLAT is UTF-8 capable and needs an UTF-8 capable database. Running MySQL with UTF-8 support (Introduces with MySQL 4.1) has shown to give a considerable performance hit (sometimes to a magnitude of 10). To avoid this performance hit, the MySQL driver mode conversion is a good alternative, although specific to MySQL. On our reference system, we are running MySQL in Latin1 (not UTF-8) and set the MySQL driver to translate to UTF-8. With this setup, the MySQL driver transparently translates between UTF-8 (OLAT) and Latin1 (MySQL DB). You'll need to configure the MySQL driver with the following parameters:

useOldUTF8Behavior=true
useUnicode=true
characterEncoding=UTF-8

The JDBC URL for such a setup would look like this:

jdbc:mysql://my.server.com:3306/olat?useOldUTF8Behavior=true&useUnicode=true&characterEncoding=UTF-8

NOTE: This is specific to the MySQL driver. Other drivers similarly may support driver-based translation but we suggest to use native UTF-8 with any database other than MySQL.

2.3.2. Postgres

There is a known installation that runs OLAT with Postgres. Thanks to Sebastian Hennebrueder (usenet@laliluna.de) for patches and testing for making OLAT Postgres compatible!

Olat is tested with PostgreSQL version 8.0.3. You can download PostgreSQL at http://www.postgresql.org. Create a new database 'olat' with a user and a group 'olat' and the password of your choice. We tested olat with unicode encoding but you may also try other encodings (e.g. Latin1) . If you are a developer and want to execute the jUnit tests, you also need an additional test-database. E.g. 'testdb' would be a good database name. For most users it is fine to just generate the 'olat' database. Below we show the steps setting up the database using the psql client, which is installed with PostgreSQL by default. You can find it in the PostgreSQL_HOME\bin directory. We will process the following steps:

1. create a database
2. create a user olat
3. create a group olat and add the user to the group
4. grant access to the database for the group olat

psql -U admindbuser template1;
#input password
# create database

CREATE DATABASE testdb
WITH OWNER = replace_with_your_admin_user_eg_pgsql
ENCODING = 'UNICODE'
TABLESPACE = pg_default;

# connect to testdb
\connect testdb
#setup user, group and privileges

CREATE GROUP olat;
CREATE USER olat PASSWORD 'olat' VALID UNTIL 'infinity';
ALTER GROUP olat ADD USER olat;
GRANT ALL ON SCHEMA public TO olat;

Finally you'll need to setup the necessary database tables for OLAT. We prepared the setup for MySQL and for PostgreSQL. Using MySQL use the following ant task: 'ant dbsetup' will do the job for you. Be careful with that target on a productive system since you will loose all your data! Using PostgreSQL use the ant task: 'ant dbsetup_postgreSQL' The ant tasks creates the tables for the database you configured in the build.properties. It will use the sql script setupPostgreSQL.sql in the directory OLAT_HOME/database. This task does not delete any tables like the MySQL task. When you want to delete all the tables you can use the sql script deletePostgreSQL.sql in the same directory.

2.3.3. Other databases

OLAT uses Hibernate as a database abstraction layer, so virtually any database can be used. We recommend using MySQL however. OLAT is well tested in conjunction with MySQL and there is only experience using Postgres as well. Make sure your database supports UTF-8.

Before you start you should make yourself familiar with the Hibernate framework. Search in the docu if your database is supported and jot down the name of the dialect for your database.

Follow this quick guide to use OLAT with another database:

Install driver for database XYZ
- Put the jdbc-jar of your database in olat3/webapp/WEB-INF/lib
- Modify in your olat3/build.properties the properties db.jdbc.driver, db.jdbc.jar and db.jdbc.url
- Modify in your olat3/webapp/WEB-INF/olat_config.xml.in the hibernate database dialect in the database module. (modify the .in file, ant config-all will overwrite the .out and the .xml file.
Configure OLAT: run ant config-all
Generate the database for your database
- Run the class olat3/webapp/WEB-INF/classes/org/olat/persistence/DatabaseSetup using the argument "net.sf.hibernate.dialect.XYZDatabaseDialect createScript"
- Check the generated database file located at olat3/database/o_database.sql. Maybe you need to adjust some things. E.g. we use the MySQL datatype "text" explicitly. This can be replaces with "blob" or something similar.
Start OLAT.....
If successful, send us your feedback, your scripts, your How-To pages...

On later upgrades the changes provide in the database/alter_x_to_y.sql must be reviewed and modified each time.

2.3.4. JUnit tests

If you have created the separate junit database you can run the junit tests. The make sense if you use an other database than the tested or do change code on manager classes accessing the database. To run the tests run manually the following class: org/olat/tests/AllTests.java with eclipse and the following attribut. attribut -Dolatdir=/your/path/to/olat3. Make shure you have changed all the necessary properties in the build.propertie file and runned 'ant config-all'

2.4. Tomcat and Apache configuration

OLAT runs in a standard J2EE servlet container as a webapplication. There is no need for EJB support. We suggest using the official servlet container reference implementation Apache Tomcat. Tomcat is available for free under an Open Source license.

Although you can use any other servlet container implementation we recommend using Tomcat since OLAT has been in production using the Tomcat servlet container for quite some time now. If you decide to use another container, please send us your installation instructions and any tip or trick that could help installing OLAT - we will include it into this manual.

During installation some example configuration files have been generated.

#cd PATH_TO_YOUR_OLAT/olat/conf/
#ls
apachelogrotate		apachelogrotate.in
httpd.conf			httpd.conf.in
httpd.conf.out		mod_jk-1.3.26.dll
server.xml						server_http_connector.xml.out		wildfire.xml
server.xml.in					server_jk_connector.xml				wildfire.xml.in
server_http_connector.xml		server_jk_connector.xml				wildfire.xml.in

If you have no server_http_connector.xml, server_jk_connector.xml and httpd.conf in this directory you need to generate them first. Make sure your build.properties file has the correct values. Copy the default property file if you haven't done this so far.

#cd PATH_TO_YOUR_OLAT/olat3/
#cp build.properties.default build.properties
#vi build.properties
#ant config-all

2.4.1. Standalone: Tomcat using the HTTP connector

When installing OLAT the first time we recommend to install it first only using Tomcat with the httpd connector to reduce complexity. When everything works fine you can then proceed installing the server.xml that uses the JK connector.

Copy the PATH_TO_YOUR_OLAT/olat3/conf/server.xml to your tomcat installation.

#cp PATH_TO_YOUR_OLAT/olat3/conf/server.xml PATH_TO_TOMCAT/conf/server.xml

In most cases the generated file works out of the box, if not look at the file and do some tweaking.

Restart Tomcat

2.4.2. Combined: Tomcat and Apache using the JK connector

For a production server it is recomended to use apache as a gate keeper in front of tomcat. This is for many reasons, mainly security issues, better support for virtual hosts, ssl, standard tools for log analyzing, availability of douzens of usefull plugins for apache like redirect, throttle etc.

There are two connectors: JK and JK2. JK2 is not in development anymore, the examples are all for the JK connector.

There are two versions of Apache: 1.3.x and 2.x. For some time now we use Apache 2.x. All examples are for Apache 2.x, however the http configuration file should also work with a 1.3.x installation.

Copy the PATH_TO_YOUR_OLAT/olat3/conf/server_jk_connector.xml to your server tomcat installation, rename it server.xml

#cp PATH_TO_YOUR_OLAT/olat3/conf/server_jk_connector.xml PATH_TO_TOMCAT/conf/server.xml

In most cases the generated file works out of the box, if not look at the file and do some tweaking.

Restart Tomcat

Copy the PATH_TO_YOUR_OLAT/olat3/conf/httpd.conf to your apache installation.

#cp PATH_TO_YOUR_OLAT/olat3/conf/httpd.conf PATH_TO_YOUR_APACHE_CONFIG_FILES/vhost_olat.conf

Now you need to include this virtual host in your apache installation. This is different on every unix platform so we can't give you any advice here. In most cases the generated file works out of the box, if not look at the file and do some tweaking.

The generated httpd.conf references to PATH_TO_YOUR_OLAT/olat3/conf/workers.properties. In most cases the generated file works out of the box, if not look at the file and do some tweaking.

Tip

The <IfModule mod_jk.c> section in the generated httpd.conf file must be outside the virtual host definition. Only one workers file can be included for the whole Apache installation. If you want to run multiple virtual hosts you need to define multiple mount points, for each virtual host one.

If you need SSL support add another virtual host based on the generated httpd.conf file.

Restart Apache.

2.4.2.1. MacOS X

Under MacOS X the Apache configuration is quite different than it is on a Linux system. Robert Frank from Univerisity of Basel contributed a setup generator script that generates the necessary files. Thank you Frank!

You will find the script at PATH_TO_YOUR_OLAT/olat3/bin/MacOSXOlatSetupGeneraror.pl

2.4.3. Tomcat VM parameters and Operating System issues

The following virtual machine parameters should be used when starting tomcat:

-Xss256k: specify the stack size. Default is 0.5 MB which is way to much and consumes a lot of memory. This is the memory used for each thread. If you have the Instant Messaging component enabled for every logged in user four threads will be created which can result in a very high memory usage. Please note that this is stack memory and is consumed in addition to the memory you specify with the -Xms and -Xmx flags!
-Xms768m -Xmx768m: specify the amount of memory tomcat can use as a maximum. With about 200 concurrent users (Userbase 7500 users) OLAT usually requires about 300 MB. This configuration example is for a very large site. Note that in addition to the memory specified here the VM uses for every thread an additional amount of memory specified by the -Xss option.
-Djava.awt.headless=trueIf you run OLAT on a Unix/Linux box with no graphics environment started (e.g. KDE, Gnome) classes using image frame buffers will not work and throw an exception. By enabling the headless mode the jvm emulates an graphics environment.

Example configuration from the tomcat5/bin/catalina.sh file:

CATALINA_OPTS="-server -Xss256k -Xms768m -Xmx768m -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true"

The following is no longer relevant for OLAT > 4.0.0, but is still an issue if you run OLAT 3.2.x

When you have the course logging enabled in the olat_config.xml file every course has at least three permanent opened filehandel. If your system is not configured propertly your Java VM can get very easily in too many opend files-condition. Unfortunately the standard configuration in Linux is to allow each process to have only 1024 opend files. Change this setting to a value that fits your needs. You can test your limits using the following commands:

#count open files for a certain process
lsof -p [process-id] | wc -l
 
ulimit -Ha
ulimit -Sa

See the manual of your operating system how to change this values

2.5. Configuring the wiki rendering engine to display TeX mathematic formulas

The integrated Wiki rendering component which is currently used in the repository type Wiki can display mathematics in Web pages. Is is based on the great javascript project called jsMath ( jsMath Website). It tries to render the formular with special mathematics fonts, if the fonts are not installed it uses images provided by the javascript library. As it is a huge number of images (20'000) we do not provide them unpacked in our distro as it makes unzipping the distro too long. Therefore you have to run 'ant jsmath' to unzip the font files to the appropriate location.

2.6. Enabling the Instant Messaging component

The fully integrated Instant Messaging System enables the following features in OLAT

Private Person to Person chat with anybody who is online in OLAT
Synchronisation of OLAT groups with Roster of Instant Messaing Server. See always the Buddies that are online and available for a chat.
Roster integrated in OLAT GUI. Instant access to your Buddies wherever you are in the OLAT GUI.
Groupchat feature for each course and the users Buddygroups.
Chat and group chat client written in Flash
OLAT Instant Messaging System based on the Jabber Internet standard (XMPP protocol)( Jabber.org)

2.6.1. Basic install with mysql database

Prerequisites

You can't enable Instant Messaging on the "easy installer". You need to setup a development environement to enable and test the Instant Messaging features. Mainly because of the needed database.
Instant Messaging Server: Wildfire (formerly knows as JiveMessenger) from jivesoftware.org. An Open Source IM server unter the GPL license written in Java. To download the latest server go to: JiveSoftware download Site and download the version that suits your os best.
Wildfire needs a Java Virtual Machine > = 1.5. (You can run OLAT as well with java 1.5)
Mysql database. Wildfire runs also with other databases (like OLAT does) but we only deliver a prepopulated database setup for mysql.
This installation describes the steps if you run the Wildfire on the same machine together with OLAT and share the same database server, but you can run the IM server as well on another machine.

Installation: Go to OLAT_HOME and edit the build.properties file like:

#Instant Messaging properties
instantMessaging.enable=true (enable or disable all instant messaging features)
instantMessaging.server.name=your.jabber.server 
#(you need a own dns name or just localhost for testing)
instantMessaging.generateTestUsers=true 
# (the OLAT test users are generated in OLAT with the default passwords also for instant messaging)
instantMessaging.admin.username=admin 
#(these are the default settings of jivemessenger, Change later!)
instantMessaging.admin.password=admin

To reflect these changes run an ant taks (run ant tasks inside the OLAT_HOME folder):

# ant config-all

Now the config file and the database file for the server is generates at OLAT_HOME/conf and you are almost ready to go. Create a database with the name "wildfire" in mysql and populate it with tables and values. Type the following command on the console:

# mysql -u [olat_db_user] -p 
# create database wildfire;
# exit;
# mysql -u [olat_db_user] -p wildfire < OLAT_HOME/conf/instant_messaging_database.sql

Now you should have a database and you only need to copy the Wildfire config file to the place where you dropped and unpacked the downloaded JiveMessenger archive.

# cp OLAT_HOME/conf/wildfire.xml WILDFIRE_HOME/conf

Your IM server is now ready to be started and tested standalone. To start and stop use either the wildfire file in the WILDFIRE_HOME/ bin directory or if it fails run.

# cd WILDFIRE_HOME/lib
# java -jar startup.jar
You should see an output like:
Wildfire 2.5.1 Started [Mar 31, 2005 5:37:03 PM]
Admin console listening at:
  http://localhost:9090
  https://localhost:9091

To test your server go the http://[your-instant-messaging-servername]:9090 and log in as admin/admin and you see the admin console. To test the Instant Messaging features open your favorite jabber client like (Exodus, PSI, Kopete, ...) and log in as one of the OLAT Test users automatically created. (user:administrator/pw:olat - user:author/pw:test - user:test/pw:test)

If you are able to chat with each other you are ready to start OLAT together with the IM server. The IM server should run at the time you start OLAT, otherwise you get an error message that you have enabled Instant Messaging but the server is not running.

2.6.2. Features explained

Technical overview
The OLAT Instant Messaging system is build with jabber components. The server (Wildfire) implements the XMPP protocol and is compatible with most Jabber clients. OLAT connects to the server using two clients. First with the open source client library SMACK from JiveSoftare and a flash client developt with the XIFF client library that is maintained also by JiveSoftware.
The SMACK client is used to connect to the server and gets the Roster and manages presence information for the buddies of the connected user. He also receives messages if the flash client is not running. As soon as the flash client gets started all messages are routed to the flash client. The flash client is very limmited in its functionality to serve as an easy client for chat beginners. If people like to use one of the fully featured Jabber clients they can use them too. If you open the flash client you can view the connection settings for this user by clicking "view -> password". The Instant Messaging server decides by the priority of each resource where to send messages. If you use your own jabber client you have to set the resource higher then the SMACK client (priority 0) and lower than the flash client (priority 5) to work best.
OLAT groups synchronizing with the Wildfire server.
OLAT provides a tool for managing personal groups with members called OLAT 'groups' When enabling the Instant Messaging features the information from the groups is used to create and maintain a roster in the instant messaging server. The roster stays always visible in the OLAT user interface and provides a overview of the availablity of the users buddies.
As the Instant Messaging server is completely separated from OLAT the information is synchronized between the components. We have to explain the mechanism first that you understand how the prozess works. As users who log in using the Shibboleth authenitification mechanism don't have a passwort in OLAT. We create an additional account for each user on the IM server. The credentials consits of the OLAT username which is unique in OLAT and a random generated passwort only used for IM. When a user logs into OLAT for the first time OLAT checks if the user has an IM password. If no, a new accounts gets created on the IM server and the password for this IM account is saved in the o_bs_authentication table with the provider name "INST-MSG". On the IM server you should see the created account as well. There are two cases when you need to synchronize OLAT with the Instant Messaging server.
case 1: You start olat for the first time with Instant Messaging enabled. Here everything should work automatically if you make shure that the IM server is already running befor starting OLAT. In this case OLAT creates the groups and the useraccouts of the members of each group in the IM server. If you log in as a user you should see your groups as a Roster containing the members.
case 2: In case of a server crash of the IM server or other problems you have to resync the data to make shure both databases are synchron. Shut down Wildfire. Log in to the mysql database:
```
# mysql -u [database user] -p
# use wildfire;
# delete from jiveGroupUser;
# delete from jiveGroupProp;
# delete from jiveGroup;

# use olat;
# delete from o_property where name='org.olat.instantMessaging.InstantMessagingModule::issynced';
  
```
Next time you start OLAT it synchronizes again the data between OLAT and the IM server.
Manually setting the Instant Messaging server properties (setting up the server not with the provided database script)
The following steps you only have to do if you set up your Instant Messaging database without the provided database script. Go to the admin interface of the server and adjust the following properties.
- Check the server name: You should use a hostname that gets resolved by the DNS server, otherwise the clients of olat will not be able to connect to the server. Server Settings -> Server Name
- Registration - Login -> Disable both properties so user cant create accounts themself.
- Resource Policy -> Choose "Always Kick".
- Group Chat -> Histroy Settings -> "Show a Specific Number of Messages"
Using the flash instant messaging client when the Wildfire IM server is installed on an other machine then OLAT itself.
The flash client can only connect to the same domain where the swf file got loaded from. If you install the IM server on an other host you have to install a simple webserver that delivers the swf file for the chat client over the same server. Otherwiese the flash client refuses connections to this server (Security feature of the flash plugin.).

2.6.3. Issues

To decrease memory usage start the server with the following virtual machine parameters. "java -jar -Xss256k startup.jar". This will dramatically decrease memory usage on Linux based systems as only 256KB memory is reserverd for each thread stack (default is 512KB)..

2.7. Enabling the Shibboleth login

Shibboleth is a standardized authentication and authorization mechanism. With Shibboleth, OLAT forwards a user to its Identity Provider (her university / company website) for authentication. The Home Organisation will (upon successfull authentication) take the user back to OLAT including an Authentication Statement which is verified by OLAT. Upon proper verification of this Authentication Statement the user is automatically logged in to OLAT.

The configuration of Shibboleth is mostly done via build.properties / olat_config.xml. The following parameters have to be set:

EnableShibbolethLogins (true/false): If set to "true" Shibboleth will be enabled on your installation.
ProviderId (String): A unique identification of your local installation.
Metadata (Path/URI): This is the file that contains the Home Organisations supported for your users to authenticate. The file is based on the Shibboleth 1.3 metadata file. The file can be either locally (relative path to web application root or absolute path) or remote (URI accessible via HTTP).
AAP (Path/URI): The file contains your Attribute Acceptance Policy (AAP). The file can be either locally (relative path to web application root or absolute path) or remote (URI accessible via HTTP). If you do not have a specific Attribute Acceptance Policy (i.e. you accept all attributes) you may also leave this empty.
ReloadMaxRetries (Integer): The number of times the watchdog should re-try to fetch the sites.xml file if previous attempts failed. A value of "0" means unlimitted retries.
CheckCertificateValidity (true/false): Wether to check if the certificate used to sign the AuthorizationStatement was issued by an recognised Certificate Authority as defined by the ssl-truststore parameter of opensaml.properties file. Use this, if you want to make sure that only Home Organisations which you trust are allowed to authenticate users.
CheckIssuerIP (true/false): If set to true, the IP of a user who is trying to authenticate will be checked when the Identity Provider returns it's authentication statement. Usually you should set this to true for security reasons. With some proxies who do not assign stateful IPs, access will be denied and you may disable this feature.
Credentials / Keystore: These are the credentials used to connect to the Attribute Authority to retrieve Shibboleth attributes upon successfull authentication. Type may be either PKCS12 or JKS.
Credentials / Truststore: These are the credentials of Identity Providers who you allow to authenticate users (i.e. your federation's certificates). Type may be either PKCS12 or JKS.
UniqueIdentifiers: Defines the attributes which allows to uniquely identify an authenticated person. Make sure, values of this attribute are truly unique. It is used to map an identity who has been authenticated by an Identity Provider to an identity within OLAT. The default unique identifier ("<Default />")is mandatory. You may specify unique identifiers on a per-site basis (e.g. <OriginSite siteName="examplesite" uidAttribute="exampleuid" />)
AttributeTranslations (contains Attribute elements): Shibboleth can provide attributes to OLAT after successfull authentication of an identity. These attributes are propagated within OLAT for use e.g. as a pre-condition within a course. To make the attribute names easier to read (and use) you may provide translations to the attributes. E.g. "eduPersonEntitlement" reads easier as opposed to "urn:mace:dir:attribute-def:eduPersonEntitlement" (which is the attribute's actual name). See the olat_config.xml.in file for an example.
OLATUserMapping : This section is used to map Shibboleth attributes to user preference attributes within OLAT. After each successfull Shibboleth authentication, the given Shibboleth attributes are mapped to the corresponding attributes in the OLAT user profile. E.g. Shibboleth Attribute "urn:mace:dir:attribute-def:givenName" would be mapped to the OLAT user profile's first name with the following statement:
```
<FirstName>urn:mace:dir:attribute-def:givenName</FirstName>
```
Valid mappings are: FirstName, LastName, EMail, InstitutionalName, InstitutionalEMail, InstitutionalUserIdentifier.

Some of these parameters can be configured through the build.properties file.

Step by step guide to enable shibboleth on your OLAT installation (Without set up of identiy provider).

Get a FQDN (DNS name) for your OLAT installation like: olat.earth.sky
Create an RSA key and out of it create a certificate request using openssl. See example from Swiss Education & Research Network: How to obtain server certificate.
Let the certificate request sign from a certificate authority
Download the signed certificate in the PEM format with the full certificate chain included.
Convert the certificate into a format that tomcat likes (PKCS12) using openssl.
Place the certificate in a directory tomcat can read and adjust the build.properties with the path and password phrase.
Your AAI foderation should provide you with Federation Metadata. The federation metadata contains information about all Service Providers and Identity Providers participating in the federation. Download the file and place it inside webapp/WEB-INF direcotry and adjust the path in the build.properties file.
If you intend to load attributes from the identiy provider you should make sure your have the rights to access them.
run the config-all task and restart OLAT.

2.8. Directory structure, which files are important

Directories under /...../olat3 source directory

bin                                useful scripts for linux/macOSX 
conf                               configuration files for apache and instant-messaging
database                           database setup file and alter files to upgrade from former OLAT versions
dist                               This directory contains the olat distribution files. (use ant)
doc                                this documentation
    misc
  dev
        eclipse
        images
    usr
htdocs                             nicer error files (404 etc) for apache
interfaces                         sample files for macromedia flash integration in olat qti
monitoring                         setup files and shell scripts for MRTG monitoring tool
   bin
   conf
   data
temp
webapp
    content                        Contains the Layouting files for the VelocityContainers
                                   (same package structure as code) 
           
    examples                       the example courses and test installed for demonstration purposes.
    help                           the help course (online help, in German only yet)
    i18n                           Release 4.X: The LocalStrings files for translating (same package structure as code)
                                   Release 5.X: These files have been moved to the respective Java-Source-Code folder
       default
           de
     en
     ...
    static                         static files
       cp_offline_menu_mat         files for the generation of offline cp packages
       css                         static file delivered to the webbrowser                        
       disclaimer
    extensions       static files from olat extensions
       flash
       help
          de
             img
       images
       js                          diverse javascript
          jscalendar
          kupu
          overlib
    scorm
       movie
       rte
    WEB-INF
        classes                    java classes
        lib                        java libraries
        patchesSrc                 patched source of patched libraries
        src                        java src and hibernate, log4j properties
            conf
            META-INF
               services
            org
               olat

Directories under /installation-path/olatdata

bcroot                             storage for most permanent files
   course                          storage for all olat courses
      71062707016999               - e.g. one olat course
         coursefolder              files uploaded into the coursefolder of the course
         foldernodes               storage for "folder"-nodes in the course          
            71063388996074
         logs                      storage for the log files for this course
   cts                             "collaboration tools storage"
      folders                      Storage for tool "folder"
          BusinessGroup            Learninggroups or rightgroups (course), buddygroups
              720897               an instance
      wikis						   Wiki's used in the group context
      	  BusinessGroup
      	  	  720897
   homes                           personal folders of all users
      fmuster                      personal folder of user with login "fmuster"
          private
   repository                      Learnresources: ContentPackaging and QTI-Tests/Selfassement/Surveys/wiki
      71062707016960
         _unzipped_
 scorm         Persisted user data from scorm content.
   tmp
logs                               the technical OLAT audit/error log and apache log
monitoring                         MRTG files
qtiser                             serialized files for tests/surveys/selfassessments in progress 
   fmuster                         for user "fmuster"
resreporting                       QTI-Resultreporting files (after completion of a test/survey/self)
   fmuster                         user "fmuster"
       Assessment                  type Assessment
        file 7122137371.xml     a resultsreporting file
       Self-Assessment
tmp                                temporary files
    qtieditor                      for qtieditor
        fmuster                    user "fmuster"
           70798419637952          instance of an editing sequence
              media                image upload for the qti image materials
           70864704465026
        zipcfjolat70864704465031
           media

2.9. Configuration Script

The configuration of OLAT is based on an ANT configuration script (build.xml) with various targets. The main targets are:

config-all: Prepares all config files and deploys them.
dbsetup: Creates the database tables needed for OLAT.

The "config-ally" target generates config files based on templates. All templates have ".in appended to their target name. The target first generates a corresponding ".out" file and then moves this file to its target destination (ommiting the ".out" extension). The files generated by the config-all target are the following:

webapp/WEB-INF/olat_config.xml: The main OLAT configuration file. Most of the features are configured by setting the config options in the build.properties file (which then get copied over to the olat_config.xml). However, some of the rarely used features can only be configured directly in the olat_config.xml file. Be aware that this file gets overwritten each time you call the config-all target.
conf/server_http_connector.xml: The Tomcat server.xml configuration file if you use tomcat without Apache.
conf/server_jk_connector.xml: The Tomcat server.xml configuration file if you run Tomcat with Apache and the JK connector.
conf/httpd.conf: The Apache config file, needed if you want to run Tomcat with Apache and the JK connector.
webapp/WEB-INF/web.xml: The OLAT web application configuration file.
webapp/WEB-INF/log4j.propertes: The OLAT logging system configuration file.

2.10. Setup graphical server monitoring using MRTG

OLAT provides some graphical monitoring tools to supervise the helthness of your server. The graphics are generated by a tool called MRTG which originally has been designed to monitor network devices. Thanks to the very generic interface it can be used to monitor anything that provides a comparable value.

The available scripts are all built for Unix. If someone creates some windows equivalents, please share them with us.

2.10.1. Install MRTG

The open source tool MRTG can be downloaded from MRTG site. Most Linux distributions have pre-built binary packages.

2.10.2. Build the configuration file

Go to your olat3 directory and use ant to generate the MRTG configuration file. Make sure the path to your MRTG is correct in your olat3/build.properties file. Usually you don't have to change anything there:

...   
# MRTG configuration for monitoring (optional)
mrtg.bin=/usr/bin/mrtg
monitoring.dir=${base.dir}/monitoring
...

Now generate the configuration for MRTG and the olat server monitoring scripts:

#cd olat3
#ant config-all

Now the necessary configuration is located at olat3/monitoring/conf/

mrtg.cfg: The MRTG main configuration file. Usually you do not need to change anything here. Review the file and proceed with the installation of the cronjobs.

2.10.3. Installing the monitoring crontabs

monitoring-crontab: This are the cron jobs you need to install. You can use any cron system. On most Unix systems you can type the following as root:

#crontab -e -u apache  (or wwwrun or whatever user is running your apache)

Now paste the generated sample monitoring-crontab file from olat3/monitoring/conf into the editor and save.

The cronjobs fetch some statisitcal values from the OLAT server every 5 minutes, plus it checks the ping time to google to see if your network is up and running and writes the information in a temporary file. MRTG uses then the files to read the value (kind of not so nice, feel free to send us a much nicer configuration file. Then if runs the MRTG program every 5 minutes to update the graphics.

2.10.4. Configuring Apache

Configure your apache to deliver the index page from the mrtg output. This is in your olatdata/monitoring directory. See the section Section 2.4, “Tomcat and Apache configuration”.Search for the monitoring section.

2.10.5. Interpreting the images

Point your browser to the location where you have your monitoring defined. You will get an overview page that contains the daily and weekly statistics. Click on any title to also see the montly and early statistics.

The images are kind of odd. They must be read from right to left and not vice verca. This means: the present is at the very left side of the image, and the past is at the right side.

Users online: number of authenticated user sessions. If a user is logged in twice, he will be counted twice. The blue line shows the users logged in using SSL.
WebDAV iser: number of users connected using WebDAV. The blue line shows the users who use an SSL WebDAV connection.
Instant messaging users: number of running instant messaging clients. This acutally shows the distinct number of online users since at each moment there can only be one instant messaging client in the system per user. The blue line shows the number of opened chat windows.
Link availability: milliseconds needed to ping google.com. A network test
Apache and MySQL processes: number of running apache processes (green) and running MySQL processes (blue). Handy to optimize your configuration.
CPU load: Unix load multiplied by 100. (MRTG can only handle integers, load is usually a number between 0 and 1. Read Unix manpages to really understand the meaning of this value. Generally you can say, a load of 1 (value of 100 in the graph) means that in the average one process is requesting CPU cycles at each moment. If you have a multi processor machine the interpretation of this value changes. A machine with four processes can handle a load of 4 to have the same busyness as a one processor machine with a load of 1.
Memory: Green is the amount of memory used (Total memory - free memory - buffered memory - cached memory) in your system. Note that this is not the memory used by OLAT but a monitoring of your system memory! Blue is the amount of swap used (Total swap - free swap). Note that it is common that the swap is used under linux even there is free memory. Read the manpages to understand the memory management of your operating system.
Runtime controllers: number of OLAT controllers held in a weak hash map. When the line drops, the garbage collector has cleaned up the unused controllers.

2.11. Setup Fulltext Search

The basic configuration of OLAT Fulltext Search is mostly done via build.properties. The following parameters have to be set:

generate.index.at.startup (true/false): If set to "true" Search index will be generated at OLAT startup. See also parameter 'restart.window.start' , 'restart.window.end' and 'restartDayOfWeek'
restart.window.start [0..24]: Define start-time of restart-window. See also parameter 'restart.window.end'. E.g. 01:00-02:59 restartWindowStart=1 restartWindowEnd=3. The hole day : 00:00-24:00 restartWindowStart=0 restartWindowEnd=24. Default = '0'
restart.window.end [0..24]: Define end-time of restart-window. Default = '24'.

The advanced configuration of OLAT Fulltext Search is mostly done via olat_config.xml under section 'SearchService Module'. The following parameters can be set:

restartInterval [0...X]: Restart Indexing after xx ms (0=no restart) Default = '0'.
restartDayOfWeek [0...7]: Restart only at this day 1=Sunday,2=Monday...7=Saturday (0=restart every day) Default = '0'.
pptFileEnabled (true/false): Enabled/Disable indexing of power-point files. Default = 'true'.
excelFileEnabled (true/false): Enabled/Disable indexing of excel files. Default = 'true'.
maxResults [0...X]: Max number of search results. Default = '100'.
maxHits [0...X]: Internal max number of search result hits. Default = '1000'
fileBlackList Space separated list of file-names which will not be indexed. Default = 'imsmanifest.xml'
indexInterval [0...X]: Sleeping time of indexing thread between adding documents in ms (0=no delay). Use this parameter to reduce CPU load. Default = '0'
documentsPerInterval [0...X]: Number of documents which will be added to index before sleeping (see indexInterval). Default = '4'.
numberIndexWriter [0...X]: Number of index-writer threads(0= no parallel working index-writers). Use this parameter for performance tuning. Default = '0'
folderPoolSize [0...X]: Number of folder-indexer threads. Use this parameter for performance tuning. Default = '0'.
pdfTextBuffering (true/false): Enable text-buffering of extracted PDF text. Use this parameter if you have many pdf files for performnace tuning. Default = 'false'.

Chapter 3. Upgrading from previous installation

See the first section to understand the general upgrading approach. Then consult the following sections that applies to your setup.

3.1. General upgrade concept

When upgrading from a previous OLAT version several things have to be done:

Download the new OLAT release and unzip it. Move your old olat first to a save location to have a backup if something goes wrong. Configure the new OLAT code as descibed in the installation section of this documentation. This involves using the correct values in the build.properties files and running of ant config-all. The following listing shows the necessary steps
```
#cd PATH_TO_YOUR_OLAT
#rm olat3 (if olat3 is a symlink, otherwise: mv olat3 olat3.1.3)
#unzip OLAT-3.1.4.zip
#ln -s OLAT-3.x olat3
#cd olat3
#cp build.properties.default build.properties
#vi build.properties
#ant config-all
#ant install
```
It is no problem to move from a OLT 3.1.0 installation directly to a 3.2 installation on the programm code level. This does not apply to the database changes as noted below.
Note that the steps above will not change any data that resides in your olatdata directory when your have your olatdata defined outside the olat3 directory. The olatdata directroy should never be located in the olat3, this is only the place for the programm code!
Changeing the database model, adding new tables, altering existing tables due to changes in the programm code. This step is not always necessary. See if there is a file in PATH_TO_YOUR_OLAT/olat3/database/alter_*_to_*.sql that matches the version you want to install.
Befor you do anything: please backup your database! Run something like this
```
#mysqldump -u olat -p olat >mydump.sql
```
It is very important that you execute this sql file in the correct order! When you are running OLAT version 3.1.2 for example it is absolutley mandatory that you run first the 3.1.2_to_3.1.3, then the 3.1.3_to_3.1.4 and only after that the 3.1.4_to_3.2.0!
Please note that the provided scripts are NOT inteded to be run automatically. Please run them manually and execute every statement after another so that you can verify that no error occures. And please do also read the comments in the files, sometimes a change can not be done by a single query.
Sometimes no database changes are necessary. In this cases, no alter file is provided.
The provided alter files are written for a MySQL database. If you use another database, make sure you reviewed the queries properly.
Modifying files on the filesystems due to changes in the programm code. This step is very rarely necessary. If it is needed it will be described in the specific upgrade section in this manual. Usually a little script will help you performing this task.
Now start tomcat. The first launch might take some time.

3.2. Upgrading from 2.x

OLAT 3.x is not compatible with OLAT 2.x. Install OLAT 3.0 as a new installation as described in chapter 2. To migrate your data from your existing OLAT 2.x installation you need to install the newest 2.x release and then follow the migration instruction described in the 2.x to 3.0 migration guide.

3.3. Upgrading from 3.0.0 to 3.0.1

Install OLAT as described in the installation chapter but DON'T RUN 'ant dbsetup'!! We recommend to move your existing installation from PATH_TO_YOUR_OLAT/olat3 to PATH_TO_YOUR_OLAT/olat3.0.0 and install the 3.0.1 package as your new PATH_TO_YOUR_OLAT/olat3. All user data is located in the olatdata directory which should not be located in the PATH_TO_YOUR_OLAT/olat3. The olatdata directory and the database won't be changed when installing the new code. Example linux shell commands:

#cd PATH_TO_YOUR_OLAT
#rm olat3 (if olat3 is a symlink, otherwise: mv olat3 olat3.0.0)
#unzip OLAT-3.0.1.zip
#ln -s OLAT-3.0.1-FINAL-20040804 olat3
#cd olat3
#vi build.properties
#ant config-all
#ant install

Update your database manually according to PATH_TO_YOUR_OLAT/olat3/database/alter_3_0_0_to_3_0_1.sql