Copyright © 2009 MELS - Multimedia & E-Learning Services, University of Zurich, Switzerland
Table of Contents
This documentation describes internas of User Activity Logging as well as Statistics in OLAT.
The user activity logging infrastructure changed substantially in 6.3. Prior to 6.3 user activity events used to be written into a file adjacent to each course. From 6.3 on all such user activity events are written into the database to allow easier processing later.
This change has several implications:
The old course log files have gone - during migration from 6.2 to 6.3 they are zipped and stored into the course' storage folder under old_course_logs.
All user activity logging events are now stored in one table: o_loggingtable. Depending on the traffic of the particular OLAT instance, this table can grow rather large. It is strongly recommended to monitor this table and keep the size reasonable. Possible solutions in this area include e.g. periodic cleanup of old data or compressing on a monthly basis and using merge tables.
It allows to process these logging information into meaningful statistics - both online as described in this documentation as well as offline for further research-type studies. This provides better e-learning controlling from both an author and an administrator's point of view.
Technically, the logging no longer happens via JMS and a single-service: RemoteAuditLogger has gone.
What used to be called 'course log file export' is still available with the new logging. The difference is that this time it is a select in the o_loggingtable compared to the copying of a file in 6.2. This means that a course log export now potentially takes longer - which is why there are restrictions on how many can do this (default is 2 per node simultaneously). It also implies that for this to be done in a performant way it might be necessary to use stored procedures (depending on the amount of data you have in your installation).
The format of the logging is no longer free text but rather structured - which means the developer needs to be aware of this and needs to follow this structure. This logging and statistics documentation describes how this is best done.
This chapter describes the User Activity Logging as it is implemented in 6.3. Note that the logging infrastructure has changed substantially as described in the introduction
To clarify terms used in this document here is a short description:
User Activity Logging: This is the term used for logging of any user action with the intension of later to be used for audit and statistics. This document concentrates on User Activity Logging.
Technical Logging: This is anything else which is logged via log4j into the good old olat.log file. This document does not describe anything about Technical Logging.
For the specification on User Activity Logging please refer to this document
There is a UML Diagram which gives a good technical overview of the core logging classes. Plus you might want to consult the JavaDoc for further API details.
The goal of logging is to fill the logging table with useful data. Here's a quick summary of what's written into the table. The full details can be found here. Note that the goal was to get data into one flat table without the need to later join the table with any other olat table. The idea was to get a snapshot of the information 'at the time the activity happens' - since course or user information might later change (e.g. a user might change university or a course' title might change etc). That's why the logging table is rather wide.
log_id, creationdate, sourceclass: these are technical fields describing a particular logging entry
session_id, user_id, username, userproperties[1..12]: these are session and user fields. The user properties is an array of 12 varchar rows which can be configured as to what is stored in them. Suggested usages are to store Shibboleth properties or other useful information about the user.
actioncrudtype, actionverb, actionobject, resourceadminaction, simpleduration: these fields describe the action itself. An action is a combination of actionverb + actionobject. The idea behind this is to allow to search for only actionverb or only actionobject. The actionverb is an enum - hence limited.
businesspath, targetresid/type/name, parentresid/type/name, grandparent.., greatgrandparent: These fields are resources and define the scope or context in which a particular action happened. Not all fields need to be set. The idea of having four such resource groups (target, parent, grandparent, greatgrandparent) is to limit the database size and also came as a result of the analysis of existing logging usages (where normally 4 is enough). Should the UserActivityLogger have more than 4 ILoggingResourceables available to store in the database, the 3 lowest and the uppermost one are stored, the others not.
There are a few things to remember when issuing a user activity log:
All logging happens via an IUserActivityLogger. The IUserActivityLogger contains context information such as Course, Node etc. During event handling - i.e. within the event() or dispose() method - the framework sets up a ThreadLocalUserActivityLogger which contains all available context information. This information is taken from the Controller's IUserActivityLogger.
Therefore each Controller has an IUserActivityLogger which is initialized at constructor time with context information that is available in the ThreadLocalUserActivityLogger. Plus any additional context information which is available in the Controller's constructor should be passed to the IUserActivityLogger.
Logging an event is done by calling IUserActivityLogger.log() method. Should there be additional context information available only at that time, it can be passed to the log() method directly.
Here is an example usage of how the logging is initialized with a course as context information.
// from RunMainController
public RunMainController(final UserRequest ureq, final WindowControl wControl, final ICourse course, final String initialViewIdentifier,
final boolean offerBookmark, final boolean showCourseConfigLink) {
super(ureq, wControl);
... other stuff ...
// initialize the logging with the course
addLoggingResourceable(LoggingResourceable.wrap(course));
... other stuff
// issue a logging action
ThreadLocalUserActivityLogger.log(CourseLoggingAction.COURSE_ENTERING, getClass());
}
Once the above is done, then during an event further logging actions can be issued as follows:
// from EditorMainController
public void event(UserRequest ureq, Controller source, Event event) {
... other stuff ...
ThreadLocalUserActivityLogger.log(CourseLoggingAction.COURSE_EDITOR_PUBLISHED, getClass());
... other stuff ...
}
Anything that is passed to the logging and stored in the logging table is treated via the interface ILoggingResourceable. The ILoggingResourceable contains the following properties which are stored to the logging table:
type: the type of the resource
id: identifier of the resource - usually maps 1:1 to the primary key of the resource
name: a verbose name of the resource
As mentioned above, you are encouraged to pass ILoggingResourceables to the IUserActivityLogger in a Controller's constructor.
There are factory methods with which to construct ILoggingResourcesables. Here are some examples:
org.olat.util.logging.activity.LoggingResourceable.wrap(ICourse); org.olat.util.logging.activity.LoggingResourceable.wrap(CourseNode); org.olat.util.logging.activity.LoggingResourceable.wrap(BGArea);
Once created, the ILoggingResourceable can be passed to the IUserActivityLogger as follows:
// within a Controller: addLoggingResourceable(LoggingResourceable.wrap(someResourceable)); // before creating a Controller with the intension to pass it to the Controller: ThreadLocalUserActivityLogger.addLoggingResourceable(LoggingResourceable.wrap(anotherResourceable));
The ILoggingAction describes the actionverb and actionobject as well as what the expected ILoggingResourceables to be stored along with such that logging action are. The idea behind this class is to have a well defined place (subclasses of ILoggingAction) where everything about the existing logging actions is defined. This can be useful when determining what statistics can be done, which parts of OLAT are logged and where more logging might be required. In short, this is a in-code documentation of the logging actions.
Therefore whenever you want to log something via the activity logger, you need to have an ILoggingAction (along with a set of ILoggingResourceables which were collected by the Controller's IUserActivityLogger - or when in a Manager in the ThreadLocalUserActivityLogger). The main log method therefor looks as follows - note that the main info you need to pass to the log() method is the ILoggingAction:
// from IUserActivityLogger public void log(ILoggingAction loggingAction, Class callingClass, ILoggingResourceable... loggingResourceInfosOrNull);
Besides being a container for actionverb and actionobject, the ILoggingAction contains a paranoia safety check for ILoggingResourceables: The ILoggingAction must be given a ResourceableTypeList. The latter is the specification of which ILoggingResourceables are mandatory, optional or not allowed when logging this particular ILoggingAction. This information is then used at runtime to check if the provided set of ILoggingResourceables is correct and issue WARNs and ERRORs if that's not the case. This should allow to - over time - find both errors in this ILoggingAction definition but also prevent you from a situation where you 'thought that the ICourse' was available in the Controller XY - but actually at runtime it is not.
As mentioned in the What's New chapter, the log file export now looks a bit different to 6.2. The logging data is now longer available in its own pre-customized log file (course_admin_log, course_statistics_log etc) but has to be extracted from the logging table directly at user's request. The log file export must take care of limiting the data which is presented to the user (e.g. anonymizing any user information, only providing non-sensitive data etc). It also takes into account, whether a logging action is of type resourceadminaction or not and exports one or the other.
In order to support huge logging tables with large amounts of exported data, there is an implementation of the log export which uses stored procedures of MySQL. If you run into scalability issues with the log file export you might want to consider using stored procedures as well. The functionality is encapsulated in the
org.olat.course.statistic.ExportManager
which is configured in the following spring file:
olat3/webapp/WEB-INF/src/serviceconfig/org/olat/course/statistic/_spring/olatdefaultconfig.xml
There are currently three implementations of ICourseLogExporter (of which one is to be configured in the above mentioned olatdefaultconfig.xml:
SimpleLogExporter: This one is uses Hibernate, is therefore the most generic one but has the downside of not being very performant.
SQLLogExporter: This one is configured with some SQL (therefore DB dependent) which it executes directly, bypassing any Hibernate. This allows logging data to be extracted from a slave DB (should the data not be available on the master DB)
UZHStoredProcedureLogExporter: This is a rather specific implementation used at UZH where a stored procedure is used directly in the slave database for performance reasons.
As with logging of any sort, it is important to maintain the resulting data. Logfiles are typically logrotated and eventually deleted. The same should be applied to data in the logging table.
Besides that, an administrator of an OLAT instance needs to have an idea of how much traffic is expected and whether this traffic can be handled by the chosen infrastructure.
For smaller installations it should be no worries to have the logging table in the main database. With bigger installations though, it is recommended to think about master/slave setups with e.g. a BLACKHOLE engine at the master side and MERGE tables with COMPRESS tables at the slave side (in MySQL). This document contains a drawing of how this can be achieved.
This chapter describes how statistics are made available in OLAT, what this means from a data(base) point of view and how you can extend the statistics to create your own.
Statistics in OLAT are implemented in a pluggable way. This means that you can create additional kinds of statistics or exclude some of the existing ones if you so wish.
There are two areas which are pluggable:
GUI: There is now a new tool for a course called 'Statistics'. In there you can plug in additional statistics.
Updater: Each statistic plugin needs a way to update its table with data from new logging actions. This part is bundled into a updater.

The GUI is integrated into OLAT via the ExtManager and a subclass of GenericActionExtension. Adding a new kind of statistic is done by adding a section in the olat_extensions.xml, providing a StatisticActionExtension with a Controller classname and a IStatisticManager implementation (see existing olat_extensions.xml for examples).
The Controller normally doesn't have to be extended and one can usually use the existing - generic - StatisticDisplayController. The StatisticDisplayController has a IStatisticManager which it uses for customizing the ColumnDescriptors it uses to display the table as well as for generating the actual statistics. The result of the statistics is encapsulated in its own object called StatisticResult.
Each statistic needs an IStatisticUpdater which is capable of updating the statistics given additional logging actions in the o_loggingtable. The way this update is done is entirely up to the IStatisticUpdater.
The configuration of the IStatisticUpdaters is done in the following file:
olat3/webapp/WEB-INF/src/serviceconfig/org/olat/course/statistic/_spring/olatdefaultconfig.xml
The suggested best practice is to write an updater which is capable of adding new statistic information to an existing table - aka incremental update.
Incremental updates can be further optimized by adding an IStatisticUpdaters (at the first position in the config) which does a
SELECT * FROM o_loggingtable WHERE ACTIONVERB='launch' AND ACTIONOBJECT='node' AND creationdate>...;
The result of this select can be stored in its own temporary delta table which all other IStatisticUpdaters then read their data from - rather than having each of a number of IStatisticUpdaters doing a SELECT in the 'big' o_loggingtable.
As mentioned here big installations of OLAT should consider using Master/Slave/Blackhole/Merge/Compress Setups for the o_loggingtable and incremental IStatisticUpdaters for using a temporary delta table for the statistics.