Reading email in the browser with Horde
Taming the Horde
The Horde webmail project has been around a long time on the free software market. Chuck Hagenbuch launched it 13 years ago for his university back when the Internet and PHP – the language in which Horde is programmed – were still cutting their teeth. Today, Horde is a free software project licensed under the GPL.
From its beginnings as a web mailer, the project has developed into mature groupware with PHP underpinnings comprising many modules and an awe-inspiring set of configuration options. But, feature richness and flexibility come at a price: Configuring the system is complex. If you are unfamiliar with the configuration, you can easily lose your way in the plethora of settings.
Package Management
The installation itself isn't too complicated, although it will probably surprise administrators to see that it differs from the standard procedure of extracting a packed archive in the web server directory. As of Horde version 4, released in April 2011, you need PEAR to install the software; specifically, the PEAR package manager that comes with PHP installations. Horde is a very modular program, split into more than 100 packages, so the PEAR package manager delivers Horde's many programs via a PEAR server [1]. This modularity allows administrators to select various features and applications at install time and necessitates the production of high-quality code because interfaces between the modules force the developers to comply strictly to program specifications.
Most of the modules make up the PHP framework for developing web applications. Webmail, the calendar, the notepad, and the wiki are all modules that build on this framework. If users need a feature that the existing Horde applications do not provide, you can easily program a module to match on the basis of the framework and integrate the module with your existing Horde installation.
The average web application comprises just a single module, and you don't need any tools to supply it as an archive, install it as an administrator, or add updates at a later stage. Of course, this is not true if you have several dozen packages to manage: A package manager is essential in this case. Whereas various Linux distributions have Apt, RPM or Yast, Ruby its "Gems," and Python its "Eggs," PHP has its PEAR packages.
Installation
This section assumes the administrator doesn't want to install packages on the underlying Linux distribution but use a separate directory for the Horde installation instead. This setup prevents the modules installed by PEAR from colliding with packages belonging to the distribution's own package manager. It also ensures that you install in a clearly demarcated environment from which the web server can deliver Horde to your users.
The official documentation for the Horde [2] installation covers potential alternatives to the approach I will be using in this article. If you are installing Horde, it is always a good idea to read the document first; it goes into far more detail than I can provide in this article.
The distribution on which you be install Horde must support both PHP and PEAR. PHP includes the latter by default. The first step is to create the directory in which you will be installing Horde:
mkdir /var/www/webmail
The directory path is freely selectable, although it will typically be under the root directory that the web server provides. The official installation guide assumes /var/www as the default.
PEAR can install itself and generate an environment that is independent, with the exception of its dependency on PHP. You can use this ability to complete the Horde installation in a way that is independent of the overall system to a great extent. The following command creates a new PEAR environment in the directory you just created:
pear config-create /var/www/webmail /var/www/webmail/pear.conf pear -c /var/www/webmail/pear.conf install pear
This step still uses the PEAR command belonging to the underlying distribution, typically /usr/bin/pear. In the following installation steps, I will use the variant I just installed (i.e., /var/www/webmail/pear/pear -c /var/www/webmail/pear.conf …).
The Horde packages are available from the Horde project's PEAR server [1]. The newly installed PEAR environment is still blissfully unaware that this package provider exists. But you can issue the following command to tell PEAR to find and remember the server:
/var/www/webmail/pear/pear -c /var/www/webmail/pear.conf channel-discover pear.horde.org
The first step in the actual installation, now installs a helper script that lets the administrator define the target directory for the Horde installation:
/var/www/webmail/pear/pear -c /var/www/webmail/pear.conf install horde/horde_role /var/www/webmail/pear/pear -c /var/www/webmail/pear.conf run-scripts horde/horde_role
The script prompts you for the installation destination, and you can reply by typing /var/www/webmail. Following this, the installation proper can begin:
/var/www/webmail/pear/pear -c /var/www/webmail/pear.conf install -a -B horde/webmail
The script terminates with the following message:
... install ok: channel://pear.horde.org/webmail-4.0.6
The directory Horde uses to deliver existing static content in an efficient way is /var/www/webmail/static/. The web server needs write permissions for this directory. Assuming the server is running on the www-data user account, the following two lines will set this up for you:
chown www-data\: /var/www/webmail/static/ chmod 644 /var/www/webmail/static/
The final step is to run the automated configuration script that generates the basic configuration. The script needs a database for this under normal circumstances. In the example, this will be a MySQL database that you create by issuing the
CREATE DATABASE webmail;
command. A matching webmail user with write privileges for the database is also needed. Besides this, PHP naturally needs to have the module for database communication. For MySQL, the Debian package for this is php5-mysql. Once these conditions are fulfilled, you can launch the configuration script as follows:
PHP_PEAR_SYSCONF_DIR=/var/www/webmail php -d include_path= /var/www/webmail/pear/php /var/www/webmail/pear/webmail-install
The script issues a number of prompts (Listing 1), initializes the database and creates an initial configuration file in /var/www/webmail/config/conf.php.
Listing 1: Initial System Configuration
Installing Horde Groupware Webmail Edition
Configuring database settings
What database backend should we use?
(false) [None]
(mysql) MySQL / PDO
(mysqli) MySQL (mysqli)
(pgsql) PostgreSQL
(sqlite) SQLite
Type your choice []: mysql
Request persistent connections?
(1) Yes
(0) No
Type your choice [0]: 1
Username to connect to the database as* [] webmail
Password to connect with
How should we connect to the database?
(unix) UNIX Sockets
(tcp) TCP/IP
Type your choice [unix]: unix
Location of UNIX socket [] /var/run/mysqld/mysqld.sock
Database name to use* [] webmail
Internally used charset* [utf-8]
Use SSL to connect to the server?
(1) Yes
(0) No
Type your choice [0]:
Certification Authority to use for SSL connections []
Split reads to a different server?
(false) Disabled
(true) Enabled
Type your choice [false]:
Writing main configuration file... done.
Creating and updating database tables... done.
Configuring administrator settings
Specify an existing mail user who you want to give
administratorpermissions (optional): admin@example.com
Writing main configuration file... done.
Thank you for using Horde Groupware Webmail Edition!
These settings install Horde in a separate environment. This setup means that the PHP code of the application is not in the normal system include path, and PHP would be unable to execute the application without you pointing the way. To do so, you need to add a couple of lines to the web server configuration (on Debian, the configuration is in the /etc/apache2/sites-available/default file):
<Directory "/var/www/webmail/"> php_value include_path /var/www/webmail/pear/php SetEnv PHP_PEAR_SYSCONF_DIR /var/www/webmail </Directory>
After restarting the web server, you can stop working at the command line for a while and complete the remaining configuration steps in the web browser.
At this point, typing the correct URL (e.g., http://www.example.com/webmail/) should show you the login dialog (Figure 1). If this is not the case, take a look at the results of the Horde test script. You can enable the script by setting the configuration $conf['testdisable'] = false; in the /var/www/webmail/config/conf.php file and then typing the URL http://example.com/webmail/test.php. The script will reveal potential configuration errors or missing dependencies. However, changing to the PEAR-based installation has definitely helped to avoid potential dependency-related errors compared with earlier versions.
Assuming that the IMAP server resides on the same machine as the webmail installation, email users should now be able to log in. If the IMAP server resides on a remote system, you also need to specify the hostname. Doing so means editing the /var/www/webmail/imp/config/backends.local.php file, which will look like Listing 2, apart from the hostname, of course.
Listing 2: Configuring the IMAP Server
01 <?php 02 03 $servers['imap'] = array( 04 // ENABLED by default 05 'disabled' => false, 06 'name' => 'IMAP Server', 07 'hostspec' => 'imap.example.com', 08 'hordeauth' => false, 09 'protocol' => 'imap', 10 'port' => 143, 11 // Plaintext logins are disabled by default on 12 // IMAP servers (see RFC 3501 [6.2.3]) 13 'secure' => 'tls', 14 'maildomain' => '', 15 // 'smtphost' => 'smtp.example.com', 16 // 'smtpport' => 25, 17 'cache' => false, 18 );
Initially, only the /var/www/webmail/imp/config/backends.php file exists, which you are not allowed to edit. PEAR overwrites it when you install updates. This rule applies to all configuration files in the config folder, except the conf.php files.
Configuration
Once you have successfully navigated the installation, you will probably be dumbfounded by the sheer number of configuration options the applications offer. The basic Horde application alone has 35 pages of configuration and up to 20 options per page (Figure 2). The other applications also have a number of separate configuration pages.
Luckily, things aren't as bad as they seem. For one thing, the installation will actually work after calling webmail-install. The basic settings don't need to be changed. You should only depart from them if the corresponding option obviously needs a different setting in your own infrastructure. Above all, resist the urge to experiment if this is your first installation.
My advice to administrators at this point is to use the static directory created earlier to cache CSS and JavaScript. This feature is disabled by default to avoid errors if the directory isn't writable for the web server. All you need to do is enable the Cache System, CSS Caching, JavaScript Caching, and Theme Caching with the default options.
The web-based configuration option helps to specify the central conf.php file in /var/www/webmail/config and the conf.php configuration files for the applications below /var/www/webmail/{imp,ingo,kronolith,mnemo,nag,turba}/config. Each of these configuration directories houses another group of special configuration files written in PHP, each of which offers many tuning options. I will just point you to the documentation for the files themselves here, because anything else is beyond the scope of this article.
Performance
For smaller systems that only serve a couple hundred users, a Horde installation without any additional optimization steps will be fine, but with a dash of optimizations, Horde can easily handle larger groups of users. For example, Portuguese Telco SAPO serves several million customers with Horde installations spread across multiple servers.
Horde developers collect basic performance tips that will be useful in most Horde installations in a separate document [3], which reflects the combined wisdom of their many years of experience tuning Horde. The instructions mainly relate to how you can optimize PHP and Apache for maximum speed and minimum resource requirements.
At the top of the list is a bytecode alternative PHP cache (APC) [4] for the web server. This module is mandatory for PHP applications on a production server; after all, it doesn't make much sense for the web server to reparse the PHP code for each request and convert it into bytecode if the code doesn't change very often – typically only after an update. APC ensures the conversion only occurs once – with significant time savings. After an update, you do need to restart the web server and thus delete the APC cache to avoid continuing to use the old code. On Debian, installing the APC cache is a three-liner:
apt-get install php-apc echo "apc.stat=0" >> /etc/php5/conf.d/apc.ini /etc/init.d/apache2 restart
The use of fast caching solutions like Memcached [5] is also recommended. Horde can cache information it has served up previously in many places and thus serve it up far more quickly to users in case of repeat access. On Debian, you just need to install the memcached and php5-memcache packages and modify the Cache System configuration in the Horde administration front end.
Besides these generic optimization options for web servers or PHP, Horde also supports a number of effective optimizations. Autoload caching is one important option. The PHP autoload mechanism allows you to link class definitions with a file path. If the PHP compiler discovers an unknown class name, it converts the name to a file path and loads the corresponding file with the code it contains to define the missing class. The Horde code typically reconverts the name for each request it receives, which eats up CPU cycles, but the class name to file path mappings usually change only after a software update. The application will thus benefit from caching just as with APC. This kind of acceleration is easily enabled by installing the required packages:
/var/www/webmail/pear/pear -c /var/www/webmail/pear.conf install -a -B horde/horde_autoloader_cache
The list of potential optimizations is far longer than I can cover within the scope of this article. Because of this, I strongly recommend the optimization document mentioned previously, if you are really looking for a high-performance Horde installation.
A similar document is available [6] that focuses on optimizing IMAP communication. It provides an essential knowledge base for administrators of heavily frequented servers.
Updates
Once you've made it this far, you might want to take a break, but the threat of upgrades is always looming around the corner, especially when a new version of Horde is released. The application comprises several dozen modules, so you might think updating them could easily take a whole day, but that's not true. Updating the application is actually a one-liner at the command line:
pear upgrade -a -B horde/webmail
After this, you might need to update the configuration in the administration front end and migrate the database to the newer schemas. However, both processes are automated and just take a couple of clicks.
By now, you will understand why the developers decided to use PEAR. In less than one year, the Horde team has turned out more than a thousand new releases via the PEAR server – and without provoking flame wars in the mailing lists.
On the contrary, the developers can now publish releases with bugfixes or security updates earlier than ever, and administrators will find it much easier to install them. Doing without the typical tar archives and the automated database update lowers the bar for what used to be a very trying update process. At the same time, the installation becomes far more secure; after all, an obsolete web mailer is the perfect target for spammers and can easily fall into the wrong hands.
Hosted
If the installation procedure is still too long and complicated for your liking and you prefer to do without tweaking dozens of options, many options can reduce the overhead.
For example, you could opt for an installation based on a Linux distribution. The effort involved for the administrator is typically less with this approach because the maintainers will have modified the packages to reflect their own distribution wherever possible. Additionally, this kind of installation uses the distribution's own package manager and avoids the need to use PEAR. This setup could be an attractive proposition for some administrators. Horde 3 comes in practical packages for large to mid-range distributions, whereas Horde 4 is still trying to catch up. Right now, packages are available for openSUSE and FreeBSD, and Debian will be available in the near future.
That said, you will not save on configuration overhead using this method. On the other hand, you do still keep control of the system and can customize it to your heart's content. If you are prepared to compromise flexibility, you can choose a preconfigured solution like Plesk or cPanel. Both are included in many web hosting offerings and allow rapid installation of a mail server with a Horde webmail front end. Both solutions only support Horde 3 right now, although work is in progress on Horde 4 updates.
An even easier approach is to subscribe to an email package with preinstalled Horde webmail. One of Germany's largest providers, domainFACTORY, offers hosting packages of this kind. An alternative is appliance solutions, such as the Univention Groupware Server. In the latest version 3.0, Univention comes with Horde 4.0 as its webmail front end.
The Horde developers encourage this kind of "repacketizing." They focus on creating code and on delivering the software via PEAR in the most flexible state possible. This approach avoids the effort involved in parallel packetizing for various systems. The resulting packages provide a solid basis that lets other people repacketize Horde 4 with very little effort – to create a product or to integrate Horde into their own solutions.
User Settings
Horde isn't just flexible in terms of the server-side configuration but also offers users a number of settings. Compared with Horde 3, Horde 4 offers a slightly more clear-cut approach to handling the multitude of user options by distinguishing between basic and expert options. This approach hides most of the possible options initially, while letting users display the advanced options at the press of a button.
However, the administrator can also enable and disable individual options for the entire server. This process is achieved by means of the well-documented prefs.php files, which are located in the config folders for the individual applications.
Responsible administrators will estimate their users' requirements and reduce the number of settings as necessary. If you typically serve occasional web mailer users, many options can be weeded out. If users complain about the lack of options, you can easily reenable them later. However, if you typically serve power users, you can just leave the whole palette of options enabled.
Features
Although older Horde versions are mainly restricted to a static view with little JavaScript, Horde 4 comes with an Ajax interface that is similar to a normal desktop application in terms of usability (Figure 3).
Increasing numbers of users rely on their smartphones to access web applications. For devices like this with a touchscreen, the dynamic view doesn't lend itself to effective use, because the controls are too small (Figure 4).
Horde 3 previously offered a view optimized for WAP-capable cells. This option still exists, but it is hardly state of the art for today's smartphones. Thus, Horde 4 introduced a jQuery Mobile-based view. The view is touch-optimized and will run on Android, iPad, iPhone, and other devices. Additionally, Horde 4.0 offered read-only access, while Horde 4.1 also lets you compose messages in the view.
With more and more smartphone users joining the fray, synchronization options are also becoming more important. Whereas Horde 3 only offered SyncML for synchronizing calendars, address books, tasks, and notes, Horde 4 additionally supports ActiveSync, the Microsoft synchronization protocol from the Exchange camp.
In recent years, the use of SyncML has decreased because of technical issues, and ActiveSync is now dominating the market. From a technical point of view, this protocol seems to be far more stable, but it still entails licensing issues because of the Microsoft patent on the method. In the United States, or for servers hosted there, you need to purchase a license from Microsoft. Although the ActiveSync implementation in Horde 4.0 only synchronized the calendar, addresses, tasks, and notes, version 4.1 will probably also be able to use ActiveSync to synchronize email.
Webmail itself offers nearly all the functionality you would expect from a desktop mail client in the static and Ajax views: reply, forward, complex searches, tagging, address book integration, calendar integration, IMAP rights management, signatures, and spam filters. Users can prepare attachments on the web server instead of squashing them through the SMTP tunnel, thus allowing the receiver to download by clicking a link. Horde supports encryption with GPG or SMIME, although you should bear in mind that a web application cannot guarantee clean end-to-end encryption in this way.
When Horde 4.0 was released in April 2011, the Horde project shifted to a six-month release cycle. The first applications to be released were the core apps IMP (webmail), Ingo (mail filter), Kronolith (calendar), Turba (address book), Nag (task management), and Mnemo (note manager). In October 2011, the developers then ported a series of applications from Horde 3 to Horde 4. The list included Gollem (an online file manager), Passwd (for changing user passwords), Ansel (a gallery application), Whups (a ticket tracker), and Wicked (a wiki).
The introduction of Horde 4.1 in April 2012 was a major step for the developers: The extension of the smartphone view for mail has been completed. The calendar now supports advanced resource management, for example, of video projectors, meeting rooms, vehicles, and so on.
Additionally, plans are underway to allow Horde 4.1 to act as a CalDAV and CardDAV server and for the system to support cloud storage in combination with WebDAV functionality already available in Gollem.
The list of candidates for 4.1 also includes support for email synchronization by ActiveSync and the Kolab format. The latter allows users to save appointments, addresses, tasks, and notes in the mail server's IMAP store, instead of relying on an SQL-based database.
In Horde 4.0, the user will still find some rough edges in the GUI, while competitor products such as Roundcube impress with their contemporary design. The Horde developers are aware of this problem and will be looking to resolve it in release 4.1, which envisages a redesign of the interface. The statuses of the various releases accomplished and planned are detailed on the Horde wiki [7].
Conclusions
Horde offers a flexible webmail system with many features, to which groupware elements can easily be added. The flexibility and the feature scope can be challenging for the administrator on initial installation and during optimization, but once the system is running, maintenance is simple. Extensions can easily be developed thanks to the Horde framework.
If you need to support mobile users with smartphones, Horde can help you provide a wide spectrum of features. Horde developers are focusing on this area in particular. The biggest downside for users is the obsolete design, and the Horde developers will have to show whether they can improve this in version 4.1.