Mass Mail

When you need to manage mailing lists, Mailman is often the tool of choice. The program is not only mature and widespread, it also offers many functions in the framework of a convenient web interface. However, alternatives do exist. Mlmmj (Mailing List Management Made Joyful) is a less well-known representative of this category, but it's well worth looking into. In this article, I'll describe how the LibreOffice project uses Mlmmj.

Getting Started – Not Easy

During planning for the infrastructure of today's LibreOffice project, mailing lists played a central role in the community's main means of communication. The requirements were specific. Traditionally, the project has multiple mailing lists with the same names in different subdomains, for example, discuss@de.libreoffice.org and discuss@si.libreoffice.org; the list is moderated by numerous volunteers around the globe who all use email.

And, it was precisely these requirements that Mailman was unable to fulfill – moderation is mainly web-based, protected by the same password for all moderators, and mailing lists of the same name will not be supported until version 3.0 is released.

After evaluating several systems, Mlmmj was finally chosen. Because of its similarity to Ezmlm, which was used by OpenOffice.org at the time, it was predestined to win because the users didn't need to change their habits. Whereas Ezmlm relies on Qmail to the exclusion of all others, Mlmmj is compatible with the most popular mail servers, including the one used in this scenario, Postfix.

Installation

Installing Mlmmj was a simple thing on our lab system, a current Ubuntu 10.04 LTS. A simple apt-get install mlmmj puts the package on the server. Mlmmj runs on its own user context for security reasons – if nothing else. On Ubuntu, the package doesn't create a user account, but typing adduser --system mlmmj will do the trick.

Additionally, it makes sense to forward mails addressed to the mlmmj system user to the postmaster; depending on your mail system, you will want to use a .forward file or /etc/aliases for this. After creating the required spool directory as follows,

mkdir /var/spool/mlmmj && chown mlmmj:/var/spool/mlmmj

you have completed the installation process to the package. The next thing to look at is integrating Mlmmj with the mail system; the Mlmmj documentation page has a very useful guide to doing this. In this article, I will look at how to connect Mlmmj with the Postfix mail server.

The list addresses should ideally be stored in separate files, and separately from the rest of the mail system – for example, in /var/spool/mlmmj/virtual.regexp and /var/spool/mlmmj/transport.

To make sure that Postfix actually processes the files, you need to add the following lines to /etc/postfix/main.cf:

virtual_alias_maps = regexp:/var/spool/mlmmj/virtual.regexp
transport_maps = regexp:/var/spool/mlmmj/transport

You should also enable this option:

mlmmj_destination_recipient_limit = 1

Mlmmj binds with Postfix as a transport. To publish the service, you need to add another line to in /etc/postfix/master.cf (see Listing 1). This entry tells Postfix to call /usr/bin/mlmmj-receive for the Mlmmj transport; it then takes care of processing messages.

01 mlmmj   unix  -       n       n       -      -       pipe
02         flags=DORhu user=mlmmj argv=/usr/bin/mlmmj-receive -F -L /var/spool/mlmmj/$nexthop/

Finally, you need to tell Postfix about the virtual domain(s) for your mailing list, if you haven't done so already; you can use an entry in /etc/postfix/virtual to do this. Also remember to create aliases for abuse@ and postmaster@ to comply with the RFC; ideally, they should be unfiltered.

This is the only way to ensure that the right contacts are reachable in case of problems, and it saves you the trouble of problematic entries in blacklists. A correct entry in /etc/postfix/virtual looks like the one in Listing 2.

01 lists.admin-magazin.de VIRTUAL
02 abuse@lists.admin-magazin.de root
03 postmaster@lists.admin-magazin.de root

Finally, issue a postfix reload and a postmap /etc/postfix/virtual to complete integration with the mail system.

Aliases and Transports

Creating the first mailing list involves several steps, which will differ depending on the mail system that you use. For Postfix, you have to start by assigning a transport in the /var/spool/mlmmj/transport file. As the documentation tells you, the ideal format is:

/^(domain\.tld--listname).*$/ mlmmj:domain.tld/listname

In the case of the LibreOffice lists, the configuration looks something like Listing 3. To assign mail addresses to the aliases, you need a second file: /var/spool/mlmmj/virtual.regexp. The format is as follows:

01 /^(documentfoundation\.org--announce).*$/ mlmmj:documentfoundation.org/announce
02 /^(documentfoundation\.org--discuss).*$/ mlmmj:documentfoundation.org/discuss
03 /^(global\.libreoffice\.org--l10n).*$/ mlmmj:global.libreoffice.org/l10n

/^(listname.*)@(domain\.tld)$/ domain. tld--${1}

Listing 4 shows the configuration for the LibreOffice lists.

01 /^(announce.*)@(documentfoundation\.org)$/ documentfoundation.org--${1}
02 /^(discuss.*)@(documentfoundation\.org)$/ documentfoundation.org--${1}
03 /^(l10n.*)@(global\.libreoffice\.org)$/ global.libreoffice.org--${1}

Experienced administrators will have noticed that these entries define three mailing lists:

• announce@documentfoundation.org
• discuss@documentfoundation.org
• l10n@global.libreoffice.org

The basic path is /var/spool/mlmmj. The announce@documentfoundation.org list is thus located in /var/spool/mlmmj/documentfoundation.org/announce; and l10n@global.libreoffice.org is located in /var/spool/mlmmj/global.libreoffice.org/l10n.

After changing the virtual and transports files, you should issue a postfix reload to enable the configuration.

First Mailing List

Once the required aliases are in place on the mail system, you can set up the lists. The steps are independent of the mail server that you use. The focus here is the mlmmj-make-ml program, which expects the spool directory (with a subdomain in this case) and the list names (Listing 5) as parameters.

01 sudo -u mlmmj /usr/bin/mlmmj-make-ml -s /var/spool/mlmmj/documentfoundation.org -L announce
02 sudo -u mlmmj /usr/bin/mlmmj-make-ml -s /var/spool/mlmmj/documentfoundation.org -L discuss
03 sudo -u mlmmj /usr/bin/mlmmj-make-ml -s /var/spool/mlmmj/global.libreoffice.org -L l10n

The tool queries the subdomain, the address of the list owner, and the language to use for the list texts. Theoretically, the new list is ready to use now – an email to listname+help@subdomain should return the help text in response. The notice from mlmmj-list, saying that you need an entry in /etc/aliases, can be ignored in the Postfix setup in this example.

Important Details

The default settings are not sufficient to regular operations, however. Mlmmj stores the configuration in the control subdirectory of each list, that is, in /var/spool/mlmmj/documentfoundation.org/announce/control. Many small files – known as tunables in Mlmmj-speak – let you customize the lists to reflect your own needs. It is easy to enable changes to the file; you don't need to restart Mlmmj. The following sections explain the most important options.

The customheaders option stores the header lines that are added to the emails. You should also insert the headers recommended in the corresponding RFC (Listing 6). These headers set Reply-To to the list address, provide a contact address for error messages, and – most importantly – tag the messages as list mail to avoid "away messages" from being sent. The list headers give the mail clients and web services additional information on addresses for subscribing and unsubscribing and on the location of the archive.

01 Reply-To: discuss@documentfoundation.org
02 Errors-To: postmaster@documentfoundation.org
03 Precedence: list
04 List-Id: <discuss.documentfoundation.org>
05 List-Unsubscribe: <mailto:discuss+help@documentfoundation.org>
06 List-Archive: <http://listarchives.documentfoundation.org/www/discuss/>
07 List-Post: <mailto:discuss@documentfoundation.org>
08 List-Help: <mailto:discuss+help@documentfoundation.org>
09 List-Subscribe: <mailto:discuss+subscribe@documentfoundation.org>
10 List-Owner: <mailto:postmaster@documentfoundation.org>

Alternatively, the List-Post: NO op tion indicates that the list is an announcement list without the option for replying.

The prefix option indicates whether or not to insert a list ID before the subject, for example, [libreoffice-users]. The text must be added to the file in square brackets but without a trailing space.

You can also configure your approach to handling unsubscribed senders. If the file subonlypost exists, only registered users are allowed to write messages; Mlmmj will discard messages from unsubscribed addresses. If you instead want to present these messages to moderator, you will additionally need to create a modnonsubposts file. The content doesn't matter because Mlmmj only checks whether the file exists.

You can also configure whether to inform users of errors. By default, Mlmmj will send messages for all errors. However, if you have the notoccdenymails, noaccessdenymails, nosubonlydenymails, or nomaxmailsizedenymails files in place, Mlmmj does not warn in these specific cases: missing list address in the receiver field, access list-based blocking, unsubscribed postings, and so on. As a general rule, you'll want to reduce the use of notifications to avoid having your own mail server being used by a malicious third-party for spamming ("backscattering").

Incidentally, you can configure the maximum mail size, including the attachment, in the maxmailsize file. The access file, which defines restrictions for incoming mail, is also important. The following

discard ^X-Spam-Flag: YES
discard ^Subject:.*Auto Reply:
allow

specifies that messages tagged as spam by SpamAssassin, with Auto Reply: in the subject line should be discarded without a comment. List moderators and list owners need to be added to the moderators and owner files, respectively (note that there is no s at the end of owner). Multiple addresses are not a problem; however, it is advisable to have the same content in both files.

Another important feature is the ability to remove certain headers using the delheaders file. If you configure a static reply address for the list, you can remove a different sender configuration from incoming mail as follows:

Reply-To:

The ability to filter return receipts is far more important – in the worst case, any mailing list can be filled up with junk due to return receipts. The following headers are responsible for this action and should definitely be added to the delheaders file:

Return-Receipt-To:
Disposition-Notification-To:
X-Confirm-Reading-To:
X-Pmrqc:

It is also advisable to create a nolistsubsemail file; this prevents non-authorized users from being able to see the list of subscribers.

Mlmmj also provides a couple of additional files that are described in the readme file for the "tunables." For example, you could configure a different SMTP port to sign your emails before sending, and you can add footers to messages. More details of this will be provided later. Besides control, a number of other interesting files and folders exist, which are explained in Table 1.

Tools

Mlmmj gives administrators a number of tools that are useful for managing mailing lists. For example, mlmmj-list provides information on the lists on their subscribers (Figure 1). The following command line

sudo -u mlmmj /usr/bin/mlmmj-list -L /var/spool/mlmmj/global.libreoffice.org/users -c -d

counts the digest subscribers in the users list, for example. The following command

sudo -u mlmmj /usr/bin/mlmmj-list -L /var/spool/mlmmj/global.libreoffice.org/users -m

outputs the addresses of the moderators. To manually subscribe and unsubscribe recipients, you can use mlmmj-sub and mlmmj-unsub, respectively. For example, this command

sudo -u mlmmj /usr/bin/mlmmj-sub -L /var/spool/mlmmj/global.libreoffice.org/users -a archive@mail-archive.com -c

creates a list archive with The Mail Archive.

Archiving

Mlmmj sees itself as a mailing list manager only and thus will archive mail from all of its lists on the filesystem. It, however, does not include a special interface for doing so, instead forcing administrators to look for a separate solution. The likely candidates here are Hypermail and the slightly long in the tooth MHonArc. Alternatively, you can use Internet archives such as Mail Archive, Gmane, or Nabble.

Advantages

Mlmmj is configured entirely at the command line; a web interface does not exist. Settings are stored in control files as described with one file per option. The fact that the program does without special tools or XML formats makes it easy to automate scripting. One really big advantage is the fact that postings for the list are stored in individual files – one file per email. This approach offers some decisive advantages compared with Mailman: backups are far quicker, because you only need to back up the new files which are enumerated in ascending order. If you've ever had to back up a Mailman archive weighing in at several gigabytes – whether across the network or using USB/DVD – you will appreciate this feature.

Admittedly, the configuration described above doesn't appear entirely intuitive; however, with some scripting you can reduce it to a single command that lets you set up and configure new lists in next to no time. And, the ability to conveniently moderate the list by mail is an important bonus that Mailman at least doesn't support.

Problems and Solutions

Despite these advantages, Mlmmj lacks a couple essential functions. For example, the team quickly discovered that, although it offers options for blocking content, the tool doesn't have filters for message content. To put this another way, HTML mails or messages with attachments are either discarded or delivered without any modifications, and neither option is a good choice. The remedy is a self-programmed extension called PyMIME that strips the undesirable elements from the messages before delivering. The totally reprogrammed version, which will be available in the near future, even supports this depending on the sender, receiver, and MIME type.

PyMIME also plugs another gap. Although Mlmmj can add a footer to each message, it fails when asked to convert. If you add a local footer composed in UTF-8 to a mail that uses the ISO-8859-15 character set, non-standard characters are illegible. To resolve this issue, PyMIME converts the footer to the correct character set and can also handle templates with variables as the new version.

Managing the differences in subscription variants, because Mlmmj also supports a digest mode and a no-mail variant for users of, say Gmane or Nabble, besides regular mail delivery, also takes some getting used to. The program doesn't take into consideration the fact that most users will only choose one type, which leads to unintelligible unsubscribe addresses such as discuss+unsubscribe-digest@de.libreoffice.org, a fact that regularly causes confusion. This behavior will be improved in a future version.

Moderation itself is handled by mail. Moderators receive messages from unknown senders or with questionable content as an attachment – if they do not respond to the moderation address within seven days, the message expires. Unfortunately, the current version of Mlmmj truncates the moderation templates after 100 characters and is also unable to decode base64 messages, but the next version promises a solution for this. The community surrounding Australia maintainer Ben Schmidt seems to be agile, and implements suggestions promptly.

Conclusions

Mlmmj is still a relatively unknown project, but despite its somewhat complicated configuration, it is an interesting alternative to Mailman [2] and Sympa [3]. The tool's approach to configuration opens up some interesting options. Newcomers or postmasters who prefer a web-based configuration will need to look elsewhere.