34,79 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Many businesses want to run their email servers on Linux for greater control and flexibility of corporate communications, but getting started can be complicated. The attractiveness of a free-to-use and robust email service running on Linux can be undermined by the apparent technical challenges involved. Some of the complexity arises from the fact that an email server consists of several components that must be installed and configured separately, then integrated together.
This book gives you just what you need to know to set up and maintain an email server. Unlike other approaches that deal with one component at a time, this book delivers a step-by-step approach across all the server components, leaving you with a complete working email server for your small business network.
Starting with a discussion on why you should even consider hosting your own email server, the book covers setting up the mail server. We then move on to look at providing web access, so that users can access their email out of the office. After this we look at the features you'll want to add to improve email productivity: virus protection, spam detection, and automatic email processing. Finally we look at an essential maintenance task: backups.
Written by professional Linux administrators, the book is aimed at technically confident users and new and part-time system administrators. The emphasis is on simple, practical and reliable guidance.
Based entirely on free, Open Source software, this book will show you how to set up and manage your email server easily.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 515
Veröffentlichungsjahr: 2009
Ähnliche
Table of Contents
Linux E-mail
Alistair McDonald
Carl Taylor
Magnus Back
David Rusenko
Ralf Hildebrandt
Patrick Ben Koetter
Ian Haycox
Linux E-mail
Set up, maintain, and secure a small office e-mail server
Copyright © 2009 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: June 2005
Second edition: November 2009
Production Reference: 1051109
Published by Packt Publishing Ltd.
32 Lincoln Road
Olton
Birmingham, B27 6PA, UK.
ISBN 978-1-847198-64-8
www.packtpub.com
Cover Image by Vinayak Chittar ( <[email protected]>)
Credits
Authors
Ian Haycox
Alistair McDonald
Magnus Bäck
Ralf Hildebrandt
Patrick Ben Koetter
David Rusenko
Carl Taylor
Reviewers
Patrick Chan
Aric Pedersen
Acquisition Editor
David Barnes
Development Editor
Ved Prakash Jha
Technical Editors
Gaurav Datar
Neha Patwari
Editorial Team Leader
Gagandeep Singh
Project Team Leader
Lata Basantani
Project Coordinator
Poorvi Nair
Proofreader
Lesley Harrison
Indexer
Rekha Nair
Graphics
Nilesh Mohite
Production Coordinator
Aparna Bhagat
Cover Work
Aparna Bhagat
About the Authors
Ian Haycox is a freelance IT consultant based in France and actively contributes to open source projects. He has twenty-five years of software development experience in the enterprise integration, telecommunications, banking, and television sectors.
Ian has a degree in Computer Science from the University of Hertfordshire, UK, and now runs his own web design company (http://www.ianhaycox.com/) and Linux programming consultancy.
My thanks to Debbie for supplying me with copious amount of coffee and cheese sandwiches.
Alistair McDonald is a software developer and IT consultant. He has worked as a freelancer in the UK for 15 years, developing cross-platform software systems in C, C++, Perl, Java, and SQL. He has been using open source software for over 20 years and implementing systems using it for the past 10 years.
Last year, he gave up his freelance career and joined JDA Software, working in a technical role in their Service Industries division.
Alistair is also the author of the book SpamAssassin: A practical guide to integration and configuration, published by Packt .
I would like to thank my wife Louise for the support she has given me throughout the writing of all my books.
Magnus Bäck has been playing and working with computers since his childhood days. He is interested in everything in the computer field, from digital typography and compilers, to relational databases and UNIX. His interests also include e-mail services, and he is an active contributor to the Postfix mailing list. Besides computers, he enjoys photography, cars, and bicycling.
Magnus holds a Master's degree in Computer Science and Engineering from Lund Institute of Technology, Sweden, and currently works with software configuration management for mobile phone software at Sony Ericsson Mobile Communications.
Ralf Hildebrandt is an active and well-known figure in the Postfix community, working as a Systems Engineer for T-Systems, a German telecommunications company.
He speaks about Postfix at industry conferences and hacker conventions, and contributes regularly to a number of open source mailing lists. Ralf Hildebrandt is the co-author of The Book of Postfix.
Patrick Ben Koetter is an active and well-known figure in the Postfix community, working as an Information Architect. Patrick Koetter runs his own company, consulting and developing corporate communication for customers in Europe and Africa.
He speaks about Postfix at industry conferences and hacker conventions, and contributes regularly to a number of open source mailing lists. Patrick Koetter is the co-author of The Book of Postfix.
David Rusenko was born in Paris, France, and spent most of his childhood overseas. He began working as a freelance Web Designer in 1996 and had his first experience with open source, a box copy of Red Hat 5.2, shortly after in 1999. After six years and as many versions of Red Hat, he now creates appealing web pages and devises solutions implementing high availability through clustering and alternate security models.
He founded Aderes (http://www.aderes.net) in 2001, a company that provides e-mail and web-based security solutions. His search for an appropriate Webmail Platform for the company led him to SquirrelMail. Initially managing all aspects of the business—from the technical concerns to customer support gave him the experience that he now contributes to the Webmail chapter of this book.
David has studied both, Information Sciences and Technology (IST) and Management Information Systems (MIS) at the Pennsylvania State University. He speaks English and French fluently, and is conversational in Arabic. During his free time and vacations, he enjoys scuba diving, backpacking, playing racquetball, and playing electronic music records.
Carl Taylor has worked over 20 years in the IT industry and has spent the majority of that time working on UNIX type systems, mainly communications or office automation projects. He was an early user of the UseNet network and taught himself to program in C through working on a variety of open source software. His experience covers roles including pre and post sales support, product development, end user training and management.
Carl now runs his own web solutions development company "Adepteo", where they specialize in intranet and workflow products building on the best open source applications available. Whilst not working or looking after his children, Carl is something of a dance addict and is currently learning Latin Ballroom and Salsa.
About the Reviewers
Patrick Chan is a programmer at Computer Bank, a not-for-profit organization that recycles and distributes donated computers to disadvantaged individuals and community groups.
He has used Linux for quite a number of years, and has fond memories of starting off learning Linux as a newbie using the Gentoo distribution. His favorite tools include vim, GNU Screen, Z shell (zsh), Secure Shell (SSH), and Mutt.
Aric Pedersen is the author of cPanel User Guide and Tutorial (ISBN 978-1-904811-92-3) and Web Host Manager Administration Guide (ISBN 978-1-904811-50-3), both written for Packt Publishing. He also served as a reviewer for CUPS Administrative Guide (ISBN 978-1-84719-258-5), published by Packt Publishing.
Aric has over 8 years of experience working as a System Administrator. He currently works for Hostdime.com, the world-class web host; and also for Netenberg.com, makers of Fantastico, the world's most popular web script installer for cPanel servers.
I would like to thank Mike Kahn for all of his assistance over the past few years and also my good friend, Capt John "Jack" Grimes, Esq. USAF JAG Corps, who is the best friend a fellow could hope for, and his new wife, Kristin, who has shown incredible fortitude by marrying Jack (*smile*). I don't want to forget Francene Brown who is a good friend and a straight shooter (so rare to find these days).
Finally, I'd like to thank my mother and Allen, because without them, nothing I've done would have been possible.
Preface
Many businesses want to run their e-mail servers on Linux for greater control and flexibility of corporate communications, but getting started can be complicated. The attractiveness of a free-to-use and robust e-mail service running on Linux can be undermined by the apparent technical challenges involved. Some of the complexity arises from the fact that an e-mail server consists of several components that must be installed and configured separately, then integrated together.
This book gives you just what you need to know to set up and maintain an e-mail server. Unlike other approaches that deal with one component at a time, this book delivers a step-by-step approach across all the server components, leaving you with a complete working e-mail server for your small business network.
What this book covers
Chapter 1: Linux and E-mail Basics takes you through the essential elements of a Linux e-mail server and the network and mail protocols that make e-mail possible. Like it or not, running a Linux e-mail server does require some understanding of the underlying networking, and this chapter is where you will start to get that understanding. This chapter explains the benefits and disadvantages of running your own e-mail server and provides some guidance on hardware sizing for a typical organization.
Chapter 2: Setting Up Postfix speaks about basic Postfix setup. Postfix is our chosen Mail Transfer Agent (MTA), which forms the heart of any e-mail server. The MTA is responsible, among other things, for moving messages between the various mail servers on the Internet.
Chapter 3: Incoming mail with POP and IMAP covers what to do with incoming e-mails. It will show you how to set up IMAP and POP access to mailboxes. This means users will be able to send and receive messages using their familiar e-mail clients.
Chapter 4: Providing Webmail Access shows how to set up webmail access using SquirrelMail. This will give users an easy, out-of-office access to their e-mail.
Chapter 5: Securing Your Installation looks at how your installation can be secured to prevent misuse of your users' data and the e-mail facility itself.
Chapter 6: Getting Started with Procmail discusses the basics of Procmail and gets you familiar with the various files that Procmail uses to load recipes, the core principles of filtering, and the options available.
Chapter 7: Advanced Procmail explores Procmail and explains a large number of services and a large amount of functionality that it can provide in getting mail under control. It also discusses the advanced features of Procmail and their benefits.
Chapter 8: Busting Spam with SpamAssassin shows the use of SpamAssassin in conjunction with Procmail to filter out the wide range of spam that afflicts the modern e-mail user.
Chapter 9: Antivirus Protection shows another way to protect users from rogue e-mail—this time the spread of e-mail viruses. Using ClamAV you can scan mail for viruses and schedule tasks to maintain an up-to-date antivirus database.
Chapter 10: Backing up your System will show you how to protect all your hardwork by backing up not only the e-mail itself, but also all of the configuration options that make up your e-mail server. Examples are provided to create an automated backup schedule to minimize data loss. Of course, you'll also learn how to restore data from these backups.
Who this book is for
This book is aimed at beginner or intermediate level System Administrators in small businesses, who want to set up a Linux-based e-mail server without spending a lot of time in becoming expert in individual applications.
Basic knowledge of Linux is also expected.
Conventions
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, along with an explanation of their meaning.
Code words in text are shown as follows: " The configuration file entry that you need to modify is DatabaseMirror.
A block of code is set as follows:
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
Any command-line input or output is written as follows:
New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "Save the file using the browser (normally, the File menu has a Save as option)."
Note
Warnings or important notes appear in a box like this.
Note
Tips and tricks appear like this.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or e-mail <[email protected]>.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book on, see our author guide on www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration, and help us to improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the let us know link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata added to any list of existing errata. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or web site name immediately so that we can pursue a remedy.
Please contact us at <[email protected]>with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
Questions
You can contact us at <[email protected]>if you are having a problem with any aspect of the book, and we will do our best to address it.
Chapter 1. Linux and E-mail Basics
If you are one of those thousands of system administrators who manage the networks and computers of small to medium-sized companies and you are thinking of hosting your own e-mail service, this book is for you.
We will start with the most basic components of an e-mail system. Together those components will allow your users to send or receive mail to or from anyone on the Internet. This might be all you need, but many companies also want to provide their users with an accessible webmail service that people can use from home or when they are on the road. Another feature that many people unfortunately cannot be without today is proper protection against viruses spread via e-mail as well as the filtering of spam messages.
We will also cover the most important aspects of security to prevent unauthorized or malicious use of the server. We will then discuss how to retain an archive of all e-mails received or sent by the server. Finally, we shall describe a process to backup and restore the server to protect all messages against data loss.
This book will cover the major features of the software in question, which will give you a solid foundation to work from.
By the end of this book, you will have a functioning e-mail server suitable for most small companies.
As the technical platform for our endeavor, we have chosen the GNU/Linux operating system and a proven selection of free software tools that will help us achieve the goal of a secure and reliable e-mail server for smaller companies. The tools we have chosen are widely known and used, written by software professionals, and are supported by a large community of users.
In this very first chapter of the book, we start with what you need to know before you even start working on your server.
By the end of this chapter, you will have a basic understanding of the main components required to run an e-mail server.
Why manage your own e-mail server
Most Internet Service Providers (ISPs) already give customers the ability to send and receive e-mail on their servers, so why would we want to own and manage it by ourselves? As you are after all reading this book, you may already have your reasons, but let us examine this question and some possible answers to it.
The most important reason for hosting and managing your own e-mail server is control. For many organizations, e-mail is an important part of the Information Technology infrastructure. Keeping control over your e-mail has many advantages.
More control requires more responsibility and more knowledge, and that is where this book comes in.
These hopefully compelling arguments aside, there are also downsides to hosting your own e-mail server. This is a task that requires a certain level of knowledge and commitment, and so should not be undertaken by everyone. With your own server, you are not only responsible for the service you provide to your users, but you also have a responsibility towards the whole Internet community. An ill-configured e-mail server can help worms and spam to spread, which is not only is a disservice to the community but can also get your server blacklisted. Even though a properly set up server can run for years without requiring much maintenance, you must keep yourself reasonably updated and be prepared to act upon new threats that may arise. This is not meant to scare you off, but just to make you think carefully before embarking on this project.
What you need to host an e-mail server
Your server needs to be available through a permanent Internet connection with a fixed IP address. In theory, it is possible to run an e-mail server with a non-fixed (dynamic) IP address but it will not be reliable when the IP address is changed, and you will risk losing messages. With a dynamic IP address, you will also face a bigger risk of being put on one of the blacklists for dynamic IP address ranges.
If you are serious about running an e-mail server, get a decent business-class Internet connection. These are relatively inexpensive these days, and investing in one will save a lot of trouble later on. E-mail traffic does not depend on high bandwidth, so the capacity of a simple DSL line should be more than adequate.
Even though you will need a fixed IP address, you do not necessarily need a public IP address dedicated to the mail server. If your company only has a few external IP addresses and uses private RFC 1918 addresses (192.168.x.y) on the inside with a Network Address Translation ( NAT) router, this is not a problem. The NAT router connects the private network to the rest of the world, and it is possible to set up the router to forward the ports required by the e-mail services to the internal e-mail server.
The next table shows which TCP ports are most likely to be used for this.
Port
Service
25
Simple Mail Transfer Protocol (SMTP)
110
Post Office Protocol (POP)
143
Internet Message Access Protocol (IMAP)
993
IMAP over TLS
If employees want to access their messages from home or from the road, all that is required is to make sure that no firewall is blocking access to the required ports, and that the NAT router (if any) forwards these ports correctly. If users want to send messages via the e-mail server, some extra configuration will be necessary to allow the host to perform authentication to prevent unregistered users sending e-mail.
Sizing the hardware of your e-mail server
When choosing a computer to use as an e-mail server, a lot of people have misconceptions regarding the hardware required to perform this task well. The constantly increasing performance of computers seems to lead people into thinking that they really need the latest and most buzzword-compliant stuff, even if they only want to handle a few thousand messages per day.
Although a certain expertise is required to assess the hardware needs for an organization, common sense goes a long way. For a company with 100 users, a reasonably high upper limit for the number of messages per day would be 5,000. That would allow each user to send or receive 50 messages every day. Even if we say that each and every message is sent within the eight hours of the working day, on an average, the system will not have to cope with more than 10 messages per minute. It is reasonable that a modern computer can receive and act upon a single e-mail message, often only a few kilobytes in size, in less than six seconds.
This little back-of-the-envelope exercise is obviously very rough and does not, for example, take into account the fact that messages typically do not arrive uniformly distributed in time, but it is still a pretty good way of estimating.
Let us now take a deeper look into what to think about when choosing the server. For an e-mail server that does not perform any content scanning (viruses, spam, and so on), the performance is typically not bound by the CPU but by the I/O performance, specifically the seek time of the hard disk(s) and the quality and configuration of the I/O controller. Throwing more CPU horsepower at the problem will not help. Modern computers are relatively better equipped CPU wise than I/O wise, so investing in a multiple gigahertz multi-core CPU configuration is probably useless. For any reasonably modern 1 GHz-class PC, a handful of messages per second is no problem. That load equates to almost 20,000 messages every hour.
Adding content scanning will probably increase the CPU load quite a lot, and the I/O system will also require more power to keep up. Still, one or two messages per second should not place a noticeable load on the system.
What we have been discussing so far is just the e-mail server. All it does is receive messages and deliver them to other hosts or local mailboxes. When choosing a server, you should not forget that people are going to want to read their e-mail too. This service is provided by additional server software. Just like the message handling software, the key requirement is I/O and not CPU. The number of users of the system is by itself an irrelevant figure; what is important are the usage patterns. How often will the users poll their mailboxes? If 100 users poll their mailboxes once every five minutes, on average there will be one every three seconds. Checking if a mailbox has any new messages, takes a fraction of a second, so the burden will not be significant.
The final, and arguably the hardest thing to consider, is disk storage. Using the expected traffic numbers, we can make some rough estimates. Let us assume 80% of our messages are under 1 KB, 15% have document attachments of 200 KB with the remainder being videos and other large files of 1 MB. Therefore, using a 200 day working year, that equates to a storage requirement of approximately 80 GB per year. A typical 1 TB disk drive would have the capacity for more than 12 years messages assuming no messages are deleted.
These guidelines may appear vague and non specific, but it is impossible to give exact figures. The performance one would expect from a given piece of hardware depends on so many factors that trying to give anything but general guidelines would be misleading. Use common sense and simple back-of-the-envelope calculations; do not buy the fanciest server you can find unless you are sure you really need it, but also do not use any old abandoned desktop machine you can find. Even if the performance of the old desktop machine may suffice, the components may be old and the service agreement or warranty may be out of date.
Main e-mail protocols: SMTP, POP, and IMAP
Why are we discussing basic network communication protocols in this book? Are we not running advanced software? Indeed we are, but knowing one's way around the protocols cannot only assist debugging a possibly non-working system but also increases the understanding of a mail system's behavior. We will start with a rather non-technical overview of the protocols, after which we will focus on the protocol details.
Overview
In the UNIX environment, traditional mail applications did not use any network protocol at all. They have instead accessed the locally stored mailbox files directly through the file system. Typically, the inbox of each user is stored in a single file in either the /var/mail or the /var/spool/mail directory with the same name as that of the user (for example, /var/spool/mail/joe). The focus of this book is to discuss Linux based e-mail solutions for a small office where users do not wish to log on to a central server with a terminal application in order to access their mail, so local mail storage will be covered only briefly.
The most important protocol in Internet mailing is the Simple Mail Transfer Protocol (SMTP). Its purpose is to transport e-mail messages between two systems. Both these computers may either be servers, or one of them may be a client machine on which the user runs the mail application—Outlook, Thunderbird, Eudora, or whatever. To collect new messages, the end user does not utilize SMTP. This is where the Post Office Protocol (POP) and the Internet Message Access Protocol (IMAP) come in.
Some proprietary systems such as Microsoft Exchange and Lotus Notes use their own protocols to access messages, and we will not discuss them here.
POP protocol
POP is the older and more widely used protocol of the two. It focuses on giving the users access to their inboxes, from which the users can download the new messages to their local computers and then delete them from the server. POP servers are not meant to be used for permanent storage of messages. The POP services of some Internet providers even prohibit users from leaving messages on the server after they have been downloaded once. The chief disadvantage of POP is that it only provides an intermediary storage medium and the users must store their messages permanently someplace else (for example, on their local hard drives). This is not only impractical for users who want to access their e-mail messages from multiple locations, but it is also a hassle for the System Administrator who may have to implement a backup solution for the users' messages on their local hard drives. POP also does not have any notion of providing multiple folders for every user; with POP a user can access his/her inbox only.
IMAP protocol
IMAP is meant as an access method to a first class mail store, that is, it is designed to allow the user to store the messages permanently on the server. This solves the System Administrator's backup problem and allows the user to access all messages from any place in the world (firewall restrictions aside). IMAP also has a more widespread implementation of TLS-secured connections, making IMAP safe to use in hostile environments. To improve performance and allow users to work with their mailboxes while not being connected to the mail server, most mail applications with IMAP support caching the downloaded mailboxes and messages in the local hard drive.
Unlike POP, IMAP supports multiple folders and stores message state information (whether or not the message has been read, replied to, or deleted) on the server. This means that a user accessing their message store from different locations, with possibly different e-mail clients, will be presented with a consistent, up-to-date view of their messages. IMAP also supports server-side searching, so the client application does not need to download all the messages to search for an e-mail.
The SMTP protocol
SMTP is a line-oriented text protocol that runs over TCP, which makes it trivial to decode SMTP transcripts and to initiate SMTP sessions using the regular telnet client found on just about any computer. An SMTP client starts a session by connecting to port 25 on the SMTP server. After the server has greeted the client, the client must respond by saying hello, or actually HELO or EHLO, followed by the client's hostname. If the server accepts the cordial greeting, the client may begin the first mail transaction.
An SMTP mail transaction consists of three parts—a sender, one or more recipients, and the actual message contents. The sender is specified with the MAIL FROM command, each recipient with an RCPT TO command, and the start of the message contents with a DATA command. If the server accepts the message, the client may continue with additional transactions or issue the QUIT command to terminate the SMTP session.
Let's be less abstract and look at an actual SMTP session to illustrate the protocol. The bold face print represents what the client sends to the server.
This example shows a host that claims to be named gw.example.net connecting to an SMTP server that calls itself mail.example.com. Because the server's first response contains ESMTP, the client decides to try Enhanced SMTP (ESMTP) and greets the server with EHLO instead of HELO. The server accepts this greeting and responds with a list of the supported ESMTP extensions.
Together with the sender address, the client sends the SIZE attribute to indicate the size of the message to the server. This is allowed because the server has stated that it supports the SIZE extension. If the size specified by the client exceeds the message size limit set by the server, the message can be rejected at once rather than after the whole message has been received and the server can assess the size.
An SMTP message can obviously have more than one recipient. This has a few consequences that must be remembered while implementing a mail system and inventing policies. In the previous example, the mail server accepts the first two recipients but rejects the third one. As two recipients have been accepted by the server, the client will try to send the message contents. Here the message is accepted by the server and queued for delivery (250 Ok: queued as B059D3C2B), which means that the SMTP server has taken over the responsibility for the delivery of the message to the accepted recipients. If the message cannot be delivered, the server will send a non-delivery message (bounce) back to the sender. The server could also have chosen to reject the whole message. If so, it would have rejected it for all recipients and not delivered it at all. In other words, in response to the message contents the server must either reject the message for all recipients or accept it for all recipients.
It is vital to understand the difference between the envelope and the header. The envelope of a message consists of the information given in the MAIL FROM and RCPT TO commands, that is, the sender and recipient information that are used to deliver the message. An SMTP server pays no attention what so ever to the From, To, and Cc message headers. In our example the To header contains just a single address with no other relation to the actual recipient addresses than the domain, but that is just a coincidence. Bounces are always sent to the envelope sender address, in this case [email protected]. The sender address of bounce messages is the empty sender address, often called the null sender. However tempting it may be for some people, the null sender address must not be blocked.
So far, we have not commented on the numerical codes given by the server at the beginning of each line. Each number has a specific meaning and it is important to learn the correct interpretation of the first digit.
Digit
Meaning
2
Server has accepted the previous command and is awaiting your next command.
3
Used only in response to the DATA command, and means that the server is ready to accept the message contents.
4
Temporary error: The request cannot be performed at the moment, but it may be successfully serviced later.
5
Permanent error: The request will never be accepted.
In SMTP, error conditions can be either temporary or permanent. Both 4 and 5 are used to signal errors. A client that receives a temporary error designated by 4 should disconnect, keep the message in the queue, and retry at a later time. Typical temporary error conditions include a full mail queue disk, a server configuration error that must be resolved before messages can be accepted, or a temporary DNS lookup error. Permanent errors are indicated by the first digit being 5 and mean that the request will never be accepted, so a client will have to remove the message from the queue and send a bounce to the sender telling him or her that the message could not be delivered.
There is a lot more to SMTP than this quick introduction has covered. For further reading there are a number of documents that cover Internet networking related topics known as Request for Comments (RFC). RFCs are memorandums published by the Internet Engineering Task Force (IETF), which are generally adopted as standards. For SMTP the most important ones are RFC 821 (Simple Mail Transfer Protocol) and RFC 822 (Standard for the format of ARPA Internet text messages).
E-mail and DNS
The Domain Name System (DNS) plays an important role in e-mailing. The DNS is used by both, e-mail clients and e-mail servers. Even if you do not intend to maintain your own DNS server, a thorough understanding of DNS's role in e-mailing is a necessity for the mail server operator. This section assumes that the reader has basic knowledge of how DNS works in general.
DNS record types used by e-mail applications
In many networking scenarios, only two DNS record types are used—the A record and PTR record. These map hostnames to IP addresses and IP addresses to hostnames respectively. These record types are also used for e-mail, but there is also a third DNS record type that is uniquely available for e-mail.
How does an SMTP server discover to which host a message for a certain domain should be delivered? The recipient domain is, not surprisingly, used as the key in one or more DNS lookups. The first lookup that is made is for the mail-specific MX record—the mail exchanger record type. The MX entry allows the DNS operator to specify the hostname or hostnames of servers that can receive mail for a certain domain. For example, MX records can be used to specify that messages to someone at example.com should be sent to mail.example.com. If the recipient domain does not have an MX record, an attempt is made to find an A record for the recipient domain. If the A record lookup succeeds, the mail will be delivered to the host. If both the MX and A lookups do not return any results, the message is deemed undeliverable and is returned to the sender.
There are two good reasons to having MX records:
Equal-priority MX records can be used as a very crude form of load balancing between two or more servers. This is also possible with A records that map to multiple IP addresses. A hierarchy of backup mail servers with different priorities can be set up for a domain using MX records that cannot be made to happen with A records. Let us look at a constructed example of an organization that uses a lot of mail servers.
Priority
Hostname
10
mx1.example.com
10
mx2.example.com
20
mx3.example.com
30
mx4.example.com
If this DNS configuration is set for the domain example.com, SMTP servers are expected to try to deliver messages for example.com to mx1.example.com or mx2.example.com first. If both connections fail, mx3.example.com should be tried, and if even that server does not respond in a timely way, mx4.example.com is the last resort. Should that fail too, the message is kept and delivery is retried at a later time.
Backup mail servers
Having a backup mail server that can receive messages if the primary server is unavailable sounds like a really good idea, but today's reliable Internet connections together with spam, worms, and other rubbish have for the most part made backup mail servers unnecessary and often even harmful. The rationale for having a backup server is that it can receive messages while your primary server is down, and then deliver them to the primary server when it is up again. However, the advantage of this is very small, as all SMTP servers are required to queue undeliverable messages for at least five days before they are returned to the sender. Granted, by having a backup server it is possible to store unavailable messages for longer time than five days. However, if the main SMTP server is unavailable for longer than five days at a stretch then there are probably bigger problems than a few lost messages.
Because a backup mail server typically does not have the same spam-thwarting configuration as the primary server, spammers often specifically target backup servers in order to bypass the stricter rules of the primary server.
Another strong reason to avoid backup mail servers is that they typically do not perform recipient validation. This means that they do not know which recipient addresses are valid for the domains they act as backup servers for. This requires a backup server to accept all messages for the backed-up domains and attempt to deliver them to the primary server. The primary server will reject invalid recipients, causing the backup server to bounce such message back to the sender. This is known as backscatter and is bad for two reasons:
A busy server that does not perform recipient validation and is hit heavily with spam may have thousands or tens of thousands of undeliverable messages residing in the queue.
Summary
In this chapter, we started by discussing why you should even consider hosting your own e-mail server. Then, we looked at some questions that need to be answered before starting work with the server—the kind of network connection, computer power and disk space requirements that are expected.
To manage an e-mail server successfully, an understanding of the network communication protocols used is important. We gave an overview of POP and IMAP, and delved more deeply into the most important of them all, SMTP.
Finally, we looked at the vital role that the DNS plays in routing messages to the correct server or a backup server if one is available.
Chapter 2. Setting up Postfix
The Mail Transfer Agent (MTA) is perhaps the most important part of a mail system. It is responsible for receiving messages from the Internet or from your own users and doing what it can to make sure that the messages arrive at their destinations—other mail servers or mailboxes of your users.
Postfix has been chosen as the mail transfer agent to be covered in this book. Postfix has a large feature set, it has an excellent security track record, it is fast, easy to configure, and under active development.
This book assumes that you are running Postfix 2.0 or later. Any feature or behavior of Postfix that is specific to releases later than 2.0 will be noted.
Introduction to Postfix
This first section gives a brief introduction to Postfix, how it works, and describes how its behavior can be controlled.
What is Postfix
Postfix is a modular mail transfer agent developed by IBM researcher Wietse Venema. It is free software and was released publicly for the first time in 1998 under the name VMailer. It is written in C and currently consists of about 105,000 lines of code (comments excluded), which makes it fairly small. It works on most non-historic variants of UNIX and Linux.
As a pure mail transfer agent, Postfix does not provide any service for allowing users to collect their mail via the POP or IMAP protocols. That task must be carried out by some other piece of software. The software discussed in this book for facilitating retrieval of mail from the host is Courier IMAP.
All official Postfix documentation, as well as the source code and links to third-party software and archives of the very active mailing list can be found at the Postfix website at http://www.postfix.org/.
Postfix architecture: An overview
This section will describe the different parts of the Postfix mail transfer agent and explain what really goes on when you send a message through the system. Although this might not be the most exciting text you have ever read, understanding the basics of how Postfix works is essential if you wish to successfully manage a Postfix server.
Postfix is divided into a number of separate daemons, or background processes, that communicate with each other. The daemons have distinct areas of responsibility, may run in different security contexts, and may have different rules for the number of processes of their type that may be created. All daemon processes are created as needed and are supervised by a mother daemon, the master. Some daemons are rarely or never restarted, but most of them will commit suicide after having served a configurable number of requests or after they have been idle for a configurable duration of time. The following figure shows how messages flow through a Postfix system, and can be used to accompany the text that follows. The solid lines show the path of the message content while dotted lines show other forms of communication.
Not all Postfix daemons will be described here, just the important ones. A complete rundown of all daemons can be found in the Postfix Architecture Overview document at http://www.postfix.org/OVERVIEW.html.
New message arrival
New messages can arrive into the Postfix system in three ways. The most common way is, of course, via the Simple Mail Transfer Protocol(SMTP). The daemon responsible for receiving messages via SMTP is named smtpd. The uncommon QMQP Submission Protocol, introduced in Daniel J. Bernstein's MTA qmail, is also supported with the qmqpd daemon. However, this book will not discuss QMQP.
The third way in which a message can arrive is via local submission with the sendmail program. This is the standard way to submit mail messages from programs and scripts running on a UNIX host. Postfix provides a sendmail program that in most regards is compatible with the sendmail program of the sendmail mail transfer agent (http://www.sendmail.org/). Many UNIX mail user agents such as Mail, Pine, and Mutt, as well as webmail software such as SquirrelMail and IMP use the sendmail interface to submit new messages, although some software offer the option to submit messages via SMTP instead.
The sendmail program hands messages on to the postdrop program, which places message files in the maildrop directory within the Postfix queue directory. The pickup daemon waits for messages to arrive into the maildrop directory, and passes them on to the cleanup daemon. From there on, sendmail-submitted messages take the same road as messages submitted via SMTP or QMQP. Messages can be submitted via sendmail even if Postfix is not running on the machine at the moment. When Postfix starts the next time, pickup will discover the queued-up message files and process them.
When smtpd, qmqpd, or pickup receives a new message, it hands it to the cleanup daemon. This daemon enforces restrictions on the message's size, acts on any content restrictions configured by the user, rewrites sender and/or recipient addresses as required by the configuration, adds any required headers that are missing, and does a few other things. The cleanup daemon uses the trivial-rewrite daemon for some address rewriting operations. When done with its business, cleanup puts the queue file in the incoming queue and notifies the queue manager.
Scheduling message deliveries
The queue manager, qmgr, is responsible for scheduling the delivery of messages. To decide how a message should be delivered to each recipient (namely the delivery method and the next destination), qmgr gets help from trivial-rewrite. The queue manager requests delivery agent processes from the master daemon and collects the results of the deliveries.
The queue manager is responsible for all messages from the point when the cleanup daemon hands them over until they are removed from the queue. The removal can be either because they have been successfully delivered to all recipients or because they have been in the queue for so long that Postfix decides that they are undeliverable. By default, messages will remain in the queue for a maximum of five days. The queue manager calls upon the bounce daemon to send a bounce message to the sender.
The queue manager uses a number of directories for different purposes. The incoming queue is monitored for new messages, and the next stop is the active queue. The active queue contains the messages that are ready for delivery and are waiting to be dispatched to a delivery agent. If a delivery attempt fails, the message is moved to the deferred queue. That queue will be scanned periodically and, if it is time to retry the delivery of a message, the queue file for the message will be moved back into the active queue. Whether a delivery of a message should be reattempted when the queue is scanned depends on two—factors how much time has passed since the message arrived and the two configuration parameters that set a minimum and maximum time interval between the reattempts.
In addition to these queues, there is also a special-purpose queue named hold. This queue contains messages that have been put on hold by the system administrator using the postsuper command. Postfix will not touch these messages at all until they are taken off hold with the same command. The hold queue can be used to temporarily stall the delivery of certain messages, for example because they are large and need to be delivered during off-peak hours, or because they are deemed suspicious and need to be inspected before they are allowed to be delivered.
The different queues used by Postfix are described in detail in the QSHAPE_README document (http://www.postfix.org/QSHAPE_README.html). This document also describes qshape, a script that ships with Postfix and analyzes the contents of the queues, and helps you identify bottlenecks.
Message delivery
Postfix comes with a number of delivery agents that are used to deliver messages using various means and protocols. The delivery agents are the last daemons that touch the messages before they leave your system.
The Postfix SMTP client, smtp (not to be confused with the SMTP server, smtpd), is used to deliver messages to other hosts via the SMTP protocol. It is very similar to the LMTP client, lmtp, which delivers messages via the Local Mail Transfer Protocol (LMTP). As a network protocol, LMTP is very similar to SMTP, but where SMTP is used to transport messages between MTAs, LMTP is used for the final delivery of messages to the mail store from which users can access the messages.
The local delivery agent, local, delivers messages to users with normal accounts on the system. It supports aliases for simple mailing lists or role addresses as well as .forward files so that users themselves can set up forwarding of their messages.
If you have virtual mailbox users—users that do not have real accounts (for example, shell accounts) on the system their messages are delivered with virtual Postfix daemon.
If Postfix's standard delivery agents do not suffice, you can write your own delivery agent and have Postfix invoke it for some (or all) messages. In that case, you can either use the pipe
