Personal Email Sever on Ubuntu 7.10
Well, this has been a good year for learning, so I thought I'd close out by sharing some of what I learned over the past week, in the process of setting up an email server. I don't know if this post will really be of any use to the people who usually read my blog (if there still are any out there), but it might just help those who stumble across it through google.
My basic idea was to consolidate all of my various Pop3 email accounts into one single IMAP account. I have several personal accounts (google, fejes.ca, etc), as well as a work account, and now that I'll be accessing my email from 4 separate computers (my work computer, my home computer, my laptop, and the one at my girlfriend's place), keeping track of emails on a pop account is a pain in the rear end.
The main lesson I'd like to go over here is the architecture of how this works. There are tons of tutorials on setting up IMAP accounts on Ubuntu (and some of them were really good) - but no one tells you how to fit all the pieces together.
The core of the whole process is that you need to set up what's called a Mail Transfer Agent or MTA. This agent or program will accept emails on port 25, and transfer them on to whatever other system should receive them. In a simple setup, the MTA will accept emails, and then pass them on to your ISP's mail server, and do nothing more. Essentially, you're simply making your MDA into a personalized SMTP server.
Unfortunately, this is severely useless. Why create your own SMTP? (simple mail transfer protocol server.) Well, if you were in a large organization, this might be useful, or if you wanted to send a lot of spam, then sure. Otherwise, it's pretty pointless - the vast majority of people don't need this functionality.
However, it does become useful if you attach the MTA to a mail delivery agent. The mail delivery agent is a local postman - it takes mail that comes in to your computer, and instead of handing it off to another computer, it delivers it into your local mail box. Thus, each user (or virtual user) can have their own personal mailbox on the local computer.
For my setup, I began by using Postfix as my MTA. After playing with it for a while, I discovered that some of the lines blur. By configuring it properly, Postfix can act not only as an MTA, but also as a MDA - it not only receives email, but also passes it into the appropriate mailbox. (A subprogram called "local" will take over this function, if it's set up in the main.cf.) This allows you to use Postfix as, essentially, a stand alone mail server.
Incoming mail arrives on port 25, to Postfix. Postfix hands it off to the "local" program, which delivers it. Neat.
Unfortunately, my system couldn't be quite so simple. My ISP blocks port 25, so no mail will ever come to computer via the regular route. No matter whether I configure the MX record correctly, and the local delivery options, email will never make it to my computer, and into my mailbox.
Thus, the next element is something called "fetchmail". Fetchmail is a program that helps get around this problem. Instead of forwarding email from my other accounts to my new local IMAP server, I have to reverse the process, and use my IMAP server to fetch my email from the other accounts, avoiding the blocked port 25. Enter Fetchmail.
Using Fetchmail, I configured my local computer to poll the servers at some interval (1.5 minutes), to grab my email on the regular POP3 port as if it were a mail client (ie. thunderbird, outlook, etc.). Unlike a mail client, Fetchmail doesn't deliver the mail to my mailbox, it hands it off to port 25 on my local computer, where it finds the above mentioned Postfix server! Tada!
Ok. So now, we have a system where email is accessed from my accounts on another machine, and dropped into my local email system. Great. 2 parts left: getting my email, and making sure it's delivered into the right folder in my email box. Fortunately, they're both handled by two parts of the same program. Dovecot.
I took advantage of Dovecot for two main reasons: it has a very easy to set up IMAP server, so I can access my email remotely, and because it's integrated with something called "sieve".
As an IMAP server, Dovecot is remarkably easy to set up. Tell it where and what form the local mailbox is, and what protocol to serve it up on. Tada, you'll have an IMAP server up very quickly. A little more work, and you can set it up with some excellent security, too.! (Highly recommended!)
That other piece, "sieve" is a little more complicated. It's essentially a language for filtering emails, though it's very simple to get the basics of it. The only caveat is that you can only use Sieve if you're using Dovecot as your MDA. This isn't a big deal - almost no setup is required, and again there are tutorials on it. By switching away from the "local" program I mentioned above, and configuring Postfix to use dovecot instead, you'll be able to create some simple filtering rules that pass each email into the appropriate folder. And there you go - you'll be all set up. (Don't forget to tell thunderbird to check each folder for new mail, however!)
So, my system works like this:
Done!
My basic idea was to consolidate all of my various Pop3 email accounts into one single IMAP account. I have several personal accounts (google, fejes.ca, etc), as well as a work account, and now that I'll be accessing my email from 4 separate computers (my work computer, my home computer, my laptop, and the one at my girlfriend's place), keeping track of emails on a pop account is a pain in the rear end.
The main lesson I'd like to go over here is the architecture of how this works. There are tons of tutorials on setting up IMAP accounts on Ubuntu (and some of them were really good) - but no one tells you how to fit all the pieces together.
The core of the whole process is that you need to set up what's called a Mail Transfer Agent or MTA. This agent or program will accept emails on port 25, and transfer them on to whatever other system should receive them. In a simple setup, the MTA will accept emails, and then pass them on to your ISP's mail server, and do nothing more. Essentially, you're simply making your MDA into a personalized SMTP server.
Unfortunately, this is severely useless. Why create your own SMTP? (simple mail transfer protocol server.) Well, if you were in a large organization, this might be useful, or if you wanted to send a lot of spam, then sure. Otherwise, it's pretty pointless - the vast majority of people don't need this functionality.
However, it does become useful if you attach the MTA to a mail delivery agent. The mail delivery agent is a local postman - it takes mail that comes in to your computer, and instead of handing it off to another computer, it delivers it into your local mail box. Thus, each user (or virtual user) can have their own personal mailbox on the local computer.
For my setup, I began by using Postfix as my MTA. After playing with it for a while, I discovered that some of the lines blur. By configuring it properly, Postfix can act not only as an MTA, but also as a MDA - it not only receives email, but also passes it into the appropriate mailbox. (A subprogram called "local" will take over this function, if it's set up in the main.cf.) This allows you to use Postfix as, essentially, a stand alone mail server.
Incoming mail arrives on port 25, to Postfix. Postfix hands it off to the "local" program, which delivers it. Neat.
Unfortunately, my system couldn't be quite so simple. My ISP blocks port 25, so no mail will ever come to computer via the regular route. No matter whether I configure the MX record correctly, and the local delivery options, email will never make it to my computer, and into my mailbox.
Thus, the next element is something called "fetchmail". Fetchmail is a program that helps get around this problem. Instead of forwarding email from my other accounts to my new local IMAP server, I have to reverse the process, and use my IMAP server to fetch my email from the other accounts, avoiding the blocked port 25. Enter Fetchmail.
Using Fetchmail, I configured my local computer to poll the servers at some interval (1.5 minutes), to grab my email on the regular POP3 port as if it were a mail client (ie. thunderbird, outlook, etc.). Unlike a mail client, Fetchmail doesn't deliver the mail to my mailbox, it hands it off to port 25 on my local computer, where it finds the above mentioned Postfix server! Tada!
Ok. So now, we have a system where email is accessed from my accounts on another machine, and dropped into my local email system. Great. 2 parts left: getting my email, and making sure it's delivered into the right folder in my email box. Fortunately, they're both handled by two parts of the same program. Dovecot.
I took advantage of Dovecot for two main reasons: it has a very easy to set up IMAP server, so I can access my email remotely, and because it's integrated with something called "sieve".
As an IMAP server, Dovecot is remarkably easy to set up. Tell it where and what form the local mailbox is, and what protocol to serve it up on. Tada, you'll have an IMAP server up very quickly. A little more work, and you can set it up with some excellent security, too.! (Highly recommended!)
That other piece, "sieve" is a little more complicated. It's essentially a language for filtering emails, though it's very simple to get the basics of it. The only caveat is that you can only use Sieve if you're using Dovecot as your MDA. This isn't a big deal - almost no setup is required, and again there are tutorials on it. By switching away from the "local" program I mentioned above, and configuring Postfix to use dovecot instead, you'll be able to create some simple filtering rules that pass each email into the appropriate folder. And there you go - you'll be all set up. (Don't forget to tell thunderbird to check each folder for new mail, however!)
So, my system works like this:
- Email goes to my pop3 server
- Fetchmail gets new mail from my pop3 server, and brings it down to my local IMAP server, where it's handed off to Postfix.
- Postfix looks at the email, and hands it off to Dovecot for local delivery
- Dovecot looks at the email, and processes it with Sieve rules
- it goes into the right folder
Done!
