Newsletter (January 3 2007)
From Apis Networks Wiki
Greetings!
First off, I would like to wish you a belated happy new year. Things have been extremely hectic around here as we near crunch time with the data center migration, so this year's "year in review" newsletter will be brief.
Contents |
2006 recap
This annual newsletter began last year as a method of keeping you up to date on the changes over the year at Apis and future outlook for the following year. Keeping in style with last year's format, let me explain what happened during 2006: esprit matured to a fully-featured (but still incomplete) control panel. Phew, that was a lengthy list! Actually although esprit's maturity was a big milestone there are some other things to note:
- disk quotas increased by 75%
- a new referral system was implemented and is accessible through esprit's "Referral Tree" section.
- Gauss represented the ephemeral third generation, which focused on security and slightly on performance increases
- PayPal is still sitting there as the preferred method of payment. If you are paying at the yearly interval though, you can always pay via check. This will be fixed later this month (January).
- Rails, subversion, CVS, and Web 2.0 support added -- Web 2.0 support??? Next step is Web 3.11 for Workgroups support.
Filtering out spam
There is also one special mention of the "SpamAssassin Configuration Wizard" that a lot of people miss (and consequently e-mail me about). If you want to delete those e-mails marked with [SPAM] in the subject line, visit the wizard listed at the bottom of this e-mail to setup that behavior on the server. Plug in your FTP login and password, which for most people should be the same as your e-mail address and password to save the changes and enjoy the server-side filtering.
Data center migration
Now that 2006's recap is done, let us progress to the next big topic: data center migration. This has become the all-consuming, epic quest for higher performance, lower long-term costs, easier accessibility, improved hardware, and of course additional redundancy. Originally I had two target dates with the first between the window of December 15 - December 25, but due to unavoidable circumstances that had to be pushed back to the second window, January 12 - January 16. During this window the first of the new fourth generation servers, Echelon, will be installed at Gnax, a local data center in Atlanta, Georgia. Roughly one week later, January 19 - January 24, the accounts on the old servers will be moved from EV1 in Houston, Texas over to the new servers housed in Atlanta. I am waiting to hear back from the Dell rep on the additional four servers, which will be named x, y, z, and t (otherwise known as "unnamed" at this time).
There are two things to mention prior to the data center move. First, you will receive a pair of notices. One a few days before the move informing you of the date and window at which your site will be transferred and a second informing you that the site is live on the new server. Secondly, I need people who love to stress test servers help work out the bugs in real time. If you are running a non-critical site and want to play with the new toys, then be sure to see the thread mentioned at the bottom of this newsletter.
I can read your mind at this time. You are getting very bored with this newsletter and your interest is waning. Fear not though for now we shall talk about what makes the fourth generation so special (at this point you close the e-mail). With every new generation of servers I like to introduce a general theme. For the first generation it was subsistence, second generation was features, third generation was hardening, and this fourth generation will focus on raw performance. Unfortunately choosing Ensim to provide your original control panel backend, things start to get pretty inefficient as you scale in size. As such, everything in the fourth generation is geared towards addressing and fixing those inefficiencies. A few minor changes have made their way back to the current generation, but let me go over the major software changes:
sendmail -> Postfix
sendmail and Postfix are both MTAs (mail transfer agents) and are responsible for receiving and delivering your e-mail. To put these two in perspective, imagine this: Postfix is the civil, upscale, and well-liked MTA, whereas sendmail is like Postfix's distant cousin that barges into its black tie gala wearing fishnet stockings, three sheets to the wind, chain smoking, and hitting on every single person there with catchphrases from Wayne's World. If this has managed to burn an image into the back of your mind, then you can thank me for when you giggle uncontrollably at the term "sendmail" in the future. sendmail gains its popularity (although dwindling these days) due to its age. Back when dinosaurs roamed the earth and cavemen used ARPANET for their Internet needs, sendmail (then called delivermail) was out there dishing out those e-mails full of Viagra and cheap diploma ads. Over the years instead of rewriting the framework, developers just added onto it and inevitably you are left with a monolithic, insecure, and mess of an application whose configuration resembles alphabet soup sometimes.
Postfix is second in efficiency to qmail, but without the terrible licensing and complex configuration that can sometimes hinder administration. Furthermore Postfix is nearly a perfect drop-in replacement to sendmail making the transition even easier. Some things will change though. For starters, mailer tables will no longer be flat-files, but rather migrated to a PostgreSQL backend. This cuts down on the memory consumption of message delivery, decreases the amount of time to load up a delivery process, and best of all makes it sound really high tech. Additionally you will be able to add a user to your account and deny IMAP/POP3/SMTP access thereby prohibiting e-mail access entirely from a user. Because mailer tables are being pulled from a database, we can also ditch the chroot() call right before mail delivery that slows things down a bit. This again should make delivery much faster and make the servers much more resilient to e-mail flooding if say perhaps you botch your mailing list and generate some 2,000 bouncebacks.
From the elimination of the chroot() call, SpamAssassin can also filter messages without having to restart every single time. Avoiding a costly client shutdown/startup after every message nets some huge performance gains. There is one change that may upset some. If you go over quota, messages will no longer be retried, but instead will return to sender with the FAILED status. More often than not users that run over quota stay over quota. Spam keeps piling up and continuously goes through every single step in the mail delivery process until it is finally deemed a failure some 20 tries later. Pay attention to your disk space usage and be sure to periodically login to the control panel once a week or so to check out what's going on with Apis, the server, and your account.
apnscp -> esprit
apnscp, the legacy control panel, will no longer be an option of the fourth generation servers. It will be making way for esprit, which you should have used at least once (or more if you tried to file a ticket through apnscp). I will add the final parts to esprit once Echelon is housed in the new data center.
ProFTPD -> vsftpd
This probably should have gone in the third generation changes, but ProFTPD is finally being given the pink slip and 50 cents for bus fare. At this time vsftpd is working fine with Ensim's custom PAM authentication, which is a good thing. The only problem is that user/group names don't show up correctly at this time, but it is a minor issue that will be dealt with once the data center move is finished. ProFTPD lands itself on CVE's listing of exploits every so often and vsftpd, well look no further that its meaning: very secure FTP daemon. Another nice feature is the ability to lock certain users into a home directory, so if you want to give bob access just to /home/bob/, then you will be able to do it in the control panel within the next couple of months.
UW-IMAP -> Dovecot
I am not quite sure where to begin with this one. Mailbox delivery will change from mbox to Maildir, which means all of those annoying error messages you get in SquirrelMail should be resolved as locking is a big problem with the mbox format. Finding mail should be quicker server-side too because messages are broken up into several smaller files instead of a single and ginormous (please don't stab me) file composed of read and unread messages. I recall one individual requesting Maildir format in a ticket and that change is for you. File change notification support will go through inotify, which is available in glibc 2.4 and newer kernels (2.6.13 and onward). This is a more efficient method of sending file notifications from the kernel-level to the userspace. inotify is used by the IDLE command in IMAP so that your mail client receives updates as messages are delivered instead of the classic method of connect to mail server, poll, download new messages if they exist. That is one (of several things) UW-IMAP didn't have going for it.
Credit Card Processor/Package Changes
A separate credit card process plus discounted two and three year subscriptions will follow once the new servers are setup as a way to offset the costs of switching data centers. This will make it much more accessible to users, so if you or your friend were burned in the past by PayPal, don't worry because finally I will be adding an alternative.
Closing
On the outside nothing should change meaning the transition should happen transparently. These are just a few of the big things you will be looking forward to internally. Ultimately the changes will yield higher performance even with the hardware changes (dual 5130 Xeons, 4 GB RAM), so we will have more cushioning for auxiliary services like Mongrel.
In closing I would like to let you know that the fourth generation is being built from the ground up. If you want to see a new feature standard, then by all means please post in the data center migration thread (at the bottom of this letter). There is a good amount of feedback thus far that translates into more toys, but by no means is the list complete. If you want something to appear stock on these servers, then let me know by posting in the thread. As I draw closer to the deadline of setting everything up I will be difficult to get hold of and may be slow on handling tickets during my normal hours of 11 AM - 2 AM. This is a warning to expect slow response times from myself. You may be better off holding off on your trouble tickets toward the 2 AM - 11 AM window or holding off in general unless you are really stuck. I am condensing a good four months of work and research into a couple of weeks, so I will be working tight scheduling.
I'm sure you have either fallen asleep now or closed the e-mail. Given that, I hope you enjoyed the New Year and are getting ready to post some requests for the new servers.
Thanks and I will see you in a couple of weeks once your account is moved.
Matt Saladna Chief Liaison/Head Apis Networks
