The Adventures of Systems Boy!

It's been quite some time since I've been able to post anything of any substance. This has a lot to do with the fact that I've been super busy relocating our department and participating in the gut renovation of our lab. This has been an immensely stressful process, but in the end I find that I've learned so much from it, I simply can't complain. I'm coming out a far better SysAdmin than I was going in. And that's a remarkably valuable thing to both me and my employers.

But moving and planning the physical aspects of the new lab has only been a portion of what I've been working on. This renovation has been the perfect opportunity to rebuild our network infrastructure, and part of said rebuilding has resulted in the near completion of our authentication unification project. At this point we've gone from eight different authentication servers — that is, anytime we created a new user, we had to do so on eight different systems — all the way on down to two. Which means that now, anytime we create a new user, we do so on two machines.

Our goal is to get it down to one, hopefully before the Fall semester begins. Our mail server is proving to be the most difficult machine to get working with LDAP authentication, mainly because it authenticates mail users through the wonders of some weird combination of authd, Courier and PAM, and we've yet to crack the magical code that gets these all working in tandem via LDAP. Aside from Mail, though, everything is done. So I thought I'd take a bit of my hard-earned vacation and loosely describe to you how it's all working.

Before I start I'd like to just acknowledge all the help I've had from my fellow SysAdmins in the department. I had a huge amount of assistance on the *NIX server side of things, as well as with network infrastructure and even some last-minute PHP finagling without which this project would have taken significantly longer. In fact, all I really had to do was build the authentication servers and clearly articulate what I wanted. I'm extremely grateful to everyone who helped out.

The little bit of network infrastructure I mentioned is our DMZ. We now have a proper — and more importantly, properly secured — DMZ on which to place an authentication server. I won't go into too much detail here, but suffice to say, having a secure DMZ gives us all kinds of options for authentication between internal and external networks, and makes me feel a whole lot better about using Mac OS X Server as our authentication system for both networks.

Yes, we are using Mac OS X Server to authenticate our entire network. The reason is because Mac OS X Server is the most mature and usable implementation of LDAP for user authentication available on the market today. Is it perfect? No. Is it completely secure? Probably not. Is there anything that even comes remotely close to being able to handle the complexities of user management and database redundancy across platforms with such remarkable ease-of-use? Nope. Nothing. We tried building our own custom LDAP server, which would have been excruciating, and would have taken forever. We tried Red Hat's Directory Server, which looks like it will eventually turn into something to match Mac OS X Server, but which just wasn't yet up to snuff. Nothing matched Mac OS X Server, which did everything we wanted it to, right out of the box and with a minimum of fuss. In fact, once the user database is built, building a Mac OS X master or replica authentication server is a complete and total breeze. At the time of our building and testing it was really the only practical option.

So, here in a nutshell, is what we have:

Internal Network
All authentication originates from the internal network. Passwords can only be changed from the internal network at this time, which is by design. Systems on the internal network include:

• Master Authentication Server
  Hosts authentication for... Well... Everything, really. This is essentially the same server we used all last year for all our internal authentication needs for Mac, Linux and Windows workstations. It's now being used to push authentication to the external network as well.
• Internal Replica Authentication Server
  This provides replication of the Master. Should the master fail, the Replica is intended to pick up services (though this doesn't always work perfectly).
• File Servers
  We have two file servers on the internal network — a Mac and a Linux box — both of which authenticate directly against the Master.
• Workstations
  We have about 30 Mac, Windows and Linux machines all authenticating to the Master.

DMZ
The DMZ sits between the Big Bad Internet (BBI) and the internal network. It has its own firewall that is fairly strict about what can get in from the BBI. All DMZ authentication originates from the internal network, but is provided by a single server which sits on the DMZ. Systems on the DMZ include:

• External Authentication Server
  This server is also a replica of our Master, but it's not intended as a failsafe. Rather, it provides authentication services to the entire DMZ. It gets its user database, of course, from the Master. But for other systems to bind to an LDAP server, its role must either be "Master" or "Replica." Setting the role to "Connected to a Directory Server" won't work. In addition to sitting on our DMZ, which is properly firewalled against the harsh realities of the Big Bad Internet (BBI), this system also makes use of its own strict local firewall for an extra added layer of security. Also, all replication communication between Replica (DMZ) and Master (Internal) is encrypted.
• Data Server
  In addition to unifying authentication, we've also consolidated data storage and access wherever possible. In the past, for instance, movies streamed from the Quicktime Server were stored on that machine's local drive. Web sites were stored on our web server. So, building a web site that used Quicktime Streaming required users to log into two separate machines — the Web Server and the Quicktime Streaming Server. Now we're storing all user-generated content on a separate, dedicated machine — our Data Server — and sharing that machine out to the various servers via NFS. Centralizing this data store means users have only to log on to one server for anything they ever want to do. And also that only that server needs to authenticate users. And yes, that server authenticates them via LDAP on our External Authentication Server. All neat and tidy. Internal and external home account data is still segregated, however — users still have separate internal and external data storage. Though, if we could figure out how to do it securely, this could change.
• Quicktime Streaming Server
  This machine also uses its own local firewall. It gets its user database from our External Authentication Server over secure channels using the "Connected to a Directory System" as its role currently. Ultimately, however, because of the Data Server, this machine will not need to authenticate users. We are leaving the ability open temporarily to accommodate legacy users.
• Drupal CMS
  Our new Community site is built on the Drupal engine. We're using the LDAP module to authenticate to the External Authentication Server. Drupal's LDAP module is simple and easy to set up, as is the Drupal system as a whole. So far we're very happy with it.
• Computer Reservations System
  This is a custom web app built long ago by a former student. We've (and by "we" I mean my colleague) basically hacked the PHP code to authenticate via LDAP rather than MySQL.
• Mail Server
  Currently not authenticating to the External Authentication Server. We're working on this and hope to have it working by the beginning of the school year.
  

The Future
Yes, there's more we want to do. It's always amazing how, once you've completed something, you immediately start seeing ways to make it better.

• More Redundancy
  Ultimately, in addition to the Replica, I would also like to automate a clone of the Master's boot drive to an external firewire drive as sort of an ultimate safety. Should anything ever go wrong with the Master, I simply plug the firewire clone into virtually any Mac system on the internal network and I'm back on my feet. It might also be wise to have some sort of failsafe for external authentication as well.
  
• More Security
  while our setup is fairly secure right now, there are a few areas I'd like to beef up even more when I get a chance. In particular, our CMS connection is not as secure as I'd like it to be. And ultimately I'd like to harden every machine on the DMZ to the best of my ability.
• More Unification
  Anything else we can unify — and at this point that's mostly internal and external data — I'm open to considering. It's going to be really interesting for me to look critically at what we've done so far and find the flaws and refine the system. But I'll constantly be looking at ways to simplify our current setup even further without compromising security. The easier our network is to use, the more useful it becomes. We've come a long way, but I'm sure we can find even better ways to do things.
• More Services
  Now that we have an infrastructure in place for user creation, we can add services freely to our network without the worry of creating users for said services. New services need only the ability to authenticate via LDAP. We're already planning an equipment checkout system, and possibly some calendaring systems.

So, I've just finalized the master authentication server. It's done. Built. Finished. Kaput. The rest of our servers are still in various states of finality, and we have until September to lock them down. But right now, unified authentication is, for all intents and purposes (and with the exception of mail), working. And we couldn't be happier. The ultimate test will be, of course, letting users loose on this new infrastructure. I'm betting they'll like it almost as much as we do. At least the ones who know the old system. New users will be none the wiser. Ain't that always the way?

*Sigh*

Labels: ExternalNetworkUnification, Lab, MacOSX, NIX, Server, Systems

This entry was posted on Thursday, June 28, 2007 at 11:29 AM. You can skip to the end and leave a response.