A month or two ago, we bought two new servers for the toolserver. One of these (nightshade) was to be the new login server (replacing hemlock, which would remain as the web server), and the other (willow) the new stable server (replacing the current stable, which is a virtual server on vandale). These are Dell PowerEdge 1950 IIIs, each with 2 quad-core 2GHz Xeons (E5405), 8GB RAM and 2 146GB SAS disks. Compared to the current login server, a dual Opteron, this is quite an improvement.
Unfortunately, when they arrived, we realised we’d forgotten to buy DRAC cards, so we couldn’t use them. (Dell Remote Access Controller is a management card that allows remote power control and serial console access to servers.) Last week, we finally got two DRAC cards installed, so I began setting up the new servers. Mostly, this was pretty simple. A few minor problems:
- The Debian Linux installer doesn’t support the Broadcom NetExtreme II network card, because it requires a so-called “non-free” firmware to be loaded. I fixed this by creating a custom initrd containing the firmware file, after which installation was relatively simple.
- Solaris does support the NetExtreme II, but it doesn’t support the PERC6/i RAID controller. Again, the fix for this was building a custom install miniroot containing the LSI driver for this card (mega_sas).
- Serial console redirection on these servers defaults to “continue after POST”, which has an unfortunate interaction when GRUB is configured to use the serial console; the server appears to hang after POST. This is easy to change, but annoying if you don’t notice it.
Since we now have two user servers, the next issue was sharing user account information between them. The simplest way to do this is just sycning the account data (/etc/passwd, /etc/shadow and /etc/group) between the two servers. There are a few problems with this, though: you have to wait until the next sync for updates to take effect on both servers; and any differences between the two on each server (for example, Debian likes to add new users during package installations) will be lost.
So, the two common options for shared account information are NIS and LDAP. NIS is an older system that provides basic key/value maps (e.g. looking up the username in the ‘passwd’ map, and getting the corresponding entry from /etc/passwd). On the other hand, LDAP provides a database of objects, each of which can have any number of user-defined attributes. The advantage of LDAP is that each user object can have additional information beyond POSIX account information (for example, email accounts). I decided to use LDAP for the toolserver, mainly for this reason.
The LDAP server I used was Sun Directory Server Enterprise Edition. Despite the name, this is a fairly lightweight LDAP server, with good documentation. In the past I’ve used OpenLDAP, but I found it quite fiddly to configure, and not so well documented. The Linux version of DSEE is only supported on RedHat and SUSE systems, but it installed and ran fine on Debian.
Importing the existing account data to LDAP using the PADL LDAP MigrationTools was quite straightforward. For some reason, Debian has two different LDAP NSS modules; libnss-ldap and libnss-ldapd. I chose the latter, which uses a daemon (nslcd) to proxy requests between user applications and the LDAP server. After configuring NSS and PAM to use LDAP, everything seems to be working.
The new stable server was simpler to set up, as it’s replacing the existing server rather than augmenting it. Installing Solaris and the web server was straightforward; all that was left was copying a few configuration files from the old stable server.
One thing I did differently was using pkgbuild for installing software packages, rather than compiling by hand. pkgbuild builds Solaris packages from RPM spec files; as specs contain all the information required to build a package in a single file, it’s easy to create repositories of spec files which can be built using pkgbuild. This is much easier to manage than the old stable server, where things were simply built and installed without using packages. And, as there’s already a large existing repository of such specs (spec-files-extra) to work from, it’s easy to add new software.
These new servers should provide enough capacity for the toolserver to last a long time, as well as improving both interactive performance and web serving. (But look out for a future post about switchboard, a fast replacement for suphp using FastCGI — if I ever get around to finishing it.)