sniping.org » Linux

A Couple of Links

fdiv_bug — Mon, 12 Nov 2007 16:31:34 +0000

I don’t usually just post links, but these are too great to not share.

The Nerd Handbook: This explains so much about me and “my people” that it borders on the uncanny. Everyone who is dating, has ever dated, or is planning on dating a nerd/geek needs to read this.
Installing MySQL on Ubuntu: A sarcastic response to a post by Dan Benjamin about how to install the MySQL database on Mac OS X. It still floors me when Mac people talk about how great it is that you can install software by hand, from source. We got over that years and years ago in the Linux world, FreeBSD’s Ports has been a solid solution for many years as well, and they’re even getting past it in Solaris. Mac OS X has MacPorts and lots of people use it, but there’s still a great deal of software-management-by-hand done in that world.

Live Resizing of RAID5 Arrays in Linux

fdiv_bug — Fri, 08 Jun 2007 21:58:18 +0000

A friend of mine asked about the benefits of software RAID versus hardware RAID in Linux, and I proceeded to voice my support for the former. It was then brought up that online resizing was one of her concerns, and I set about to figure out how to do just that. These instructions were crafted in a CentOS 5 virtual machine, and come with no warranty or guarantee of any sort; I put them here on the off chance that they might help someone.

First, I started by adding four additional SCSI drives to my CentOS 5 virtual machine. I then booted the VM up, and partitioned the disks, creating one partition that encompassed the entire drive and setting its type to “fd”:
```
for i in a b c d ; do echo "; ; fd ;" | sfdisk /dev/sd$i ; done
```
Then I created the new RAID5 array with four devices, none of which should be kept as spares:
```
mdadm --create /dev/md0 --spare-devices=0 --level=5 --raid-devices=4 /dev/sd{a,b,c,d}1
```
Third, create a physical volume for LVM to use:
```
pvcreate /dev/md0
```
After that, I made a volume group:
```
vgcreate vg0 /dev/md0
```
Then I made a logical volume in that group using all free physical extents:
```
lvcreate --extents=100%VG --name=storage vg0
```
Sixth, slap a file system on that guy:
```
mke2fs -j /dev/vg0/storage
```
Mount it, put some data on it, and enjoy.

After you’ve been using the array for a while (or, after it’s finished resyncing if you’re just following along at home), you add some new drives to the machine that you want to add to the array. I shut my VM down, added some additional virtual disks to it, and powered it back up; your steps would presumably be somewhat similar. After the system came back up, I added the new drives like so.

Partition them as with the other drives in the set:

for i in e f ; do sfdisk -d /dev/sda | sfdisk /dev/sd$i ; done

Add them to the RAID array:
```
mdadm --add /dev/md0 /dev/sd{e,f}1
```
Expand the array to encompass all the disks (note that this will probably take a while, so you might want to do it overnight):
```
mdadm --grow --raid-devices=6 /dev/md0
```
Expand the LVM physical volume to include the new space:
```
pvresize /dev/md0
```

Then expand the logical volume as well:

lvresize --extents=100%VG /dev/vg0/storage

Finally, grow the file system:
```
resize2fs /dev/vg0/storage
```

With the exception of powering down to add the new drives, nothing had to be brought offline, and if your storage controller supports the hot-adding of devices then you may not even have to that (though the scope of doing so is beyond this tutorial, since I don’t have ready access to such hardware).

Hope that helps, and please let me know if you see any inconsistencies or run into any problems!

What I Look for in a (Linux) Server System

fdiv_bug — Tue, 03 Apr 2007 15:05:13 +0000

A coworker of mine and I had discussed my stance on Apple — it hasn’t really changed much since I wrote that, in case you’re wondering — and she asked me for my thoughts on what I look for and expect in a Linux system, so I offered to write them down here. Most of these considerations and observations also apply to other Unix, or even non-Unix systems, too.

Manageability

I should be able to deploy fifty systems as easily as I can deploy one system.

This is easily the most important aspect to any system. The various Linux distributions provide a handful of ways of doing it, and we’ve been very successful with Red Hat Kickstart functionality in the CentOS distro. With Kickstart and network booting via PXE, I can deploy a new Linux box without even really having to set hands on it — our operations team can power the system on and hit F12. Solutions like this exist for other systems, and are included out-of-the-box with Mac OS X (provided you have a Mac OS X Server machine to serve up the NetBoot images or are clever enough to hack it together yourself), so it’s hardly unique to Linux, but I would not deploy any OS that doesn’t have rapid provisioning and installation capabilities.

Packaging

I should be able to install a piece of software on fifty servers as easily as I can install it on one server.

The point of a server is to provide services to users, and to do so requires software, whether it’s a web server like Apache or IIS, an NFS server daemon, Windows file sharing services, or any of another effectively limitless number of options. Sometimes this software is part of the core system — in the case of a Windows file server, you don’t need to add anything else to the system for it to be able to fulfill that role. Other times it’s included with the system but is a separate part — this is more commonly the case in the Unix world where, for instance, Apache might be available out of the box but it’s not a core part of the OS. And in the third case you’ll find software that needs to be obtained from somewhere else and built or installed on the systems on which it will run — commercial software and lesser-known or newer Open Source software tends to fall into this category since it’s either unavailable for wide distribution or isn’t popular enough to come with your distribution. It’s largely these latter pieces of software which concern me the most, since they’re the ones that cause the most trouble. By providing me with an “easy” mechanism for packing and deploying a piece of software and all its dependencies, I can manage the software installed on my various systems as cohesive units. Think of it like a mason considering each brick in the wall he’s building: He might get most of his bricks prefabricated, while forging some of his own for applying a special unique style along the top, for instance. He can rely on the bricks he bought from a brickyard just as I can rely on the software packages provided with my distribution and depend on them for packages I’m building. Most of the Linux distributions have this pretty much nailed down, Solaris provides decent facilities for creating new packages, and even Windows has MSI bundles that can be deployed via Group Policy Objects in Active Directory. Mac OS X, however, is laughably bad in this area. There are no standards or even common practices for packaging software, and while the tools are available for creating them, no one’s packages can be expected to play well with anyone else’s. If you can’t count on the package infrastructure, then you can’t count on packaging.

Transparency and Accountability

I should be able to know exactly what code is going on to my systems and why.

This is Linux’s — and, to be fair, the various BSDs, and OpenSolaris as well — strength in the enterprise. When I’m installing updated software or OS bits, I want to have the opportunity to examine exactly what I’m putting on there and if things have changed, I want to know who changed them. Bug tracking systems such as Red Hat’s Bugzilla and Ubuntu’s Launchpad enable me to see precisely what changed and who changed it, leaving me with a feeling of safety and comfort that I’m not pushing changes which may cause other breakages. Furthermore, with Red Hat Enterprise Linux, Red Hat is very careful to only push bugfixes or security enhancements and not new features. You don’t necessarily want a server to have the latest and greatest features; you want it to be stable and do its job. Obviously this level of transparency isn’t possible with Windows, and only some pieces of Mac OS X are Open Source, but at least you do have a vendor you can talk to if any problems do surface.

Monitoring

I should be able to see what’s going on with my servers and services using open protocols and standardized tools.

If I want to keep track of system load, memory usage, network bandwidth consumption, temperature, or any of another number of parameters, I should be able to use the tools of my choice. Linux excels in this area by utilizing the net-snmp tools, allowing me to query anything I can conceive of using the open SNMP protocol. Mac OS X Server also utilizes net-snmp, but last I tried to work with it I had a whole host of problems trying to get a reasonably recent version to build, and then I had to wrestle with packaging the blasted thing. Our Linux boxes install it automatically during their Kickstart, but even if it had gotten left out I could either yum install net-snmp or apt-get install net-snmp and call it a day. There are SNMP agents available for Windows, as well, but as I understand it all the best ones cost money. net-snmp can also build on Windows, I believe, but I don’t even want to know about what that’d require for deployment to servers.

Hardware Availability and Affordability

I should be able to run my systems on hardware from the vendor of my choice.

Whether I go with a certain vendor because of their lower prices, better support, higher performance, or any other reason, I need the OSes I deploy to be able to run on that hardware. Linux has excellent hardware support, especially in the low- to mid-range server space, and can readily be deployed on commodity systems. Windows has exceptional hardware support, as well. Mac OS X Servers must be bought from Apple or one of Apple’s authorized resellers, and will only — officially — run Mac OS X. If I buy a system from Sun and put Linux on it, then later decide that it’d be better serving us as a Solaris box, I can do that. I could even put Windows on it. If I bought an Apple Xserve and Mac OS X Server wasn’t doing the job, I’d be out a machine and a few thousand dollars.

I believe this covers the major bases. I might update or enhance this with additional information as time goes by, so you may want to watch this space.