Linux DevCenter    
 Published on Linux DevCenter (http://www.linuxdevcenter.com/)
 See this if you're having trouble printing code examples


Linux in the Enterprise

Consolidating Servers Under Linux

08/16/2001

In the last article in this series, I talked about how to make sure you're not "downsized" out of a job during the current IT slump. In this piece, we'll explore another kind of downsizing that is far less painful and may help you maintain your position in this turbulent time.

"Server consolidation" is a term that is often used in IT jargon to mean "we're getting rid of platform X" where "X" is SPARC, Intel, Hewlett-Packard, or some other unlucky victim of a corporate popularity contest. These days, you should be thinking about a different kind of server consolidation -- how to reduce the number of servers that your company has to support and how to bring more services together on the servers that remain.

Multiprocessor machines that can run 1.2GHz processors and multi-gigabyte memories present a compelling opportunity to get back some floor space and some money back into the IT budget by selling all those surplus machines on eBay.

Consolidating services under Linux: Part one

By now it's obvious to most people that Unix in general (and Linux in particular) is a work-horse when it comes to server-based applications such as email, web services, and other things that just don't belong on a desktop.

What most people haven't gotten too comfortable with is the idea that these servers are actually powerful enough to support multiple applications at the same time. For some reason, many shops still have have a "one-machine, one-service" orientation. For some high-volume applications this makes sense. For example, joining email and web servers is probably not a good idea for an external web server -- there are too many security issues that run together and in a hostile external environment, you want to button things up as tightly as possible.

However, for internal applications using a single machine for a web server, mail server, and file server can make sense if you have a powerful enough machine, enough I/O capbilities, and the load presented by any one application isn't excessive. Fortunately for most businesses, an intranet web site and email do not represent machine-destroying applications. File services on the other hand can really bring a machine to its knees.

The best way to make sure that you have enough resources is to do some simple profiling of your applications and then configure the new consolidated machine to match the sum of the resources (especially in terms of memory) of the machines you're replacing. One of the most important things you can do is to make sure that your data is spread across multiple drive spindles.

Comment on this articleHave you run into any services that just don't work well on the same box?
Post your comments

Also in Linux in the Enterprise:

Unfinished Business Part 2: Closing the Circle

Unfinished Business: The One Missing Piece

It's a Cycle of Life Thing: Managing Linux Releases

In the old (and I mean old, like the 1960s) days, disk drives had multiple moving heads that made random file access over a single disk a reasonable proposition (of course, the washing-machine-sized drives of the day could only store about 200MB, but that's another story). As hard disks matured, they traded off their multiple moving heads for overall speed and, of course, incredible storage capabilities.

Unfortunately, this means that for modern applications the real bottleneck in disk I/O performance is not so much a factor of the disk interface (modern SCSI and IDE interfaces are plenty fast) but how many read-write heads can be put into service at once.

The best way to ensure that you have a lot of heads moving is to use some flavor of RAID (Redundant Array of Inexpensive Disks) in laying out your storage. This can be something as simple as disk striping (known as RAID-0) where data is split between two disk spindles, or a full blown RAID-5 where data is split among five (or more) disks. There are also choices to be made about when to use the software RAID that comes with Linux, and when a hardware RAID card or an external RAID system makes more sense. You can always start at the low-end and use the tools built into most Linux distributions and migrate upward as your business needs dictate.

Consolidating services under Linux: Part two

Another way to consolidate servers if you can't run all of your applications under Linux (if you're tied to Windows for Workgroups, for example) is to downsize your server count though hardware emulation.

The emulation of computer systems, more popularly known as "virtual machines" has been around for a long time. IBM pioneered virtual machine technology back in the 1960s as a way to allow its customers to retain their investment in software by allowing emulation of entire computer systems to run on their latest hardware. On a smaller scale, virtual machines have been used for years in everything from USCD Pascal to Java, allowing programmers to write software for one set of APIs that has the capability to run on multiple hardware platforms. For our uses, we're interested in figuring out how to use one or more virtual machines to run a number of different operating systems on one physical box.

Most people have heard of VMWare and its virtual machine for emulation of desktop systems, VMWare Desktop. It's a great boon if you run a Linux laptop or desktop system but still need to run applications such as Visio or Microsoft Office. VMWare is a versatile performer. It allows you to run not only Windows (3.1 all the way up through XP), but you can also run Net/Open BSD or even another Linux virtual machine -- you can even run all of these at the same time.

One drawback, until recently, was that VMWare was a good desktop/laptop solution but it didn't scale to the enterprise. There are issues such as CPU performance and I/O bandwidth if you are trying to force a desktop machine (and VMWare Desktop) into server-level duty. Recently VMWare released new versions of its virtual machine software that can be used to run large-scale servers. The new enterprise versions of VMWare, called VMWare ESX and VMWare GSX, allow VMWare to be used for data center consolidation or departmental consolidation respectively.

The ESX version actually runs its own kernel and can host 4 to 20 Linux, Windows, or BSD virtual machines and supports 1Gb per second of network bandwidth. This is really an industrial-strength solution to support database machines and streaming content for those who want to cut down the number of physical servers they support in a data center while maintaining a single management interface.

The GSX version of VMWare is meant for departmental servers and runs on top of Linux (or Windows) and supports four to eight virtual machines. This is the perfect solution for businesses or departments that either have too many machines and not enough support staff, or have seen their IT budgets slashed and need to support more applications on disparate operating systems.

Once of the nice features of VMWare's enterprise solutions is their web-based interface that allows a system administrator to build and control all of the virtual machines from a single unified interface. This means that you can dispense with KVM switches and manage virtual machines right from your desktop.

Screen shot.
Click for full-size view. (Image courtesy VMWare, Inc.)

VMWare's solutions are not cheap -- a VMWare enterprise license can run $2,500 -- but given the number of machines they can replace and the value they add in terms of being able to support more applications on your existing hardware they're quite a bargain.

Resources

David HM Spector is President & CEO of Really Fast Systems, LLC, an infrastructure consulting and product development company based in New York


Read more Linux in the Enterprise columns.

Return to the Linux DevCenter.

Copyright © 2009 O'Reilly Media, Inc.