The Internet helps businesses grow by cutting costs and increasing efficiency. The “always on” nature of the Internet is pushing organizations, such as banks and online stores, to deliver high-availability services to an ever-increasing number of users. In such a context, downtime means an immediate loss of revenue.

The adoption of Linux as the platform of choice for high-availability clusters is driven by a number of factors. First, many organizations find that Linux clusters match or exceed the performance of proprietary solutions while dramatically reducing the total cost of ownership (TCO). Lower TCOs are the consequence of combining inexpensive commodity hardware with open source software. Second, the open source character of Linux gives organizations the flexibility to tailor a solution to fit their specific needs. Finally, by using open source software, organizations avoid licensing fees that can otherwise become prohibitive as the number of nodes increases in a cluster, and also avoid vendor lock-in.

What is High Availability?

Mission critical services and systems must provide high levels of availability and reliability. High availability (HA) makes critical services and systems as resilient as possible against unplanned outages due to hardware, software, network or power failures.

Reliability is usually defined in terms of “mean time between failures” (MTBF). This is a measure, in number of hours, of how reliable a hardware component is. For example, a typical hard disk drive may have a mean time between failures of 300,000 hours. Hardware manufacturers develop a figure for MTBF by intensive testing and analysis of known factors. Given the MTBF, customers can get an idea of how much service to plan for. Actual downtime, “mean time to repair” (MTTR), is a measurement of the time required to bring a failed system or service back on line.

Availability is defined as the probability that a system is available at any given instant, and can be expressed in terms of MTBF and MTTR with the formula…

Availability = MTBF/(MTTR + MTBF)

So, availability can be maximized by increasing the time between failures and decreasing the time required to restore a failed system.

What causes service disruptions?

Service disruptions can be either planned or unplanned. Planned events are those that have been determined ahead of time, usually as part of a maintenance schedule. Upgrades to hardware and software or system configuration changes, may require the system to be rebooted or powered down, resulting in downtime.

Most service disruptions are caused by unplanned events. Software bugs in applications, operating systems or device drivers, coupled with system design problems, account for almost 40 per cent of the downtime caused by unplanned events. Another 30 per cent is due to hardware problems, such as failure of moving parts, like CPU fans, power supply fans and hard disk drives. The remaining 30 per cent is due to operator error, usually unplugging the wrong power or network cable. Disruptions that are beyond the scope of HA system design are breaches in physical system security and natural disasters (such as floods, fires, earthquakes).

Designing an HA cluster

Performance, redundancy and data sharing are key factors to consider when designing a high-availability system. It is important to look for single points of failure (SPOFs) in your design. These are components, hardware or software, whose failure can bring down the whole HA system. Analyzing these factors will help you shape an HA system configuration tailored to meet your requirements.

Performance

One of the components of HA performance is the amount of time an HA configuration takes to recover from a failure. In other words, how long will a critical service be unavailable before it is noticed by users. Much of the design process will be structured around this time constraint. To determine the overall time constraint, we have to consider fault detection, notification and recovery. Fault detection is typically implemented by nodes exchanging “heartbeat” messages via Ethernet or serial links. To ensure reliable interconnection for heartbeat messaging, dedicated Ethernet or serial ports on each node are connected via crossover or null modem cables. The actual heartbeat message is usually a simple string like “node is up”. A node that fails to provide heartbeats after the preset time interval has elapsed is considered failed by peer nodes. This elapsed time interval before notification and the heartbeat polling interval should be tuned to meet the time requirements of the specific applications running on the cluster.

When a fault is detected, the cluster management application performs a recovery procedure known as a “failover”. The failed node is deactivated and a substitute node takes over its workload. The amount of time necessary for the failover depends on the number and complexity of the steps required for initializing specific applications that were running on the failed node. If the cluster does not use shared external storage, then data synchronization activities, like replaying large database query logs or performing file system integrity checks on large disk partitions, can take some time. With the emergence of journaling file systems such as ext3 and ReiserFS, lengthy integrity checks are becoming a thing of the past.

Redundancy

Redundancy provides the basis for recovery after fault detection. The idea is simple: if one node fails, another should take over its function, with minimal impact on users. Redundancy in an HA system is configured according to how applications will be distributed across individual nodes in the cluster. The two most commonly used configurations in dual-node systems are active-active and active-passive.

In an active-active configuration each node in the cluster runs one or more unrelated application, sharing the total cluster workload. If a node fails, the other one will pick up the load of running all its services. Although active-active configurations are more difficult to implement, the financial investment in replicated hardware may be worth the dramatic improvements in load sharing that are possible. The total capacity of the HA cluster is maximized because all nodes are active and hardware is not just sitting idle waiting for a failure. In a dual-node system, while cluster performance is maximized when both nodes are active, it is reduced to half when one of them fails. This should be kept in mind when deciding which configuration to go with for your HA cluster.

An active-passive configuration is desirable when a single large application needs to be run on the cluster. Few applications support concurrent data access models and are consequently restricted to running on a single node. In a dual-node system, a primary active node handles all requests, while a secondary passive node sits idle. The unused computing power can be used by running additional applications to replicate and/or synchronize data between both nodes, so reducing downtime in the event of a failover. One of the benefits of active-passive clusters is that overall performance remains constant, since only one node is active at any given time.

Data sharing

Online services, such as banking and travel reservations, demand high-performance shared disk storage solutions, which are always available. In addition to high performance and availability, a shared disk storage solution should provide data sharing across all nodes and disks. A cluster file system, such as Sistina’s Global File System (GFS), allows multiple nodes to share a single, common view of storage data, mitigating data synchronization issues. GFS provides locking mechanisms to allow multiple simultaneous writes to the same file, storage allocation control by assigning quotas to applications, data journaling for small file performance, and out-of-box integration with Red Hat and SuSE. Red Hat has promised to make GFS open source, following its acquisition of Sistina late last year.

Linux HA solutions

Whether you want to build your own HA cluster or buy an off-the-shelf solution, here is a brief look at some of the well-known high availability solutions that are being used by many large organizations building and deploying Linux clusters.

Linux HA project: Widely used in many real-world high availability solutions, Linux HA implements network and serial heartbeating for node monitoring and combines multiple IP address takeover for simple dual-node active-passive clusters. Find out more at linux-ha.org.

Linux Virtual Server (LVS): LVS achieves scalability and high availability by virtualizing a cluster of real servers, which are front-ended with load balancers. Users interact with this cluster as if it were only a single virtual server, allowing nodes to be added or removed transparently to improve scalability. Similarly, node failure detection reconfigures the cluster as needed to deliver high availability. Find out more at www.linuxvirtualserver.org.

Red Hat Cluster Suite: Originally based on Kimberlite from MissionCriticalLinux, this solution is well suited for database, file server and Web server applications. It offers both active-active and active-passive configurations along with GUI management software. Additionally, you can combine Red Hat Enterprise Linux and Piranha load balancing software to deliver a variety of high availability services. Find out more at www.redhat.com/software/rha/cluster.

EmicNetworks: The Emic Application Cluster (EAC) software targets high performance, highly available database applications, and Web servers with dynamic load balancing as well as tools for failover and recovery. Find out more at www.emicnetworks.com.

Conclusion

By using a combination of open source software components, successful online services, like Amazon and Google, have redefined the standard for high-availability applications. With the emergence of frameworks like High Availability Open Source Cluster Application Resources (HA-OSCAR) (www.cenit.latech.edu/oscar), we will see ideas from the worlds of high performance computing (HPC) and high availability converge to deliver even more innovative and powerful solutions in the near future.

The best OSS efforts spring up around problems of compelling technical interest. Web services, system management, office automation and operating systems, function like subject areas in a college curriculum and have already yielded best-of-breed projects. Like classes in a curriculum, each project must articulate a shared objective, which is usually expressed as a desire to demonstrate a capability or service. Success pre-supposes appropriate technical ability and dedication to the articulated goals by at least a sustaining set of participants.

To continue this analogy, just as a curriculum benefits from outstanding faculty, OSS projects prosper by the guidance of benevolent, high-quality technical leadership. In the same way that a faculty body evolves, the best OSS projects incrementally improve by the simultaneous accretion and pruning of the working team over multiple generations. Practicing leaders, like college deans, must nurture collaboration and contribution and organize the constituency of participants. Great OSS deans today include Rasmus Lerdorf of PHP, Guido van Rossum of Python, and, of course, Linus Torvalds of Linux.

Cathedral Model Lacks Flexibility

Some projects are initiated from scratch, but many are born from earlier work. Large-scale software projects may fail if built and maintained using the so-called cathedral model of development because, as Eric S. Raymond (ESR) has famously asserted, its monolithic design and development style is too rigid. Not accidentally, however, some of the most influential OSS projects have actually started out as cathedrals. OpenOffice.org is a large, proprietary product that Sun Microsystems bought in 1999 and turned into OSS. Mozilla is an extraordinarily complex codebase which Netscape re-purposed as OSS in 1998. Clear objectives and discipline in an initial cathedral foundation can result in a successful launch, while the collaboration and adaptability of OSS can guide the subsequent evolution of the project.

Bazaar Model Lacks Collaboration

Contrary to popular wisdom in the OSS community, however, grand OSS projects cannot be built using the alternative “bazaar” model. Actually the bazaar is an improper metaphor. Romantic notions notwithstanding, the principles underlying the fluid market exchange of the bazaar are inappropriate for software development. Essentially the bazaar is an environment of mutual distrust. It is a fiercely competitive marketplace that holds little respect for its consumers or producers, and even less for honesty.

Collaboration, Iteration, Progress

Grand OSS projects demand an unswerving dedication to the pursuit of truth. They are more akin to setting up a college, than working in a bazaar. The explicit goal of a college is to increase the effective knowledge of all participants and to provide the tools needed to accomplish this goal. Within their domains of interest, grand OSS projects aspire to the same objectives and use the same methods. Participants, as colleagues, collaborate about software design and implementation. Details are continuously reviewed, enhanced and pruned. At its best, each OSS project forms an evolving center of knowledge and practice.

This alternative collegiate model fits well for governments who want to explore Open Source methods. It would be foolish for any legitimate government program to be drawn into the chaos of a bazaar model of development. And while it is easy for governments to imagine that their large software requirements can be implemented by cathedral-like building blocks and controls, history is littered with obsolete and forgotten attempts. Chances of success are improved when the government selects a software model which itself embodies collaborative and evolutionary processes of knowledge and technology development. Furthermore, just as OpenOffice.org and Mozilla have proved the success of open sourcing previously closed projects, governments can examine modernizing some of their repertoire of projects by re-purposing appropriate existing projects as open source.

The Red Hat Open Source Desktop product adds to the growing landscape of Linux client systems now being offered from Sun (JDS), Novell (XD2), Linspire, Lycoris, Mandrake and others all claimants to the throne currently ruled by Microsoft.

Earlier this year, when Red Hat’s CEO Matthew Szulik was visiting India, he mentioned that the market was ready for a Linux desktop revolution and that Red Hat was working on a new product “Red Hat Desktop”. This desktop package was going to target the enterprise and small-to-medium size businesses that need inexpensive, yet high quality, stable and supported client systems. “The ambitions are grand, but the expectations are going to be moderate at the outset,” Szulik said in an interview. “What we’re focused on for the next 12 to 18 months is doing a great job in the enterprise, the government and academic marketplaces.”

The Red Hat Open Source Desktop package has the classic combination of personal productivity tools, such as Evolution for e-mail, Open Office for word processing, spreadsheets, presentations and the like, Mozilla for Web browsing, GAIM for instant messaging (IM), professional Agfa Monotype fonts and other desktop applications, an Apache Web server, and Samba and NFS for file sharing. It also comes with full support and software updates through the Red Hat Network at an annual subscription price tag of $2500 for ten clients and $3500 for 50 clients (about $70 per system, per year). Red Hat doesn’t sell individual subscriptions - instead, it’s offering bundled subscriptions.

Red Hat Desktop is compatible with other Red Hat Enterprise Linux products and provides complementary technology and services. It is available in configurations that include Red Hat Network Proxy or Satellite Server, facilitating manageable and secure deployment. Red Hat Desktop supports single CPU systems with up to 4 GB of main memory.

Installation

Method

To install Red Hat Desktop, we used a bootable floppy disk to load the installer. The boot floppy diskette can be created from the images/boot.img file on the first installation CD. We also needed a network drivers diskette (from the drvnet.img image on the same CD). We chose the NFS installation method since it saved us from burning CDs of the ISO disks. This assumes that you have another Linux or UNIX system available to act as the NFS server.

Installation process notes

The graphical installer is similar to other Anaconda-based Red Hat distributions. The first couple of screens for the configuring language and keyboard layout are text-based. The actual graphical installer is launched after successfully detecting your video card, monitor and mouse.

Next, you are presented with the disk partition configuration screens. You have two options to choose from. You can let the installer “Automatically partition” your drive(s), or you can “Manually partition” them using Disk Druid. Most users will opt for automatic partitioning. However, if you want to preserve data located in a particular partition, either on the primary or a secondary hard drive, you will want to configure the partitions manually.

Next is a series of screens to configure your system s mouse, firewall, network interfaces, time zone and root password. Then the package selection screen is presented. You can choose to install the “Default” packages or “Customize the set of packages”. The default packages include the Desktop shell (GNOME), Office Suite (OpenOffice), Web browser (Mozilla), e-mail (Evolution), instant messaging, sound and video applications, games, and administration tools. I chose to install the “Default” packages.

After you complete package selection, the installer performs a check for package dependencies. If no conflicts are found, the disk partitions are formatted. The actual package installation process begins now. The default package set consists of 528 individual RPM files totaling 1,827 MB. Package installation takes about 15 minutes.

Once package installation is complete, you are asked to configure the resolution and color depth for your desktop environment. After this, your system is rebooted, ending the installation phase.

First boot

Upon boot up you will notice that the usual display of system messages has been replaced by Red Hat Graphical Boot, a simple computer icon with a progress bar below it. You can press “F2″ to see the messages. Since this is the first boot up, a program called “firstboot” is launched. This presents you with a set of post-install configuration screens for the license agreement, system date/time (NTP) setup, additional user account creation, Red Hat network registration, and software installation from additional CDs. Once you complete the “firstboot” screens, you will see a graphical log in screen, similar to Fedora Core 1. After you type in the user name and password, the Red Hat Desktop finally appears.

Red Hat Desktop

Red Hat Desktop comes with slightly dated versions of GNOME (v2.2), OpenOffice (v1.1.0), Mozilla (v1.4.2), and the Linux kernel (v2.4.21). With the recent release of Fedora Core 2, customers of Red Hat Desktop will be a little behind the times. But this lagging behind is supposed to be offset by the vendor’s promise of greater testing and integration of stable components.

Although Mozilla is a very capable Web browser, Red Hat does not provide the standard plug-ins for handling Macromedia Flash or Java. So the user ends up having to download and install these separately. The selection of fonts available to the user is satisfactory for creating simple OpenOffice documents and presentations.

Evolution does a great job of providing the user with a complete e-mail/calendaring solution. It supports standard mail protocols (POP3, imap) and message storage formats. Plus it provides useful components to handle calendaring, address books, and daily task lists. Users waiting to try out GIMP 2 will have to wait a bit longer until Red Hat integrates GNOME 2.4, since GIMP 2 will not work with the older GNOME 2.2.

Industry Comments

From the vendor - Red Hat

At a press conference in London this May, Matthew Szulik discussed how Red Hat’s client strategy was forged from the need for choice in the technology industry to drive innovation and forward thinking. Javed Tapia, director, Red Hat India, said, “The launch of Red Hat Desktop spells Red Hat’s commitment to the client environment. We have ensured that Red Hat Desktop provides a complete user experience with e-mail, Web browsing, instant messaging, office suite, imaging and multimedia, plus installation support and a year of upgrades and updates.” With Red Hat Desktop, Red Hat plans to address the growing demand for PCs in India. He added, “In India, PC manufacturers have already started offering Linux OS with their products. This has fueled the demand for PCs and laptops as prices have plummeted. In fact, India is a step ahead of the rest of the world when it comes to the adoption of Linux on the desktop. Governments, corporates and even the education segment have already started using Linux on the desktop.”

From integrators and users

Major industry players seem open to this new desktop.

HP: “A key component of HP’s Linux strategy is to provide its customers choice, and Red Hat’s initiative to support Linux on the desktop aligns with that vision,” spokesperson Elizabeth Phillips said.

Dell: “We don’t see a significant demand from our customers for factory installation of Linux on the desktop, but we’ll continue to monitor what customers want, including products from key software partners like Red Hat,” spokesperson Carmen Maverick said.

LVM: “We are considering the Red Hat Desktop solution because of the open source merits of security, manageability and cost-effectiveness,” said Matthias Strelow, technical team leader at LVM, one of the largest insurance companies in Germany. LVM is a heavy user of Linux at the desktop level, with more than 8,400 currently deployed Linux desktops.

The Challenge

Red Hat Open Source Desktop is an attempt to offer a level of integration for the client desktop, which is competitive with what is offered by Microsoft. But whether or not it can begin to unseat Microsoft dominance, only time will tell.

A problem with the current suite of open source desktop products is how to balance stability and reliability with incorporating the latest features and innovation. New features and rapid innovation are the major attractions of open source, but tend to be less stable. So today’s desktop products from major companies use older, more tested components. This strategy is well suited for the enterprise customer, but may be less successful in the consumer market.

So the next challenge is to increase the range of features and innovations packaged in the OSS desktop, yet ensure an ease of use and stability that individual consumers can rely on. As the OSS desktop platform matures, even your grandma should have the best of both worlds.

In the life of a company, an IPO is celebrated only once and then its whole world changes. On April 29, 2004, Google filed for an IPO, and then there was no looking back. Everyone and everything now is in preparation for “Going Public Day”. Different people at Google will behave in ways yet unknown to them, reacting to their new-found wealth. But one result is known the Google parking lot will showcase even more of the latest works of German engineering.

The financials are pretty impressive: almost a billion dollars in revenues in the last year, 1907 employees, profits in excess of $100 million and growing. After six years of focused effort to be crowned king of the search industry, all run on private investment and management, Google is ready to raise a serious $2.7 billion ($2,718,281,828 dollars to be exact - the value of the mathematical constant “e” - in true Google style) from the IPO and have a market capitalization of $20 billion to $25 billion.

Run Private, Go Public

Both the company’s founders, Brin and Page, intend to preserve Google’s innovation-driven culture by creating a dual-class system of shares with two kinds of voting rights. Class A shares, to be sold in the public offering, will have one vote per share, whereas Class B shares, held by the founders, executives and employees, will have 10 votes per share so allowing the founders to maintain their decision-making authority. This set-up also supports Google’s philosophy as stated in a seven-page letter by the founders - of creating a company which intends to stay in business long-term and provide “long-term” value, opportunities, success and growth to “make the world a better place.” The IPO is going to be run as an innovative online Dutch auction, which makes both the big and small investors compete fairly for shares, democratizes the IPO process and minimizes bank fees too. Definitely, not a traditional Wall Street IPO and definitely Google.

Cool Technology

Cutting-edge innovation has always been Google’s recipe for success. Technology such as “Page-Rank”, a unique search algorithm based on quantifying the number of links to a page, Web alerts, personalized Web searches, and its smart scheme “AdWords” of paid keyword advertisements linked to search results, has given Google the ability to outrun its competition at critical times. Also, running all this search technology, day-to-day systems management and caching four full copies of the Internet’s content using clusters of commodity hardware and robust Linux and Open Source software, has given the company tremendous cost effectiveness and a technology edge that is hard to beat. Google employs Linux for everything it does on servers and desktops for development, system administration, and research. Using the world’s largest Linux cluster, Google has also opened its programming interface so developers can query over four billion Web pages from their own programs.

Are We Ready to have Fun?

Life today without Google is unimaginable. Google has already become synonymous with the word “search” in the unofficial dictionary of the English language. Wall Street is infatuated, Main Street is excited and Silicon Valley is energized. The Securities and Exchange Commission’s (SEC) website, which hosted Google’s IPO registration filing, is getting overwhelmed with hits. Once Google goes public, the search competition will intensify. For the first time, Google will have the challenges of intense scrutiny in a public company and, at the same time, it will have to maintain its flow of innovative technology to pulsate its engineers and balance sheets alike.

Google is already getting rated as being as relevant as Microsoft or Cisco. So, watch the IPO happen in the next few months. Hope you’re ready for Googlopoly!