Enterprise Computing Capacities of the Linux* 2.4 Kernel

Submit New Article

February 22, 2009 8:00 PM PST


by Thomas Wolfgang Burger

Introduction

Does the 2.4 kernel have what it takes to make corporate America embrace Linux as an enterprise solution?

There are a large number of changes and additions to the Linux* 2.4 kernel release that will result in more speed and reliability and a much larger base of supported hardware.

These modifications have three major goals in addition to bug fixes and general improvements:

  • Greater speed, especially on large-scale servers. There should no longer be any configuration and load combination in which Linux is slow.
  • Greater market penetration, because of broader (and easier) hardware support.
  • Fewer kernel variants, in order to eliminate the plethora of custom versions; one kernel will now work, as is, on an embedded device, a laptop, a desktop, or a network server.

 

Linux ELF and POSIX Foundation Strengthened

Linux kernel 2.4 is more dependent on the ELF (Executable and Linking Format) than Linux 2.2. ELF is an advanced binary format that includes support for multiple code and data sections and increases the support for shared libraries. Fully exploiting the ELF binary format allows Linux kernel developers to increase code modularity and to simplify maintenance. ELF allows drivers to be initialized based on how they are linked, rather than by having an explicit initialization line in the core code.

With the adoption of support for POSIX clocks and timers, Linux 2.4 becomes more POSIX-compliant, allowing for non-RTC (Real Time Clock) devices to be used as clocks internally.

Enterprise Support: Threading Scheduling and Capacity

Three core changes in Linux 2.4 add greater capacity in enterprise-level environments, without degrading performance in standard environments:

  • First, Linux 2.4 can handle many more simultaneous processes or threads, because it is more scalable on multiprocessor systems and provides a configurable process limit. The threading model used in Linux 2.4 is now a scalable or soft version. Previously the thread limit was 1024, which led to poor performance with large numbers of users. The limit is now set at run time, so the only real limit is the amount of memory available.
  • Second, the scheduler has been revised to be more efficient on systems with more concurrent processes.
  • Third, capacity has been greatly increased in several areas:
    • The new kernel can handle (theoretically) about 4.2 billion users and groups.
    • 4 gigabytes of RAM is supported on Intel® architecture.
    • 16 Ethernet cards can be installed on one system.
    • 10 IDE controllers can be installed on one system.
    • The file size restriction of 2 gigabytes has been removed.

 

Resource-management code has been completely rewritten to make it much more scalable and general-purpose. This change allows for greater control over the resource-management subsystem and makes it easier to use.

Integrated Web Server

Newly integrated into Linux 2.4 is a kernel Web server, khttpd. This kernel-space http daemon¹ doesn't have to exist in user-space. It sends data to kernel-space to be taken to the network connection, resulting in faster response times. The khttpd can, however, only run static pages. Apache or another httpd will have to be used to run CGI programs, because khttpd is not designed as a replacement for Web servers. If it receives a request for CGI, khttpd will pass the request to user space where a Web server can process it.

Networking

Linux 2.4 has a completely rewritten networking layer. It has been made as un-serialized as possible, to allow for better scaling. The network subsystem is redesigned for stability on multiprocessor systems and many races² have been eliminated. There are also many updates to the existing network driver set, as well as many new devices, including support for ATM network adapters.

The Linux 2.4 rewrite includes placing firewall and Internet protocol functions into the kernel. The network subsystem has been split into two pieces: a packet filtering layer and a network address translation (NAT) layer. Each is more generic than the corresponding previous version and allows sophisticated routing through any Linux box. The rewrite also includes a new user-space tool to manage the available functionality.

Much of the PPP (point-to-point protocol) code has undergone major rewriting and modularization. The kernel now combines the PPP layers from the ISDN layer and the serial device PPP layer (like those used for dial-up connections with modems). ISDN is updated to support many new cards. The PLIP (PPP over parallel ports) layer is improved as well and now uses the new parallel-port abstraction layer and PPP over Ethernet (PPPoE) support, a protocol used by some DSL providers.

Enterprise-level changes to networking introduce features that will better enable Linux to be integrated into existing network infrastructures. Linux 2.4 adds DECnet support for interoperating with specialized Digital*/Compaq* systems and ARCNet protocols, as well as hardware that allows for better interoperation with specialized systems.

Linux 2.4 is able to auto-detect Windows* shares based on the Server Message Block (SMB) protocol. This makes Linux a better candidate for use in mixed (heterogeneous) networks. The Linux 2.4 kernel also removes the compile-time requirement of selecting support for mounting drives from Windows 9x or NT. Linux 2.4 can auto-detect the remote system type.

The UNIX network-sockets model has changed. With Linux 2.2, processes waiting on an event from a network socket were all awakened at event detection. Only one succeeded in getting the event, and the rest put themselves back to sleep. Eliminating this "stampede effect" increases network speed significantly.

The IPv4 implementation is more scalable, and the use of colon-mode for IP "aliasing" has been removed. Linux is still the only operating system completely compatible with the IPv4 specification. The newer IPv6 is being incorporated into the 2.5 development kernel and should make it into release 2.6.

¹ Definition of a daemon
The ancient Greeks defined a daemon as a protective spirit that presided over a person's destiny. In UNIX, a daemon (Disk And Execution MONitor) is a server process that offers services. It runs in the background and generally lies dormant until a local or remote process calls it.

² Note on "races"
A race is a software condition that occurs when the execution of a program is affected by the order and timing of a thread's execution. Race conditions can be corrected using synchronization variables to control the threads' timing and access of shared resources. If a program depends on order of execution, then threading that program may not provide a good solution, because the order in which threads execute is non-deterministic. Programming languages that support threading have mechanisms to force thread synchronization.

Improved Hardware Support

One key aspect of Linux release 2.4's bid for greater market penetration is vastly-improved hardware support.

CPU Support
Linux 2.4 adds support for three new architectures: Intel® Itanium® processor-based systems, IBM S/390*, and Hitachi SuperH*. Changes made to the 2.4 kernel are based on the current specifications and existing support for current 64-bit CPU architectures like the Alpha* and Sparc64*.

Linux 2.4 hardware support, in general, is very similar to Linux 2.2. All generations of processors from the Intel386™ processor through the Intel® Pentium® 4 processor are supported.

Support for high-end hardware to increase speeds comes from support of non-Intel varieties of the Memory Type Range Registers (MTRR) for the AMD K7* processors and the Cyrix* processors' variation called MCR, which will improve performance on some high bandwidth devices. Where Linux 2.2 included support for the Memory Type Range Registers (MTRR) used on the newest Intel processors to increase performance of some kinds of high-bandwidth devices, Linux 2.4 takes this even further by supporting MTRR (and MCR) variants from AMD and Cyrix.

The 2.2 kernel scales well on up to four processors. Linux 2.4 supports 16 or more processors, depending on the architecture.

Device Nodes
The Linux 2.4 kernel implements a "raw" I/O device, which is a completely new block-device feature. A raw device is a block device with the ability to access a low-level device directly (instead of through the cache layer), immediately and concurrently with the data (whether the data is on the disk or elsewhere). A raw device is especially useful in cases where a sophisticated application wants complete control over its data caching but doesn't want the expense of the usual cache. A raw device can be used in data-critical situations - for example, if you need to ensure that the data is written to the disk immediately so that no data is lost in the event of a system failure.

The concept of raw-device nodes was adapted from mainstream UNIX. This raw device I/O came from a traditional UNIX implementation that involved the creation of "raw" devices corresponding to every block device. Linux developers used a pool of raw-device nodes rather than the method of correspondence used in UNIX.

Extensions to the concept of pooling device nodes, such as ejecting a disk, are handled through ioctls (I/O Controls). All the block device drivers have, to some extent, been rewritten for Linux 2.4. The API has been changed to remove legacy garbage from the interface and to completely separate the block and file APIs at the kernel level. Nevertheless, the changes to the block device drivers are not major. (For coders who maintain modules outside of the main Linux development tree, it is important to check whether or not you require an update.)

Journaling File Systems
For Linux to be marketed as an enterprise OS, it requires a journaling file system³ feature. The 2.4 feature freeze meant that a journaling file system was not included as a standard distribution element. However, several vendors have developed systems that can be added:

  • IBM's Journaled File System Technology* for Linux
  • ext3*: New Journaling File System in Red Hat
  • ReiserFS*
  • XFS: A high-performance journaling file system

 

Bus Support
A large portion of the Linux 2.4 improvements and additions concern internal and external bus support for a variety of hardware platforms and configurations. Improved bus support means that Linux 2.4 can be used or adapted to almost all personal computers.

  • Universal Serial Bus: Universal Serial Bus (USB) support is currently available in the developer version of Linux 2.3.x and has been successful with USB past the 2.3.15 level. Kernel USB support (as of 2.3.99-pre9) includes USB scanners, printers, cameras, audio, joysticks, keyboards, mice, and modems.
  • PCMCIA support: Personal Computer Memory Card International Association (PCMCIA) devices are found mainly in laptops and handheld devices. Support for them has been placed into the standard kernel instead of a custom kernel release like 2.2.5-15 (the -15 indicates PCMCIA support). Placing it into the standard kernel eliminates the problem that the PCMCIA distribution-specific kernels are behind the current Linux 2.4 release.
  • Standard buses: The existing bus drivers (ISA, PCI, and MCA) have been improved, and more devices are being supported on many of them, including Micro Channel Architecture. One Linux 2.4 improvement to bus support has been stringing the drivers into a new Linux resource subsystem.
  • The I2O Intelligent Input/Output (I2O) bus is also planned for the Linux 2.4release. I2O is a superset of the PCI bus. Its purpose is to allow OS-independent drivers to be written for many of the devices. Capability to write the drivers will allow for more devices and drivers that are independent of operating systems to exist in the future. I2O SCSI controllers and other I20 devices are already supported.
  • SCSI: Linux 2.4 will also include a number of new SCSI devices. The SCSI subsystem has been largely rewritten. A number of new SCSI controllers are supported in this release, and further SCSI improvements are expected sometime during the 2.5 development cycle.
  • Logical Volume Manager (LVM) is a system that is standard in enterprise-class UNIX versions such as HP-UX* and Tru64 UNIX* (formerly Digital UNIX*). It manages file systems and vol umes. The LVM subsystem has been moved into the mainstream kernel. The LVM allows file systems to span disks and be resized and managed more flexibly than with partition tables. The LVM subsystem can be replicated with the md (multiple devices) driver, or with a user-space tool. The Linux 2.4 version of the LVM subsystem offers this support in a (de facto) standards-compliant manner that will be familiar to commercial UNIX users.
  • IDE: In addition to the increase in the number of IDE controllers, IDE CD changers are now also supported. Linux does not yet support the capability to mount multiple CDs in the changer at the same time, however. Linux 2.4 is also smarter in terms of detecting IDE controllers that have buggy chipsets and working around them. Moreover, it better supports advanced features like ATA66.
  • Firewire support is included in the Linux 2.4 kernel. Firewire is a popular option for many high-bandwidth devices, although not many drivers or devices exist for this architecture yet. Firewire is standard on the Macintosh* and is fast enough for connecting to external hard drives. It is mainly used for digital video devices.

 

³ Note on "journaling file systems"
A journaling file system is, very simply, one that stores a record of a read or write transaction before the system actually performs it. If the system crashes, the record of what was to be done remains intact, and the transaction can be completed when the system is restored.

The Future

The future of Linux, beyond the release of 2.4, will put emphasis on improvements for desktop environments, embedded devices, and enterprise-level systems.

Linux is well established as a server OS, but because of problems such as installation and configuration complexity, a lack of hardware community support, and a lack of current market momentum, it has not made much of an inroad as a desktop OS. Perhaps a more successful target will be the fast-growing Web-appliance/information-device/handheld-computer embedded market. Because Linux is inherently small and flexible, this market seems to provide its greatest consumer potential.

The current 2.5 development focus is on server support with a new networking device interface, improved networking performance, and asynchronous I/O support, all working to improve Linux's position in the corporate world. The official release date for 2.6 (development and release versions are numbered differently for clarity) is set for late 2002, although this is the earliest time possible in the eighteen-month release window.

Other planned changes include the following:

  • Increasing scalability beyond the 2 Terabyte device limit
  • Support for asynchronous I/O, to speed up enterprise database performance
  • Support for new IETF-approved Internet networking protocols
  • Enhanced support for Intel's Advanced Configuration and Power Management (ACPI) standard, including support for hibernate and sleep mode.
  • Support new HotPlug architectures to enhance hot-swapping capabilities
  • Enhancements for ccNUMA and SMP clustering

 

Conclusion

The Linux 2.4 kernel is the first in a series of steps for Linux to addres s the needs of Enterprise Computing. Development efforts concern not only making changes and additions to Linux itself, but also in the way Linux is developed, to ensure corporations can trust new Linux releases to be stable, timely, and relevant.

Related Resources

Related Content

 

About the Author

Thomas Wolfgang Burger is the owner of Thomas Wolfgang Burger Consulting. He has been a consultant, instructor, analyst and applications developer since 1978.