Linux Affinity

Richard Ferri

The success of Linux and open source over the recent months has made many of the traditionally high-end UNIX vendors take notice. IBM, Sun Microsystems, and Hewlett-Packard have all announced steps to embrace Linux on their traditionally high-end, enterprise UNIX operating systems, AIX, Solaris, and HP-UX, respectively. All three companies have either provided a method of porting applications from Linux to their operating system or ported a number of Linux applications and made them available to the public, or both. What interest do these time tested, UNIX vendors have in this upstart Linux? After all, it's clear to those who follow UNIX that Linux is the least mature operating system of the four. Why would these lofty distributors care about embracing Linux?

Strategic Approaches to Linux Affinity

If you look at the explosion of the open source community, examine the curriculum among college and graduate students, and check out various publishers' book offerings, the signs are clearly there -- Linux is coming. With its acceptance by the major UNIX vendors, one might even say that Linux has arrived. Given the number of talented developers working in Linux and open source, the number of machines now installed with Linux, the skill set that new programmers coming out of college have acquired, IBM, Sun, and HP have all realized that Linux skills and numbers of devotees are growing at a faster rate than for their respective operating systems. So, why not accommodate these new Linux devotees with the best of both worlds? They can have the look and feel of Linux, and their favorite Linux applications, on more mature operating systems like AIX, Solaris, and HP-UX.

The three vendors have taken similar approaches to Linux affinity. HP-UX provides libhplx, which is a library that contains about 200 Linux-compatible APIs, related file headers, and their sources. This library is a set of APIs not available with HP-UX, taken from GNU 2.1.3, and is available under the GNU Public License (GPL). At LinuxWorld Expo in February, HP announced that they will be shipping the open source Ximian GNOME Desktop as their default desktop for HP-UX workstations. HP also announced they will be using GNOME as the open standard to provide a common desktop for both their HP-UX and HP Linux workstations. Similarly, Sun has announced that it will be using a GNOME-based desktop for Solaris. Gnome provides Solaris with programming libraries, and access to Sun's Star Office desktop office suite. The intent is that the user will not be able to tell if he's running RedHat Linux or Solaris under the hood (ZDNet). As the result of collaboration between Sun and the open source lxrun project, the lxrun utility allows Linux binaries to run without modification on Solaris Operating Environment on Intel platforms. lxrun is a software layer that sits between Solaris and the Linux Intel binary executable and remaps system calls "on the fly" allowing them to run unmodified on Solaris (http://www.sun.com). lxrun appears to provide a robust environment, supporting such diverse applications as WordPerfect, GNOME, GIMP, and Quake III.

IBM has made a big commitment to Linux affinity with its AIX operating system. After all, the "L" in the AIX 5L release due out later this year is for "Linux". Many applications developed on and for Linux can run on AIX 5L with a simple recompilation of the source code, allowing customers to combine Linux applications with the advanced scalability and availability features of AIX. Linux affinity on AIX includes Linux application source compatibility, compliance with emerging Linux standards, and a GNU/Linux build-time environment with tools and utilities that combine to facilitate the development and deployment of Linux applications on AIX 5L (http://www.ibm.com/servers/aix/overview/linux.html). Having given a flavor of what is going on in the industry with regard to Linux affinity, I will explore how IBM is going about providing Linux affinity in AIX.

AIX/Linux Affinity

IBM plans to provide Linux affinity in two phases. The first phase is the release of the AIX Toolbox for Linux Applications, available at http://www.ibm.com/aix. The AIX Toolbox for Linux Applications provides GNU and other commonly used tools for recompiling Linux applications on AIX. The Toolbox provides an API approach for Linux application affinity. This approach does require a recompile of the Linux application for AIX using the traditional AIX APIs and the Toolbox APIs. It does not provide binary compatibility -- no attempt is made to run Linux applications directly on AIX. Given that the Linux applications are completely recompiled with the AIX APIs and Toolbox, these applications have access to the reliability, scalability, and availability traditionally associated with AIX (see Hebert whitepaper in references section).

The second phase of the Linux affinity plan is an integration of Linux APIs and header files into AIX 5L Version 5.1. Since AIX itself was developed to UNIX industry standards, there is inherently a high degree of compatibility between AIX and Linux. Many Linux applications can be recompiled and run on AIX 4.3.3 and AIX 5L Version 5.0 today, using the AIX Toolbox. AIX 5L Version 5.1 plans to add greater compatibility by adding additional API support for functions that are not presently similar between Linux and AIX (Hebert).

API versus ABI issues

The two major methods of providing affinity for applications running on two different opearating systems are the API (Application Programming Interface) and the ABI (Application Binary Interface) methods. In the API approach, programs are ported to a target operating system (AIX in this case) from a source operating system (Linux). These applications should port virtually unchanged -- this is because the headers and library calls that these applications rely on have been previously ported to the target operating system. These ported library routines will result in different operating system service calls on the target system than they did on the source system. However, the underlying system service calls are completely hidden from the application developer. To the application, the library invocation remains exactly the same. In the ABI approach, binaries are copied directly from the source system to the target system. These binaries can then be executed on the target system in an "emulation mode". There are some advantages and disadvatages of using the API versus the ABI affinity approaches.

Recompilation

Clearly, the main disadvantage of the API approach is that each application must be recompiled when brought over from Linux to AIX. However, since the port has to be done only once, and the set of APIs provided in AIX is rather complete, this may not be all that great a disadvantage. A clear advantage of taking the API approach is that once an application is recompiled for AIX, it becomes an AIX application and inherits all the stability that AIX provides.

Performance

The issue of performance using the ABI approach is a touchy one. One would think that the API approach would outperform the ABI approach. This is because a program ported using the API approach is recompiled to use the native operating system calls, instead of running in emulation mode. However, open source projects that provide commands like lxrun claim that the performance hit is negligible, in fact, it may not even be measurable. They prefer not to call lxrun an emulator at all, stating that it's really just remapping a few system calls, and the rest of the time it remains undetectable. Systems calls, after all, are used for things like reading and writing files, displaying information to the screen, and accessing the network. System calls are not used in computation. Since the overhead that most I/O requires is far greater than the amount of time it takes to remap the system calls, the performance degradation in using the ABI approach instead of the API approach may be slight or none at all. Admittedly, there will be some noticeable slow down when running graphics-intensive applications, such as X Windows.

Application Availability

Using the API approach, IBM has ported most of the major Linux tools to AIX, including some GNU Tools, such as Gcc Compiler, Gdb Debugger, Emacs, Automake, Autoconf, Libtool, and hundreds of other tools, including:

• Application Development -- gcc, g++, gdb, rpm, cvs, automake, autoconf, libtool, bison, flex, gettext
• Desktop Environments -- Gnome and KDE
• GNU Base Utilities -- gawk, m4, indent, sed, tar, diffutils, fileutils, findutils, textutils, grep, sh-utils
• Programming Languages -- Guile, Python, Tcl/Tk, Rep-gtk
• System Utilities -- emacs, vim, bzip2, gzip, git, elm, ncftp, rsync, wget, lsof, less, samba, zip, unzip, zoo
• Graphics Applications -- ImageMagick, transfig, xfig, xpdf, ghostscript, gv, mpage
• Libraries -- ncurses, readline, libtiff, libpng, libjpeg, slang, fnlib, db, gtk+, qt
• System Shells -- bash2, tcsh, zsh
• Window Managers -- Enlightenment, Sawfish

One of the reasons why the ABI method works is that it requires the underlying architecture of the source and target machines to be the same. So, when an application binary is copied from Linux to Solaris, and executed with lxrun, it requires the same IA-32 architecture to be present on both source and target machines. lxrun merely executes all the instructions in native mode until it gets to a system call and remaps the system calls for the target operating system. This approach does not apply when porting across architectures, however. There is no easy way to port an IA-32 Linux application to say, a Sparc Station running Solaris unless something fairly exotic like a machine-level emulator is employed. Using the API approach, porting across architectures is fairly straightforward, providing all the standard interfaces are maintained. This means that a Linux application developed for an IA-32 platform and compiled using the GNU tools should port to a RS/6000, PowerPC, or IA-64 machine running AIX.

Summary

By porting many standard Linux library routines and headers to AIX, IBM has attempted to make AIX an operating system friendly to Linux developers. They have made it friendly to Linux end users by porting many favorite desktop environments and applications to AIX. While IBM's API approach to AIX/Linux affinity does require more work on the part of the porter than an ABI approach, a recompile is a relatively small price to pay to gain the stability and reliability inherent in AIX. With AIX 4.3.3, and AIX 5L 5.0, IBM has demonstrated its commitment to Linux applications and users on AIX; with AIX 5L 5.1, IBM will further demonstrate that long-term commitment.

Acknowledgments and Bibliography

Much of the background information used in this article was taken from corporate and news Web sites. The sites used as background material are:

http://www.hp.com
http://www.ibm.com/aix
http://www.devresource.hp.com/LPK/libhplx.html
http://www.linux.hp.com
http://www.linux.hp.com/news_events/ximian.html
http://www.sun.com/linux
http://www.zdnet.com
http://www.ugcs.caltech.edu/~steven/lxrun/lxrun-FAQ.html
http://www.ibm.com/servers/aix/overview/linux.html
http://www.ibm.com/servers/aix/news/aix_affinity.html

http://www.ibm.com/servers/aix/products/aixos/linux/affinity_linux.pdf