| October 20, 2008 12:00 AM PDT | |
by Thomas Wolfgang Burger, twburger@weststartech.com
September, 23, 2003
In order to prevent incompatibility between Linux* distributions the Linux Standard Base* (LSB) was formed. This group is working to create a binary compatibility specification that allows an LSB compliant application to run on any LSB compliant distribution of Linux. This will simplify development efforts and remove confusion in the marketplace by creating one universal Linux standard.
This paper's intended audience is Linux programmers and describes, in general terms, the steps required to write an LSB compliant application.
The Linux Standard Base* (LSB) group task is to provide the infrastructure with which to create and certify the binary compatibility specification that allows an LSB compliant application to run on any LSB compliant distribution of Linux*.
The Linux Standard Base (currently LSB 1.2 for IA32) specification is a binary compatibility effort. LSB 1.1 defines an Application Binary Interface (ABI), which is similar to the POSIX.1 and POSIX.2 source specifications. It specifies the binary environment in which an LSB compliant application executes. This is differs from application programming interfaces (APIs). APIs apply only to building applications, LSB attempts to provide Linux distribution cross-platform compatibility where each LSB compliant application's binaries will execute, without changes, on any LSB-conformant platform of the same processor type. The goal of LSB is to assure cross-distribution and backward compatibility of Linux applications without impeding innovation in Linux by providing a base model. This is done by defining a minimal set of libraries as part of the base distribution. LSB gives approved libraries unique names identifying them as LSB-compliant libraries, the LSB version of libcrypt may be libcrypt.lsb.1. The naming convention establishes a known set of library definitions upon which application developers rely.
LSB not only defines standards. They coordinate efforts to recruit software vendors to port and write products for Linux. The current LSB members list includes: Caldera Inc., Corel Corp., The Debian Project, Enhanced Software Technologies Inc., IBM, LinuxCare, Linux for PowerPC, MandrakeSoft, Metro Link Inc., The Open Group, Turbolinux Inc., Red Hat Inc., SGI, Software in the Public Interest Inc., SuSE. GmbH, VA Releases Linux and WGS Inc.
Linux uses shared libraries to provide applications with a set of standard functions and utilities while saving storage space. However, shared libraries can cause many application compatibility issues. The lack of strict version control or inappropriate version use is common problems. Linux includes essential and common libraries such as glibc, pthreads, libm, Xt, ncurses and others. Any application that is compiled to use a given version of a shared library expects to find that precise version at run time. The application may run correctly with later versions of a library, but backward compatibility is not guaranteed and there is no guarantee it will run properly with a later library. Currently, the method to defeat the problem is to include multiple versions of libraries wit hin a Linux distribution. This usually works, but the increased disk space and memory used defeats the value of having shared libraries.
With LSB, compliance customers are assured that any LSB compliant Linux distribution purchase will be compatible with all LSB compliant applications. LSB assures a customer that the application is thoroughly tested for LSB compliance. Developers and independent software vendors (ISVs) will be able to insure their applications run on a majority of Linux platforms. Developers and ISV will reach all possible users without having to form an alliance with every possible Linux distributor and then code for each individually, thus maximizing the availability of open source software on Linux. The LSB provides a clear framework for Linux distributors around which they can add individual enhancements to target a particular audience without causing applications to stop functioning.
The Linux Standard Base includes a written specification, a compliance test suite, and a sample implementation of an Open Source LSB-compliant base distribution of Linux.
The LSB standards include the monitoring and standardization of the following components:
- The most widely used shared libraries
- System commands
- Filesystem hierarchy
- System initialization process and scripts
- POSIX.1 compliance
- glibc library extensions
- Sockets network interface
- X11 (minimal implementation) graphical interface
- Basic software installation
- The object file format - ELF
Figure 1 - LSB Compliant Linux Configuration
The LSB naming convention prevents installation programs from overwriting LSB-compliant libraries with new versions. If a Linux provider included a version of libcrypt that provides new features added since libcrypt.lsb.1, it would be placed in the library directory using a non-LSB naming convention such as libcrypt.so.1.2. This file may replace libcrypt.so.1.1, but it would not replace libcrypt.lsb.1 until it was sanctioned as LSB compliant and renamed as an LSB library. This makes the new features available to applications that need them without interfering with applications that need the LSB libraries. LSB allows new libraries and LSB-compliant libraries to coexist to ensure compatibility without preventing individual innovation and allows a backward compatibility that is lacking in legacy proprietary operating systems under the control of a single vendor. No matter how updates to an LSB compliant Linux offering occur, they will not break LSB compliant applications since the LSB libraries will remain untouched.
LSB considers certain commands such as cat, sleep, more, man, sed, nice, tail as mandatory for a base distribution. Linux and Linux applicat ions rely heavily on shell scripts to install, remove, initialize and shut down software. LSB defines commands that must be present so scripts can run without and users fearing that the basic tools they require will be missing. In order for these components to be located where they can be found in any Linux distribution LSB adopted the Filesystem Hierarchy Standard (FHS) to accomplish this goal. FHS defines how files, libraries and commands are arranged in an LSB compliant system.
LSB defines a standard procedure for the placement and contents of initialization files and scripts allowing application vendors to install the proper initialization scripts with the assurance that other initial services upon which it may depend are already started. The standard also prevents circular dependencies, causing never-ending system initialization loops.
LSB compliant systems support the POSIX.1-1990 standard and the glibc extensions. This provides known set of functions and a reference upon developers can rely. The reader will note that in the "Developing Platforms..." section below POSIX heads the list of standards related to the LSB. The importance of POSIX is two-fold: First, it provides developers with a well-established set of standards and documented functions. The code is already known, proven and accepted. Programmers do not have to learn new material and the problems associated with developing new code libraries are avoided. Secondly, POSIX compliance brings Linux closer to UNIX compatibility, making Linux more acceptable to the UNIX development community as a platform and the UNIX user community as a viable alternative or accompaniment to UNIX because POSIX compliant code built for UNIX platforms will run on Linux unaltered.
LSB also specifies the following Linux specifications:
- The sockets interface for network and Internet access.
- The lowest levels (those elements common to GNOME, KDE and other desktops) of the X11 graphical user interface.
- The de-facto standard executable object file format - Executable and Linking Format (ELF) that is the used with Linux.
It is equally important to note what elements of Linux the LSB does not address. LSB focuses only on Linux elements that affect application compatibility - installation programs and the applications themselves. LSB defines a boundary starting just above Linux kernel space and ending at the application level where components are required to remain in a state necessary to ensure compatibility for applications. This provides Linux distributors a clearly defined boundary outside of which they may freely individualize their products. An exception to this is the special adaptations of Linux, such as embedded Linux where LSB compliance is impractical due to limited markets and specialized needs.
In order to develop applications that comply with the LSB Specification, you must be familiar with interfaces and utilities that are normally present on UNIX, Linu x or similar systems. The Related Standards section of the LSB Specification lists several sources for this type of information (with abbreviations shown in parentheses):
- IEEE Std POSIX 1003.1-1996 (POSIX.1)
- IEEE Std POSIX 1003.2-1992 (POSIX.2)
- The Open Group CAE Specification, System Interface Definitions, Issue 5, 1997 (SUS-XBD)
- The Open Group CAE Specification, Commands and Utilities, Issue 5, 1997, (SUS-XCU)
- The Open Group CAE Specification, Networking Services, Issue 5, 1997, (SUS-XNS)
- The Open Group CAE Specification, System Interfaces and Headers, 1997, Issue 5 (SUS-XSH)
- AT&T, The System V Interface Definition, Issue 3, 1989 (SVID.3)
The LSB developer must not only be familiar the above standards, but also with the C or C++ programming languages since the LSB Specification and related documentation uses the C language forms of the interfaces.
As you develop your application, be sure to refer to the LSB specification, and its associated documentation, and use only the interfaces, definitions and utilities specified to be part of an LSB conformant platform.
When testing begins for your application, use the lsbappchk utility as a standard part of each testing cycle. This will ensure that no interfaces are being used which are not supported in the LSB Specification.
Porting an application to LSB is similar to porting to another platform, the difference is the lsbappchk utility can be used as soon as the application compiles and links to check for interfaces or definitions that are not part of the LSB Specification.
Non-LSB Library Use
Static
Non-LSB libraries can be dynamically and statically linked to LSB compliant executables, so there are no hindrances to the scope of an application. After statically lining a library to the application the lsbappchk utility will verify that the only missing symbols are the LSB definitions. Dynamic libraries must be supplied with the application. The LSB Specification does not provide a means to detect missing package dependencies. This functionality must be provided in installation scripts.
To statically link a library to utilize a non-LSB function:
|
Shared
Shared libraries are libraries that are loaded by programs at startup. The rules to finding and utilization them are the sale as dynamically loaded libraries, but they are linked to the application at compilation. The difference to a statically linked library is that the shared library i s defined when it is created using this format:
|
In this example two object files (x.o and y.o) are created and then a shared library is built that contains both of them. The compilation generates object files (using -c), and includes the required -fPIC option to enable required "position independent code" generation:
|
Dynamic
Dynamically loaded (DL) libraries are shared libraries that are loaded at times other than during the startup of a program. They are useful for implementing a plug-in or modules by permitting the application to wait to load the module until when, if ever, it is needed and allowing program elements to be unloaded and changed to configure and then reconfigure an application as required. The dlopen() function opens a library and prepares it for use. The C prototype is:
void * dlopen(const char *filename, int flag) |
If filename is an absolute path dlopen() will just try to find and open the library at that location. If not dlopen() will search for the library in the following order:
- A colon-separated list of directories in the user's LD_LIBRARY path environment variable.
- The list of libraries in /etc/ld.so.cache (generated from /etc/ld.so.conf).
- /lib, followed by /usr/lib.
To use a dynamically linkable version of the, include it with the application and install it inside the application's directory tree (installation should conform to the Filesystem Hierarchy Standard 1 (FHS)). To instruct the dynamic loader where to search this library, you can compile the application with the -rpath linker switch. The advantage of this method is that other binaries are not affected (like suid/sgid). However this does make the application non-relocatable. An alternative is the environment variable LD_LIBRARY_PATH. This colon-separated set of directories defines where libraries a re searched for before the standard set of directories. This is useful for using a nonstandard library for special purposes, but this direct control can cause problems.
The minimal directory structure and files implementation required by the LSB specification for an LSB compliant application to install, configure and execute includes:
- ls-lsb.so.1 (may be a symbolic link to ld-linux.so.2)
- A tool which can install LSB compliant packages
- The 'lsb' dependency is known to the tool which installs LSB packages
- LSB compliant versions of shared libraries (libb, libm, libdl, libcrypt, libpthreads, librt, libz, libutil, libcurses, libX11, libXext, libXt, libSM, libICE, libGL)
- Commands and utilities (lsb_release, useradd, sh)
- cron infrastructure
- kernel (this one is not a clear requirement, though the libraries especially libc will not behave in the correct manner without a kernel which provide POSIX compliant behavior).
The LSB build environment will, at a minimum, provide:
- LSB header files
- LSB stub libraries
- C compiler/linked
- Basic build tools (make, lex, yacc)
- Mechanism to create LSB compliant packages (package format)
- Tools like editors and system build tools provided by the system that build environment is installed on.
The "LSB Conformance" of a Linux distribution or application means that the supplier states, without endorsement from the LSB, that their deliverable follows the LSB standard.
Specifically for Linux distributions:
- Follows the LSB written specification
- Passes the binary LSB TET test suite provide by the LSB
- Passes the binary lsblibchk
- Passes the FHS checklist
- Documents in their release notes all LSB exceptions with a plan of resolution and due date
- The overloading of LSB functions is not recommended.
- The GNU C compiler, gcc and any other compiler whose output conforms to the LSB Specification can be used.
- The use of interfaces not specified in the LSB Specification will render an application no longer LSB compliant.
- Libraries in /lib/lsb can be linked to libraries in /lib if the libraries in /lib provide all of the functionality detailed in the LSB Specification and do not change their behavior from that detailed in the LSB Specification.
LSB testing procedures and programs are separated into three main categories:
- Runtime Environment
- Application
- Build Environment
There are two runtime environment-testing programs. lsb-runtime-test is made up of four separate test suites, combined into a sin gle binary rpm:
- POSIX.1 test suite tests for compliance against the POSIX.1-1990 standard.
- LSB-OS test suite test for compliance of Linux specific interfaces as defined in the LSB specification and other interfaces from POSIX and the Single UNIX Specification required by the LSB specification but not covered by the POSIX.1 test suite
- FHS 2.2 test suite tests for compliance against the FHS 2.2 specification.
- LSB Users/Groups test suites to test for compliance against the user and group components of the LSB specification (commands and compulsory users and groups).
For certification testing, the binary rpm must be used.
The second program is lsblibchk. This program looks for LSB required libraries and checks that those libraries contain the symbols required by the specification. If it is required that the symbol be versioned it also checks that the symbol with the correct version exists in the library.
The test program for application testing is lsbappchk. It checks that the supplied binary only uses dynamically linked symbols that are either part of the LSB specification or are defined in a shared library supplied with the application. There is also an LSB Filesystem Hierarchy Checklist. This is a checklist of FHS related application behavior and packaging.
Build environment testing is performed by lsblibchk and devchk. The lsblibchk test program is described above. The devchk program verifies binary values such as struct sizes and macro definitions in header files that are used by LSB required interfaces.
lsb-runtime-test
In addition to the requirements specified by the LSB specification, the following applications must also be installed for the tests to run correctly:
- pax
- perl 5.x
1.Install the binary rpm package
# rpm -i lsb-runtime-test-1.0-1.i386.rpm |
2.Log out and log in as the vsx0 user. Set the password for this account, it is not set during the installation of the package.
3.Run the tests. It is normal for the test suites to several hours to run.
vsx0@lnxdog~$ run_tests |
After the tests have finished running a journal file will be available in ~vsx0/results/0001e/journal.
4. Produce the report file
vsx0@lnxdog~$ vrpt ~/results/0001e/journal >
lsb-runtime-test.report
|
5. Display the test cases which failed during the test run and do not appear in the waiver list. The latest version of the waiver list file can be downloaded from the site l isted below. For a runtime environment to be conformant there should be no tests that fail and not listed as waived.
vsx0@lnxdog~$ tjreport -w waivers-1.1.txt
~/results/0001e/journal
|
The tjreport tool is available to download from the freestandards ftp site.
The tests have been built in the UNIX98 test mode for the POSIX.1 portion of the test suite. The test suites is the expect POSIX.1 values such as PATH_MAX is required to include the null terminator . As stated previously, many of the tests run a long time - the signal related tests may take several hours to execute.
lsblibchk
1. Install the binary rpm package.
#rpm -i lsblibchk-1.1.2-1.i386.rpm |
2. Execute the test program as any user.
# /opt/lsblibchk/bin/lsblibchk |
Any problems with the libraries will cause error messages to be displayed. If the runtime environment contains more than one version of libraries required by the specification you should also verify that the correct library is located by the test program. Directories are searched in the following order:
1. /lib/lsb
2. /usr/lib/lsb
3. /lib
4. /usr/lib
5. /usr/X11R6/lib
Detailed information on the lsblibchk tests is saved in a file named journal.lsblibchk. The tjreport tool bundled with lsb-runtime-test can be used to produce a summary of the results.
lsbappchk
1. Install the binary rpm package.
|
2. Execute the test program as any user on all of the dynamically linked binaries bundled with your application. The program is able to process more than one executable at a time and the symbols gathered from the extra libraries will be used for all of the programs specified.
|
Error messages will be displayed if any problems are discovered. If your binary is dynamically linked against shared libraries that are bundled with your application you will need to supply an extra parameter which specifies where these shared libraries reside on the file system:
|
A file named journal.appchk.appname (where appname is the name of the first program on the command li ne) is created containing information on the tests performed, and the results of those tests. The tjreport tool can be used on this file as well.
devchk
The devchk tests are a source based test suite designed to test header files. This is done by compiling against the current header files and so a compiler will have to be installed on the system. In addition to the devchk tarball if you do not already have TET installed on your system (the version in lsb-runtime-test binary rpm is not sufficient) you will also have to download the source Test Environment Toolkit (TET), available at the link listed below (The Test Environment Toolkit (TET), is a multi-platform uniform test scaffold, into which non-distributed and distributed test suites can be incorporated). TET supports tests written in C, C++, Perl, Tcl, Shell (sh , bash), and Korn Shell). The testing can be done as any user on the system.
1. Install the source tarballs
|
2. Compile TET
|
3. Build the test suite
|
4. Visually inspect ~/tet/devchk-1.1.0./results/0001b/journal and confirm that there are no compilation errors.
5. Execute the test suite
|
6. Create a summary of the results using the tjreport tool
|
The journal file (~/tet/results/0002e/journal) contains more details for any failed test cases
Further Requirements
LSB Runtime environment certification also requires testing of two sample applications from the LSB Application Battery available from http://www.linux-foundation.org/en/LSB*.
- Caldera OpenLinux Server Release 3.1.1 with lsb-1.1-2 package, 18-Sep-2002
- Mandrake Linux ProSuite 8.2 + first update CD, 02-Aug-2002
- Red Hat Linux 7.3 with glibc 2.2.5-39+kernel 2.4.18-10 or later, 10-Aug-2002
- SuSE Linux 8.0 Professional + aaa_base and Kernel Update, 09-Aug-2002
Thomas Wolfgang Burger is the owner of Thomas Wolfgang Burger Consulting. He has been a consultant, instructor, writer, analyst, and applications developer since 1978. He can be reached at twburger@weststartech.com.
- Linux Standard Base*
- LSB Certified Distributions of Linux*
- LSB test suites FTP site from the Free Standards Group*
- LSB Application Battery*
- LSB Certification Test Suites*
- LSB Tests*
- LSB Waiver List File*
- LSB 1.0: Filesystem Hierarchy Checklist*
- Filesystem Hierarchy Standard 1 (FHS), Version 2.1*
- Filesystem Hierarchy Standard*
- Free Standards Group: Linux Foundation Free Standards Group* and SourceForge Free Standards Group
- Freestandards FTP site*
- POSIX*
- ELF*
- Why LD_LIBRARY_PATH is Bad*
For more complete information about compiler optimizations, see our Optimization Notice.
Comments (0) 
Trackbacks (0)
Leave a comment 
TW Burger
|
