Mixed Environment Linux with efficient execution (MELEE)

MELEE is a small kit of software which helps setting-up a PC or server to run multiple Linux distributions effectively in parallel, with access from one Linux system to another. It uses application-level virtualisation rather than hardware virtualisation.

MELEE is efficient compared with running several Linux virtual machines in that (i) there is only one kernel and one process scheduler, and no hypervisor, (ii) there is no processing overhead or emulation or need for hardware assist, (iii) files on one system can be shared within another system via simple bind-mounting, rather than having to access via NFS or SAMBA even where they are local, (iv) it is possible to invoke programs directly within one distribution to run within another distribution.

Don't use MELEE when you want to divide your machine(s) into isolated system instances which are unaware of each other's presence and cannot communicate, except via a network. That's what VM methods are good at, and is the opposite of what we want to achieve with MELEE.

MELEE doesn't help when you want to run a virtual Linux within Windows, or vice versa. It's a Linux-only solution.

Overview

MELEE makes use of the long-standing Unix chroot facility, combined with the ability to bind-mount directories in several different places in the base file tree. Binaries running within the chroot make use of the run-time libraries of the image, rather than those of the base system. The same one kernel provides the system hooks for those run-time libraries, and kernels are backwards-compatible, so a binary performs the same as if it was running natively on its original system.

Example

We have a cluster and a set of desktop PCs where the base system is Fedora 18 (at the time of writing), but where there are also available complete SL4, SL5, and SL6 images (where SL is Scientific Linux, similar to CentOS and based on Red Hat Enterprise Linux), as well as an earlier Fedora distribution and Ubuntu 12. There's the opportunity to extend that to have a bigger set of different sub-systems, including 32-bit and 64-bit versions of the same distro, if we wanted them.

The distributions are located on the PC in subdirectories of /image, like /image/sl58, /image/sl63, /image/u12, and so on.

To start a shell session in a distro image (say SL6) other than the base image, the user types sl6, or clicks on a panel icon which creates a window running the sl6 command. The sl6 command invokes the same shell as is default for that user, but within that distro image.

To invoke a particular command in a different distro environment, such as a firefox script for SL6 running within a Fedora 18 base system, the user can type sl6 firefox, and the corresponding version of firefox will be started within an SL6 environment.

Also, a script which is headed by the string #!/bin/sl6, for example, will be invoked in the context of the different distro.

Important files

The following files are useful in a MELEE context:

imageswitch.c: this source file, when compiled with the supplied make-sysshell script, gives a binary which performs the chroot to a different distribution image. It switches to the compiled-in image directory, reverts privilege to that of the issuing user, and changes directory to the same directory within the image as that when invoked. Chrooting normally needs setuid privilege (which is dropped by this binary as soon as possible).

imagemounts: this script, normally run once at boot time, bind-mounts all the necessary file trees and system mounts (like /proc) into the distribution images in /image.

Torque batch compatibility

The MELEE environment is compatible with and complementary to the Torque batch facility, in that it provides the initialised environments that can then be utilised by Torque via its qsub -D option. For example, qsub -D /image/sl63 myscript will run the script myscript within a chrooted environment on a worker. Torque provides this chrooting capability natively, and so there is no need for a MELEE binary based on imageswitch to perform the chrooting. Normally, the ordinary user will be protected from having to specify this option by virtue of wrapper scripts, for example: qsub5, qsub6.

Index