ch-fromhost man page

ch-fromhost — Inject files from the host into an image directory

Synopsis

$ ch-fromhost [OPTION ...] [FILE_OPTION ...] IMGDIR

Description

NOTE:

This command is experimental. Features may be incomplete and/or buggy. Please report any issues you find, so we can fix them!

Inject files from the host into the Charliecloud image directory IMGDIR.

The purpose of this command is to provide host-specific files, such as GPU libraries, to a container. It should be run after ch-tar2dir and before ch-run. After invocation, the image is no longer portable to other hosts.

Injection is not atomic; if an error occurs partway through injection, the image is left in an undefined state. Injection is currently implemented using a simple file copy, but that may change in the future.

By default, file paths that contain the strings /bin or /sbin are assumed to be executables and placed in /usr/bin within the container. File paths that contain the strings /lib or .so are assumed to be shared libraries and are placed in the first-priority directory reported by ldconfig (see --lib-path below). Other files are placed in the directory specified by --dest.

If any shared libraries are injected, run ldconfig inside the container (using ch-run -w) after injection.

Options

To specify which files to inject

-c, --cmd CMD

Inject files listed in the standard output of command CMD.

-f, --file FILE

Inject files listed in the file FILE.

-p, --path PATH

Inject the file at PATH.

--cray-mpi

Cray-enable an MPICH installed inside the image. See important details below.

--nvidia

Use nvidia-container-cli list (from libnvidia-container) to find executables and libraries to inject.

These can be repeated, and at least one must be specified.

To specify the destination within the image

-d, --dest DST

Place files specified later in directory IMGDIR/DST, overriding the inferred destination, if any. If a file’s destination cannot be inferred and --dest has not been specified, exit with an error. This can be repeated to place files in varying destinations.

Additional arguments

--lib-path

Print the guest destination path for shared libraries inferred as described above.

--no-ldconfig

Don’t run ldconfig even if we appear to have injected shared libraries.

-h, --help

Print help and exit.

-v, --verbose

List the injected files.

--version

Print version and exit.

--cray-mpi Dependencies and Quirks

The implementation of --cray-mpi for MPICH is messy, foul smelling, and brittle. It replaces or overrides the open source MPICH libraries installed in the container. Users should be aware of the following.

  1. Containers must have the following software installed:

    a.

    Open source MPICH.

    b.

    PatchELF with our patches. Use the shrink-soname branch.

    c.

    libgfortran.so.3, because Cray’s libmpi.so.12 links to it.

  2. Applications must be linked to libmpi.so.12 (not e.g. libmpich.so.12). How to configure MPICH to accomplish this is not yet clear to us; test/Dockerfile.mpich does it, while the Debian packages do not.
  3. One of the cray-mpich-abi modules must be loaded when ch-fromhost is invoked.
  4. Tested only for C programs compiled with GCC, and it probably won’t work otherwise. If you’d like to use another compiler or another programming language, please get in touch so we can implement the necessary support.

Please file a bug if we missed anything above or if you know how to make the code better.

Notes

Symbolic links are dereferenced, i.e., the files pointed to are injected, not the links themselves.

As a corollary, do not include symlinks to shared libraries. These will be re-created by ldconfig.

There are two alternate approaches for nVidia GPU libraries:

  1. Link libnvidia-containers into ch-run and call the library functions directly. However, this would mean that Charliecloud would either (a) need to be compiled differently on machines with and without nVidia GPUs or (b) have libnvidia-containers available even on machines without nVidia GPUs. Neither of these is consistent with Charliecloud’s philosophies of simplicity and minimal dependencies.
  2. Use nvidia-container-cli configure to do the injecting. This would require that containers have a half-started state, where the namespaces are active and everything is mounted but pivot_root(2) has not been performed. This is not feasible because Charliecloud has no notion of a half-started container.

Further, while these alternate approaches would simplify or eliminate this script for nVidia GPUs, they would not solve the problem for other situations.

Bugs

File paths may not contain colons or newlines.

Examples

Place shared library /usr/lib64/libfoo.so at path /usr/lib/libfoo.so (assuming /usr/lib is the first directory searched by the dynamic loader in the image), within the image /var/tmp/baz and executable /bin/bar at path /usr/bin/bar. Then, create appropriate symlinks to libfoo and update the ld.so cache.

$ cat qux.txt
/bin/bar
/usr/lib64/libfoo.so
$ ch-fromhost --file qux.txt /var/tmp/baz

Same as above:

$ ch-fromhost --cmd 'cat qux.txt' /var/tmp/baz

Same as above:

$ ch-fromhost --path /bin/bar --path /usr/lib64/libfoo.so /var/tmp/baz

Same as above, but place the files into /corge instead (and the shared library will not be found by ldconfig):

$ ch-fromhost --dest /corge --file qux.txt /var/tmp/baz

Same as above, and also place file /etc/quux at /etc/quux within the container:

$ ch-fromhost --file qux.txt --dest /etc --path /etc/quux /var/tmp/baz

Inject the executables and libraries recommended by nVidia into the image, and then run ldconfig:

$ ch-fromhost --nvidia /var/tmp/baz

Acknowledgements

This command was inspired by the similar Shifter feature that allows Shifter containers to use the Cray Aires network. We particularly appreciate the help provided by Shane Canon and Doug Jacobsen during our implementation of --cray-mpi.

We appreciate the advice of Ryan Olson at nVidia on implementing --nvidia.

Reporting Bugs

If Charliecloud was obtained from your Linux distribution, use your distribution’s bug reporting procedures.

Otherwise, report bugs to: <https://github.com/hpc/charliecloud/issues>

See Also

charliecloud(1)

Full documentation at: <https://hpc.github.io/charliecloud>

Referenced By

charliecloud(1).

2019-09-04 00:00 Coordinated Universal Time Charliecloud