charliecloud - Man Page

Charliecloud is a lightweight, fully unprivileged, open source container implementation for HPC applications. It can handle the entire container workflow, including building images, pushing/pulling to registries, dealing with accelerators like GPUs.

Charliecloud uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. This simple approach avoids most security risks while maintaining access to the performance and functionality supercomputers already offer.

Charliecloud takes a unique approach to containerization while remaining compatible with the broader container ecosystem where it matters. Notable distinctions include:

• A fully unprivileged end-to-end workflow with no root, no sudo, and no setuid/setcap helper programs, and it’s been that way since the beginning. Fully unprivileged run and build were both industry firsts.
• No security boundaries within Charliecloud; we instead rely on existing, well-tested UNIX principles enforced by the Linux kernel and userland.
• Clear focus on containerizing HPC applications, without technical or usability overhead of other use cases.
• Root emulation to enable unprivileged container build based on a simple, novel, zero-consistency seccomp system call filter.
• A layer-free build cache based on Git rather than union-mount (overlay) filesystems and tarballs.
• Dockerfiles are parsed with a formal grammar.
• SquashFS image mounts are managed internally, linking directly with libsquashfuse.
• Small code base with minimal dependencies.
• Fully independent from other container implementations, built using a thoughtful approach that balances UNIX traditions with technical novelty.
• A spirit of openness and public service. For example, all of Charliecloud’s key innovations have been described in open-access papers.

We develop Charliecloud under what we call “the philosophy of Charliecloud”, which is a set of three principles to guide our work, namely:

1. Be transparent, not opaque. For example, in Charliecloud, containers are regular files and directories rather than a novel storage format. We aim to be compatible with standard UNIX tools and conventions. We assume the user is intelligent but unfamiliar with Charliecloud internals, and we are honest and straightforward with them about what is going on. We try to be explicit, rather than implicit or magical (too much “magic” leads to over-helping and confusion).
2. Be simple, not complex. In Charliecloud, containers are just user processes. We use the minimum namespaces and implement only the right features; Charliecloud is far smaller in lines of code than competitors. We use technology familiar to HPC people and minimize dependencies. We embrace UNIX: “make each program do one thing well”. (UNIX traditions and concepts have staying power for a reason.) In particular, we don’t re-invent, re-build, and/or re-optimize what HPC has already done.
3. Be unprivileged. Writing secure code is difficult and risky, so we don’t do it. Charliecloud does not maintain a security boundary. Instead, we trust the kernel and let it enforce security, just like HPC has done for years. Thus, Charliecloud stays fully unprivileged, with no root, no sudo, and no setuid or setcap helper programs.

• Documentation: https://hpc.github.io/charliecloud
• GitLab repository: https://gitlab.com/charliecloud/charliecloud
• Mailing lists: https://lists.charliecloud.io
• We wrote an article for USENIX’s magazine ;login: that explains in more detail the motivation for Charliecloud and the technology upon which it is based: https://www.usenix.org/publications/login/fall2017/priedhorsky
• For technical papers about Charliecloud, refer to the Technical publications section below.

Contributors:

• Richard Berger <rberger@lanl.gov>
• Lucas Caudill <lcaudill@lanl.gov>
• Rusty Davis <rustyd@lanl.gov>
• Hunter Easterday <heasterday@lanl.gov>
• Oliver Freyermuth <o.freyermuth@googlemail.com>
• Shane Goff <rgoff@lanl.gov>
• Michael Jennings <mej@lanl.gov>
• Christoph Junghans <junghans@lanl.gov>
• Dave Love <dave.love@manchester.ac.uk>
• Jordan Ogas <jogas@lanl.gov>
• Kevin Pelzel <kpelzel@lanl.gov>
• Megan Phinney <mphinney@lanl.gov>
• Reid Priedhorsky <reidpr@lanl.gov>, co-founder and project lead
• Tim Randles <trandles@lanl.gov>, co-founder
• Benjamin “The Storm” Stormer <bstormer@lanl.gov>
• Meisam Tabriz <meisam.tabriz@mpp.mpg.de>
• Matthew Vernon <mv3@sanger.ac.uk>
• Peter Wienemann <wienemann@physik.uni-bonn.de>
• Lowell Wofford <lowell@lanl.gov>

Use our GitLab page: https://gitlab.com/charliecloud

Bug reports and feature requests should be filed as “Issues”. Questions, comments, support requests, and everything else should use our “Discussions”. Don’t worry if you put something in the wrong place; we’ll be more than happy to help regardless.

We also have mailing lists for users and developers: https://lists.charliecloud.io

Patches are much appreciated on the software itself as well as documentation. Optionally, please include in your first patch a credit for yourself in the list above.

We are friendly and welcoming of diversity on all dimensions.

If Charliecloud helped your research, or it was useful to you in any other context where bibliographic citations are appropriate, please cite the following open-access paper:

Note: This paper contains out-of-date number for the size of Charliecloud’s code. Please instead use the current number in the FAQ.

Other publications:

• We compared the performance of three HPC-specific container technologies against bare metal and found no concerns about performance degradation.

  Alfred Torrez, Tim Randles, and Reid Priedhorsky. “HPC container runtimes have minimal or no performance impact”, 2019. In Proc. CANOPIE HPC Workshop @ SC. DOI: 10.1109/CANOPIE-HPC49598.2019.00010.

• We demonstrated how low-privilege containers solve increasing demand for software flexibility in HPC centers.

  Reid Priedhorsky, R. Shane Canon, Timothy Randles, and Andrew J. Younge. “Minimizing privilege for building HPC containers”, 2021. In Proc. Supercomputing. DOI: 10.1145/3458817.3476187.

• Charliecloud’s build cache performs competitively with the standard many-layered union filesystem approach and has structural advantages including a better diff format, lower cache overhead, and better file de-duplication.

  Reid Priedhorsky, Jordan Ogas, Claude H. (Rusty) Davis IV, Z. Noah Hounshel, Ashlyn Lee, Benjamin Stormer, and R. Shane Goff. “Charliecloud’s layer-free, Git-based container build cache”, 2023. In Proc. CANOPIE HPC Workshop @ SC. DOI: 10.1145/3624062.3624585.

• We introduced a new root emulation approach that turns privileged operations requested by package managers into no-ops. Surprisingly, package managers don’t notice and the build works fine.

  Reid Priedhorsky, Michael Jennings, Megan Phinney. “Zero-consistency root emulation for unprivileged container image build”, 2024. In Proc. CANOPIE HPC Workshop @ SC. DOI: 10.1109/SCW63240.2024.00023.

Charliecloud is open source software; you can redistribute it and/or modify it under the terms of the Apache License, Version 2.0. A copy is included in file LICENSE. You may not use this software except in compliance with the license.

Copyrights and patents are retained by contributors. No copyright assignment is required to contribute to Charliecloud. Please use the Git history to determine copyright for any specific contribution.

Notable copyright holders include:

• Triad National Security, LLC (contributions by LANL employees)

ch-checkns(1), ch-completion.bash(7), ch-convert(1), ch-fromhost(1), ch-image(1), ch-run(1), ch-run-oci(1), ch-test(1),

Full documentation at: https://hpc.github.io/charliecloud

These man pages are for Charliecloud version 0.42 (Git commit 0.42).

contributors to the Charliecloud project, which has been established as “Charliecloud a Series of LF Projects, LLC”. For web site terms of use, trademark policy, and related information, please see: https://lfprojects.org.

ch-checkns(1), ch-convert(1), ch-fromhost(1), ch-image(1), ch-run(1), ch-run-oci(1), ch-test(1).