Introduction to Apptainer on Torch
Why Containers?
Researchers often rely on complex software stacks that include programming languages and libraries. Installing and maintaining these dependencies can be challenging when different projects require different software versions.
Containers provide a way to package software together with the environment it needs to run. Instead of manually installing every dependency on a system, users can run software inside a container that already includes the required libraries and tools.
What Is Apptainer?
Apptainer is a container platform that allows users to run software inside isolated environments without requiring administrator privileges on the host system.
Apptainer is the continuation of the Singularity project. The open-source Singularity project was renamed to Apptainer and continues to be developed under the Linux Foundation.
Unlike Docker, Apptainer does not require a privileged daemon running on the system. This makes it well suited for shared HPC environments where security and multi-user access are important considerations.
Torch uses Apptainer as its supported container platform. Many container images distributed through Docker registries can be used directly with Apptainer, allowing researchers to take advantage of existing software environments while working on the cluster.
What Are .sif And .sqf Files
Torch provides container images with both .sif and .sqf extensions.
.sif is the standard Apptainer image format and can be used directly with commands such as apptainer exec, apptainer run, and apptainer shell.
Some Torch-provided application images use .sqf (Squash File System) files and are typically accessed through wrapper scripts such as run-anaconda3-2024.10-1.bash. These scripts handle the details of launching the application environment and are often the recommended interface for users.
When available, use the wrapper script documented for that application. For general Apptainer examples in this tutorial, we use .sif images because they can be launched directly with standard Apptainer commands.
Apptainer Image Cache
While Apptainer doesn’t have a local image repository in the same way as Docker, it does cache downloaded image files. Images are simply .sif files stored on your local disk.
If you delete a local .sif image that you have pulled from a remote image repository and then pull it again, if the image is unchanged from the version you previously pulled, you will be given a copy of the image file from your local cache rather than the image being downloaded again from the remote source. This removes unnecessary network transfers and is particularly useful for large images which may take some time to transfer over the network.