4. Building Containers¶
Lesson Objectives
- Know how to write a definition (
def) file - Use the
buildcommand to build a container
Questions
- How do I build my own container from scratch?
- How do I describe what goes into a container and what it does when it runs?
It is often very useful to be able to build your own containers. This is because you might want to:
- Create an environment with toolchains that don't exist or don't exist exactly as you need them on Mahuika
- Want to create environments that are transferable to other scientists on other HPC systems
Here, we will learn how to build a container using Apptainer from a script called a definition file, known as a def file.
1. Creating a def (Definition) File¶
A def file is a simple script that contains information about how to construct a container and what the container should do. A simple skeleton of a def file looks as follows:
In this section, we will write a def file containing the most important sections
The Bootstrap and From Section¶
To build a container, you need a base to build it on. Bootstrap and From allow Apptainer to pull the base that you desire:
Bootstrap: The cloud or local archive that the base of the container comes from.From: What is the name of the base you want to pull from theBootstrapcloud or local archive.
You must always include Bootstrap — it is the only keyword required for every build, and the build will fail without it. (If you want to build a container without a base operating system, you can set Bootstrap: scratch).
For example, if I wanted to build the container based on Ubuntu 24.04, I would include the following in the Bootstrap and From sections of the def file:
Another example is from ghcr.io (the Github Container Registry, which is a cloud-based service for storing and managing Docker and OCI-compliant container images):
Bases located on your computer
Your base might not come from a cloud. Your base could be set to a local file that is sitting on your computer.
If your base is an Apptainer sif file, include the following at the start of your def file:
If you obtained your base image from Docker and have it stored locally (using docker save -o my_docker_archive_file.tar mydocker):
We will come back to this in Questions 3a and 3b.
The %labels Section¶
Used to store metadata such as the author or software version within the image. You can define a version label here to track image iterations. For example:
The %environment Section¶
The %environment section defines environment variables that are set every time the container runs. You do not always need to include this, only if you have environment variables you need to set in the container. An example of %environment section is:
The %post Section¶
This section allows you to build your container the way you want it and install all the programs and packages that you want your container to contain.
For example, if you want to create a container that contained Python 3.12 (as well as all the other packages that Python needs), you could write it like so (Note: there are many ways one could write this def file. This is just one of those ways):
%post
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y tzdata software-properties-common
ln -fs /usr/share/zoneinfo/UTC /etc/localtime
dpkg-reconfigure -f noninteractive tzdata
add-apt-repository -y ppa:deadsnakes/ppa
apt-get update
apt-get install -y python3.12
The %runscript Section¶
This section is responsible for determining what happens when you perform apptainer run. This is optional, as you could always execute your container using apptainer exec. For example:
2. Build your Container¶
Now that we have the basics of our .def file (given below, and available on GitHub):
Bootstrap: docker
From: ubuntu:24.04
%labels
Author Your Name
Version 1.0.0
Description "Example Apptainer container with Python 3.12"
%environment
export PYTHONNOUSERSITE=1
export LANG=C.UTF-8
export LC_ALL=C.UTF-8
%post
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y tzdata software-properties-common
ln -fs /usr/share/zoneinfo/UTC /etc/localtime
dpkg-reconfigure -f noninteractive tzdata
add-apt-repository -y ppa:deadsnakes/ppa
apt-get update
apt-get install -y python3.12
%runscript
python3.12 -c 'print("hello world")'
We can now build our container. Our container in Apptainer is called a sif file, which stands for Singularity Image Format (Singularity is the predecessor of Apptainer). To build our container, we type into the terminal:
where <name-of-sif-file> is the name of our sif file, and <name-of-def-file> is the name of our def file. For example, if we set the name of our sif and def files as my_python3.12.sif and my_python3.12.def respectively, we would type into the terminal:
Giving the output:
user.name@computer-name:~$ apptainer build my_python3.12.sif my_python3.12.def
INFO: User not listed in /etc/subuid, trying root-mapped namespace
INFO: The %post section will be run under the fakeroot command
INFO: Starting build...
INFO: Fetching OCI image...
28.4MiB / 28.4MiB [===================================================================================================] 100 % 0.0 b/s 0s
INFO: Extracting OCI image...
INFO: Inserting Apptainer configuration...
INFO: Running post scriptlet
+ apt-get update
...
INFO: Adding runscript
INFO: Creating SIF file...
3. Run or Execute your Container¶
You can now run your container using apptainer run <name-of-your-sif-file>. For example:
We can also use the exec command to run python3.12 to say "Hello Mars!"
Other Useful Tips and Tricks¶
Using the $1, $2, $3, and $@ symbols in apptainer run¶
Sometimes you want to pass a command into the apptainer run command so you can easily do different things. To do this, we use the $1, $2, $3, and $@ symbols. For example, lets consider that we want to write a container that will say hello to some input planet. We could write the following def file (tip1.def):
We can now run this container and give it 1 input for a planet we want to say hello to:
We could now build a container that allows us to say hello to three planets. We can do this by using $1, $2, and $3 (tip2.def):
Doing this gives:
apptainer build tip2.sif tip2.def
apptainer run tip2.sif Mercury Venus Earth
Hello Mercury, Venus, and Earth!
Finally, maybe we don't want to have any sort of limit to the number of planets that we say hello to. In this case, we can use the $@ symbol which will access all arguments (inputs) given to apptainer run. For example, consider the following def file (tip3.def):
If we run this script, giving the input as Mercury Venus Earth Mars Jupiter Saturn Neptune, we will get:
apptainer build tip3.sif tip3.def
apptainer run tip3.sif Mercury Venus Earth Mars Jupiter Saturn Neptune
Hello Mercury Venus Earth Mars Jupiter Saturn Neptune!
Exercises¶
Question 1
Write a def file that will allow the user to build a container that runs lolcow from their terminal.
Hint: The instructions for building lolcow are:
and the way you run lolcow is by doing the following in the terminal:
Hint: fortune and cowsay install into /usr/games, which is not on the container's default PATH. You will need to add it using an %environment section so the container can find them:
Solution
The def file for lolcow (available as lolcow.def) is:
Question 2
Using the answer from Question 1, how would you build and run the lolcow container?
Solution
Type into the terminal:
You will see something like this as the lolcow container is being built:
user.name@computer-name:~$ apptainer build lolcow.sif lolcow.def
INFO: User not listed in /etc/subuid, trying root-mapped namespace
INFO: The %post section will be run under the fakeroot command
INFO: Starting build...
INFO: Fetching OCI image...
28.4MiB / 28.4MiB [================================================================================================================================================================================] 100 % 9.4 MiB/s 0s
INFO: Extracting OCI image...
INFO: Inserting Apptainer configuration...
INFO: Running post scriptlet
+ apt-get -y update
...
done.
INFO: Adding environment to container
INFO: Adding runscript
INFO: Creating SIF file...
[============================================================================================================================================================================================================] 100 % 0s
INFO: Build complete: lolcow.sif
You can then run the lolcow container by running in the terminal
This will give an output like this:
Question 3a
A user has a container file called my_base.sif they would like use to as a base for their def file. How would this user include my_base.sif as the base image in their def file?
Question 3b
A second user has a Docker archive file called my_docker_base.tar they downloaded from Docker (using docker save -o my_docker_base.tar my_docker). The user would like to use this Docker archive file as a base for their Apptainer def file. How would this user include my_docker_base.tar as the base image in their def file?
Question 4
Using the skills you have learnt in this tutorial, create your own container for running a program that would be useful for you on Mahuika.
Keypoints
- Can write a
deffile that Apptainer can use to build a container, including the:BootstrapandFromsections for downloading the base of the container.%labelssection for storing metadata such as the author and version.%environmentsection for setting environment variables that are available when the container runs.%postsection for customising what is built when building the container.%runscriptsection for writing the desired command you would commonly want the user to run when using the container.
- Understand how to use the
buildcommand for building a container from thedeffile - Know how to include
$1,$2,$3, and$@symbols in%runscriptso the user can pass arguments intoapptainer run