|
Introduction to containers
|
|
|
Basics of Singularity
|
Singularity can run both Singularity and Docker container images
Execute commands in containers with singularity exec
Open a shell in a container with singularity shell
Download a container image in a selected location with singularity pull
|
|
Share files with the host: BLAST, a bioinformatics example
|
By default Singularity mounts the host current directory, and uses it as container working directory
Map additional host directories in the containers with the flag -B
|
|
Set up writable containers: another bio example with Trinity
|
Use linux tool from a Ubuntu container to create a filesystem image file
Mount a filesystem image using the flag --overlay
Make a SIF container ephemerally writable with the flag --writable-tmpfs
|
|
Molecular dynamics with GPU containers
|
|
|
Build your own container image
|
Build images using sudo singularity build
Use the remote builder with the flag -r, if you need to build images from a machine where you don’t have sudo rights
You can share you Singularity Image File with others, as you would do with any other (big) file
Upload images to a web registry with singularity push (Sylabs account required)
|
|
Computational Fluid Dynamics with MPI containers
|
Singularity interfaces with HPC schedulers such as Slurm, with some requirements
You need to build your application in the container with an MPI version which is ABI compatible with MPI libraries in the host
Appropriate environment variables and bind mounts are required at runtime to make the most out of MPI applications (sys admins can help)
|
|
GUI enabled applications: RStudio in a container
|
An interactive session can essentially be executed as any other containerised application, via singularity exec
Use the %startscript section of a def file to configure an image for long running services
Launch/shutdown long running services in the background with singularity instance start/stop
|
|
Making Python not awful with containers
|
You can use idiomatic syntaxes of singularity exec to launch interactive sessions of Jupyter Notebooks, similar to RStudio
Run the same workflow through command line or via Notebook using the same container
|
|
Reproducible scientific workflows
|
Some workflow engines offer transparent APIs for running containerised applications
If you need to run data analysis pipelines, the combination of containers and workflow engines can really make your life easier!
|
|
Overview of Docker
|
Use docker pull to download container images
Use docker run to execute commands in containers
Build images starting from a Dockerfile recipe with docker build
Push images to a web registry with docker push
|
|
Other notable container tools
|
HPCCM can be useful to write image recipes that are compatible both with Docker and Singularity
Other than Singularity, at the moment other interesting container engines for HPC are Sarus, Charliecloud and enroot
|