Adding packages to a container

You may need to install additional packages if your existing container is missing software required for your work. The recommended practice is to create a copy of the container and modify this. This allows you to make the changes you want without affecting the original container.

Step 1: Create a copy of the container

You can either copy a container from the repository /ceph/container/ Or use a container from your own directory /ceph/home/student.aau.dk/user

Use the command ls to see which containers are available in the choosen folder. Every file which ends with .sif is a container.

Choose the container you want to modify and navigate to the folder where it is located by using cd. Here's an example:

cd /ceph/container/pytorch/

Run the following command to create a copy of your selected container:

cp pytorch_26.04.sif ~/my_container.sif

What does the command?

• cp – copies a file.
• pytorch_26.04.sif – the original container (source file).
• ~/my_container.sif – the new copy is saved in your home directory "~/" with the name "my_container.sif"

Step 2: Check which packages are in the selected container

Before adding a new package, you can check which Python packages are already installed in the container.

Navigate to the folder where your container is stored.

cd ~/

srun singularity exec my_container.sif pip list

What does the command?

• srun – asks Slurm to run the command as a job on a compute node.
• singularity exec – tells Singularity to run a command inside the container.
• my_container.sif – the name of the container you want to use.
• pip list – displays the Python packages installed in the container, along with their versions.

A reccomendation is to make a list of the packages you are missing in the container you want to modify.

Step 3: Build a sandbox

A sandbox is a writable version of a container where you can make changes and install additional packages. If you copied the container you want to modify to your home directory, you can build a sandbox with the following command:

srun singularity build --sandbox ~/my_container/ ~/my_container.sif

• srun – asks Slurm to run the command as a job on a compute node.
• singularity build – is used to create a new container from an existing container.
• --sandbox – tells Singularity to create the new container as a writable sandbox directory instead of a .sif file.
• ~/my_container/ – the location and name of the new sandbox.
• ~/my_container.sif – the existing .sif container that the sandbox is created from.

Note: Building the sandbox can take a couple of minutes. When the proces is complete, you will get the info message INFO: Build complete: /ceph/.../my_container/

Step 4: Install packages in your sandbox

To be able to install additional packages, navigate to the folder where your sandbox is located and open a interactive session:

cd ~/

srun --pty singularity shell my_container/

What does the command?

• srun – asks Slurm to run the command as a job on a compute node.
• singularity shell – opens an interactive shell inside the container.
• my_container/ – the sandbox directory you want to open.

You can now tell that you are inside your sandbox because the terminal prompt shows: Singularity>

Now you are free to install the additional packages you need!

pip install package_name

To see all installed Python packages, run:

pip list

If you want to check whether a specific package is installed, run:

pip show package_name

Congrats you have now installed the additional packages in your sandbox!

When you are finished, exit the Singularity shell by running: exit

Did you exit your shell before you were done? Or have you closed the terminal and want to continue installing packages? Just navigate to the folder where you placed your sandbox and then run the following command:

srun singularity shell my_container/

Run scripts in your sandbox

Once you have installed the required packages, you can use your modified sandbox to run your scripts without converting it to a .sif container first. This is useful while you are still testing your environment or making changes to the installed packages.

Run a script in the sandbox

Run your script directly from the sandbox with:

srun singularity exec my_container/ python my_script.py

• srun – asks Slurm to run the command as a job on a compute node.
• singularity exec – runs a command inside the sandbox.
• my_container/ – the sandbox containing your installed packages.
• python my_script.py – runs your Python script inside the sandbox.

When Should You Convert the Sandbox to a .sif Container?

A sandbox is ideal while you are installing packages, testing your environment, and making changes because it is writable.

Once you are finished modifying the container, it is recommended to convert the sandbox into a .sif container.

A .sif container is a better choice for long-term use because it:

• provides a fixed and reproducible environment,
• is easier to share with others,
• is more suitable for running production or batch jobs, and
• prevents accidental changes to the container.

Convert the sandbox back into a .sif container

To convert your sandbox back into a .sif container, create a Slurm batch script and run it with sbatch. If needed, revisit the Using containers to run jobs guide.

Using a batch script is recommended because it:

• continues running even if you close your terminal,
• reduces the risk of the job stopping due to time limits or a frozen terminal, and
• allows you to choose the appropriate number of CPU cores and amount of memory to complete the conversion more efficiently.

Add the following content to your batch script:

#!/bin/bash
#SBATCH --job-name=job_container
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=16
#SBATCH --mem=64G
#SBATCH --time=12:00:00
#SBATCH --output=logs/%x_%j.out
#SBATCH --error=logs/%x_%j.err

mkdir -p logs

singularity build ~/my_container_updated.sif ~/my_container/

Note: that the job can take some time.

Verify and test the new container

After the job has finished, check that a .sif container with the name you specified has been created by running this command:

ls -lh ~/*.sif

Start the container by running:

srun singularity shell ~/my_container_updated.sif

python -m pip list

Congrats, you can now run a job with your modified container and share your container with others!