Building and managing container images with Buildah

buildah is a open source tool which can be used to build OCI compliant container images without using docker engine. It can also be run in a rootless mode, thereby reducing the attack surface area and also inside a container image itself.

You can use buildah to built container images from existing container images, from Dockerfiles and from scratch (read empty images) as well. OCI images built using buildah are portable and can be run on different hosts as well with different container engines such as CRI-O, Podman, Docker Engine, etc.

Since buildah does not include build tools within the image itself, as a result the images produced using buildah are comparatively smaller in size. It also make container images more secure by not having the software used to build the container (like gcc, make, etc) within the resulting images. By default, buildah stores images in an area identified as containers-storage (/var/lib/containers).

Installing and Configuring Buildah

As of writing of this blog post, buildah is not supported on platform other than linux. However, it is supported on a number of linux distributions. See buildah’s installation instructions for a full list of prerequisites, and the buildah installation section in the Red Hat documentation for RHEL-specific instructions. Most of the time, it is as simple as instructing underlying package manager to grab and install it.

There are couple of extra steps involved with a rootless mode configuration. The setup required for this is listed on the Podman GitHub site here. Buildah has the same setup and configuration requirements that podman does for rootless users.

Verify the Buildah Installation

To check this, we can run some basic commands like buildah version (to show buildah version installed), buildah images (to show existing OCI images on the system), buildah containers (to show containers on the system):

# show buildah version installed
[cloud_user@3965c7b6ce1c ~]$ buildah version
Version:         1.16.7
Go Version:      go1.14.7
Image Spec:      1.0.1-dev
Runtime Spec:    1.0.2-dev
CNI Spec:        0.4.0
libcni Version:  
image Version:   5.6.0
Git Commit:      
Built:           Thu Jan  1 00:00:00 1970
OS/Arch:         linux/amd64

# show existing images on the system
[cloud_user@3965c7b6ce1c ~]$ buildah images
REPOSITORY                TAG      IMAGE ID       CREATED      SIZE


# show existing containers on the system
[cloud_user@3965c7b6ce1c ~]$ buildah containers
CONTAINER ID  BUILDER  IMAGE ID     IMAGE NAME                       CONTAINER NAME

Building an image from Dockerfile

You can use buildah to build images from Dockerfiles, if you have them handy. The build-using-dockerfile, or bud for short, takes a Dockerfile as input and produces an OCI image.

TFor example, we have a Dockerfile in our current directory which builds on the busybox image. Here, we are copying a simple script which echoes "Hello from buildah..." to the container image and marking it as entrypoint:

[cloud_user@3965c7b6ce1c buildah]$ ./echo.sh
Hello from buildah...

[cloud_user@3965c7b6ce1c buildah]$ cat Dockerfile
# Based on the busybox image
FROM busybox:latest
MAINTAINER Mohit Goyal noreply@mohitgoyal.co

WORKDIR /app
COPY echo.sh .
RUN chmod +x echo.sh

ENTRYPOINT ["/app/echo.sh"]

We can now build image with buildah as buildah bud -f Dockerfile, which is very similar to Docker CLI:

[cloud_user@3965c7b6ce1c buildah]$ buildah bud -f Dockerfile -t buildah-hello:1 .
STEP 1: FROM busybox:latest
Getting image source signatures
Copying blob 8b3d7e226fab done  
Copying config a9d583973f done  
Writing manifest to image destination
Storing signatures
STEP 2: MAINTAINER Mohit Goyal noreply@mohitgoyal.co
STEP 3: WORKDIR /app
STEP 4: COPY echo.sh .
STEP 5: RUN chmod +x echo.sh
STEP 6: ENTRYPOINT ["/app/echo.sh"]
STEP 7: COMMIT buildah-hello:1
Getting image source signatures
Copying blob 2983725f2649 skipped: already exists  
Copying blob b4f9e4e12143 done  
Copying config 1e452678c9 done  
Writing manifest to image destination
Storing signatures
--> 1e452678c91
1e452678c9154629c2a63bc512895666f070b979ea45279eb8ce2c8c4c588d64

Once the image is build, we can use buildah images to list container images:

[cloud_user@3965c7b6ce1c buildah]$ buildah images
REPOSITORY                  TAG      IMAGE ID       CREATED          SIZE
localhost/buildah-hello     1        1e452678c915   10 seconds ago   1.46 MB
docker.io/library/busybox   latest   a9d583973f65   2 weeks ago      1.45 MB

Now we can use container engine of our choice to run this image. In our case, we’ll be using podman:

[cloud_user@3965c7b6ce1c buildah]$ podman run localhost/buildah-hello:1
Hello from buildah...
[cloud_user@3965c7b6ce1c buildah]$

Similar to docker cli, buildah bud defaults to Dockerfile in absence of -f parameter.

Using buildah to inspect image and containers

We can use buildah inspect to show the metadata about an container image or a container:

# inspecting an image with buildah
[cloud_user@3965c7b6ce1c buildah]$ buildah inspect localhost/buildah-hello:1
{
    "Type": "buildah 0.0.1",
    "FromImage": "localhost/buildah-hello:1",
    "FromImageID": "b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0",
    "FromImageDigest": "sha256:8b14f44f9d9f4ec6d820c1d9caa1b373665b5a7dbecdf3bd880cb17da60b1b33",
    "Config": "{\"created\":\"2021-03-29T19:36:41.354041447Z\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\",\"architecture\":\"amd64\",\"os\":\"linux\",\"config\":{\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Entrypoint\":[\"/app/echo.sh\"],\"WorkingDir\":\"/app\",\"Labels\":{\"io.buildah.version\":\"1.16.7\"}},\"rootfs\":{\"type\":\"layers\",\"diff_ids\":[\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\"]},\"history\":[{\"created\":\"2021-03-09T21:26:43.567167128Z\",\"created_by\":\"/bin/sh -c #(nop) ADD file:b49bf6359240618f2ada9386537eabc4474a5be15d02912da968dcd0417d1294 in / \"},{\"created\":\"2021-03-09T21:26:43.742604734Z\",\"created_by\":\"/bin/sh -c #(nop)  CMD [\\\"sh\\\"]\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.144240542Z\",\"created_by\":\"/bin/sh -c #(nop) MAINTAINER Mohit Goyal noreply@mohitgoyal.co\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.255503941Z\",\"created_by\":\"/bin/sh -c #(nop) WORKDIR /app\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.710428711Z\",\"created_by\":\"/bin/sh -c #(nop) COPY file:3ac720fd9e26244587213fe6733e2f2e357ec54358eacedd19c2c47b6d9dcdb8 in . \",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:41.259850684Z\",\"created_by\":\"/bin/sh -c chmod +x echo.sh\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:42.000249645Z\",\"created_by\":\"/bin/sh -c #(nop) ENTRYPOINT [\\\"/app/echo.sh\\\"]\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\"}]}",
    "Container": "",
    "ContainerID": "",
    "MountPoint": "",
    "ProcessLabel": "",
    "MountLabel": "",
...
...

# inspecting a container with buildah
[cloud_user@3965c7b6ce1c buildah]$ podman ps -a
CONTAINER ID  IMAGE                      COMMAND  CREATED             STATUS                         PORTS   NAMES
e44067c14035  localhost/buildah-hello:1           About a minute ago  Exited (0) About a minute ago          sleepy_galois
[cloud_user@3965c7b6ce1c buildah]$ 
[cloud_user@3965c7b6ce1c buildah]$ buildah inspect sleepy_galois
{
    "Type": "buildah 0.0.1",
    "FromImage": "localhost/buildah-hello:1",
    "FromImageID": "b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0",
    "FromImageDigest": "sha256:8b14f44f9d9f4ec6d820c1d9caa1b373665b5a7dbecdf3bd880cb17da60b1b33",
    "Config": "{\"created\":\"2021-03-29T19:36:41.354041447Z\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\",\"architecture\":\"amd64\",\"os\":\"linux\",\"config\":{\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Entrypoint\":[\"/app/echo.sh\"],\"WorkingDir\":\"/app\",\"Labels\":{\"io.buildah.version\":\"1.16.7\"}},\"rootfs\":{\"type\":\"layers\",\"diff_ids\":[\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\"]},\"history\":[{\"created\":\"2021-03-09T21:26:43.567167128Z\",\"created_by\":\"/bin/sh -c #(nop) ADD file:b49bf6359240618f2ada9386537eabc4474a5be15d02912da968dcd0417d1294 in / \"},{\"created\":\"2021-03-09T21:26:43.742604734Z\",\"created_by\":\"/bin/sh -c #(nop)  CMD [\\\"sh\\\"]\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.144240542Z\",\"created_by\":\"/bin/sh -c #(nop) MAINTAINER Mohit Goyal noreply@mohitgoyal.co\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.255503941Z\",\"created_by\":\"/bin/sh -c #(nop) WORKDIR /app\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.710428711Z\",\"created_by\":\"/bin/sh -c #(nop) COPY file:3ac720fd9e26244587213fe6733e2f2e357ec54358eacedd19c2c47b6d9dcdb8 in . \",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:41.259850684Z\",\"created_by\":\"/bin/sh -c chmod +x echo.sh\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:42.000249645Z\",\"created_by\":\"/bin/sh -c #(nop) ENTRYPOINT [\\\"/app/echo.sh\\\"]\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\"}]}",
    "Manifest": "{\"schemaVersion\":2,\"config\":{\"mediaType\":\"application/vnd.oci.image.config.v1+json\",\"digest\":\"sha256:b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0\",\"size\":1462},\"layers\":[{\"mediaType\":\"application/vnd.oci.image.layer.v1.tar\",\"digest\":\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"size\":1449984},{\"mediaType\":\"application/vnd.oci.image.layer.v1.tar\",\"digest\":\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\",\"size\":6656}]}",
    "Container": "sleepy_galois",
    "ContainerID": "e44067c1403593562083f7257072926e6fcb06cb8c340d9270b7fa4d2855d161",
    "MountPoint": "",
    "ProcessLabel": "",
    "MountLabel": "",
    "ImageAnnotations": null,
    "ImageCreatedBy": "",

Building an image from scratch

In order to meet certain requirements, you may want to start from scratch rather than starting from another base image. In such scenarios, instead of starting with a base image, you can create a new container that holds no content and only a small amount of container metadata. This is referred to as a scratch container.

Building images and containers from scratch are very minimal in size and ideal for production usage since they are devoid of any utilities that can be used to alter the application inside container. This works fine when you are copying application binaries into the image from local system. However, this becomes an issue if you want to install some dependencies inside image such as using package managers like yum or dnf or apt etc. or if you want to compile code inside a container created using such image.

The special image name scratch tells buildah to create an empty container. The container has a small amount of metadata about the container but no real Linux content:

[cloud_user@3965c7b6ce1c buildah]$ newcontainer=$(buildah from scratch)

We can see this new empty container by running buildah containers:

[cloud_user@3965c7b6ce1c buildah]$ buildah containers
CONTAINER ID  BUILDER  IMAGE ID     IMAGE NAME                       CONTAINER NAME
2236fca85b74     *                  scratch                          working-container

However, if we run buildah images, we would not see it listed:

[cloud_user@3965c7b6ce1c buildah]$ buildah images
REPOSITORY                  TAG      IMAGE ID       CREATED       SIZE
localhost/buildah-hello     1        1e452678c915   4 hours ago   1.46 MB
docker.io/library/busybox   latest   a9d583973f65   2 weeks ago   1.45 MB

This is fine as it just indicates that there is no real image yet. i.e. It is containers/storage but there is no representation in containers/image. It has nothing in it yet. It is essentially an empty layer on top of the kernel. So how do we modify it? We need to first mount it on our filesystem and then enter the mount namespace by executing the buildah unshare command (for this we need to work as root user):

[root@3965c7b6ce1c ~]# scratchmnt=$(buildah mount working-container)

[root@3965c7b6ce1c ~]# echo $scratchmnt
/var/lib/containers/storage/overlay/e3f02c79964083eccc68ab49476a2e26abc09836018c90dee5a5648f71b423c8/merged

Note that our image layer is an overlay on top of /var/lib/containers. Now we can initialize an RPM database within the scratch image and add the redhat-release package:

[root@3965c7b6ce1c ~]# yum install -y --releasever=8 --installroot=$scratchmnt redhat-release
Red Hat Update Infrastructure 3 Client Configuration Server 8                                                                            23 kB/s | 3.8 kB     00:00    
Red Hat Enterprise Linux 8 for x86_64 - AppStream from RHUI (RPMs)                                                                       62 MB/s |  26 MB     00:00    
Red Hat Enterprise Linux 8 for x86_64 - BaseOS from RHUI (RPMs)                                                                          77 MB/s |  28 MB     00:00    
Last metadata expiration check: 0:00:01 ago on Mon 29 Mar 2021 03:28:01 PM UTC.
Detection of Platform Module failed: No valid Platform ID detected
...
...
Running transaction
  Preparing        :                                                                                                                                                1/1 
  Installing       : redhat-release-eula-8.3-1.0.el8.x86_64                                                                                                         1/2 
  Installing       : redhat-release-8.3-1.0.el8.x86_64                                                                                                              2/2 
  Verifying        : redhat-release-8.3-1.0.el8.x86_64                                                                                                              1/2 
  Verifying        : redhat-release-eula-8.3-1.0.el8.x86_64                                                                                                         2/2 

Installed:
  redhat-release-8.3-1.0.el8.x86_64                                                redhat-release-eula-8.3-1.0.el8.x86_64                                               

Complete!

Now we can install the httpd service to the scratch directory:

[root@3965c7b6ce1c ~]# yum install -y --setopt=reposdir=/etc/yum.repos.d \
>      --installroot=$scratchmnt \
>      --setopt=cachedir=/var/cache/dnf httpd
Last metadata expiration check: 2:04:04 ago on Mon 29 Mar 2021 01:25:40 PM UTC.
Dependencies resolved.
========================================================================================================================================================================
 Package                                    Architecture          Version                                               Repository                                 Size
========================================================================================================================================================================
Installing:
 httpd                                      x86_64                2.4.37-30.module+el8.3.0+7001+0766b9e7                rhel-8-appstream-rhui-rpms                1.4 M
Installing dependencies:
 acl                                        x86_64                2.2.53-1.el8                                          rhel-8-baseos-rhui-rpms                    81 k
 apr                                        x86_64                1.6.3-11.el8                                          rhel-8-appstream-rhui-rpms                125 k
 apr-util                                   x86_64                1.6.1-6.el8                                           rhel-8-appstream-rhui-rpms                105 k
 audit-libs                                 x86_64                3.0-0.17.20191104git1c2f876.el8                       rhel-8-baseos-rhui-rpms                   116 k
 basesystem                                 noarch                11-5.el8                                              rhel-8-baseos-rhui-rpms                    11 k
 bash                                       x86_64                4.4.19-12.el8                                         rhel-8-baseos-rhui-rpms                   1.5 M
...
...
  util-linux-2.32.1-24.el8.x86_64                                                  which-2.21-12.el8.x86_64                                                             
  xkeyboard-config-2.28-1.el8.noarch                                               xz-5.2.4-3.el8.x86_64                                                                
  xz-libs-5.2.4-3.el8.x86_64                                                       zlib-1.2.11-16.el8_2.x86_64                                                          

Complete!

Finally, we can add some httpd specific configuration to the image and save the container with buildah commit:

[root@3965c7b6ce1c ~]# echo "Built httpd image from scratch..." > $scratchmnt/var/www/html/index.html

[root@3965c7b6ce1c ~]# buildah config --cmd "/usr/sbin/httpd -DFOREGROUND" working-container

[root@3965c7b6ce1c ~]# buildah config --port 80/tcp working-container

[root@3965c7b6ce1c ~]# buildah commit working-container localhost/buildah-httpd:1
Getting image source signatures
Copying blob 0c30c4933dd5 done  
Copying config 52427f76ae done  
Writing manifest to image destination
Storing signatures

52427f76aec07d575be4eea374340b786f211ae58f8ccfc7d6730cf0a0abe558
[root@3965c7b6ce1c ~]# 
[root@3965c7b6ce1c ~]# buildah images
REPOSITORY                TAG   IMAGE ID       CREATED              SIZE
localhost/buildah-httpd   1     52427f76aec0   About a minute ago   637 MB

As we can see above, buildah images list the image localhost/buildah-httpd:1 we just created from scratch. We can now use podman to create a container based on this image and then see it in action:

[root@3965c7b6ce1c ~]# podman run -d -p 8080:80 localhost/buildah-httpd:1
51334e0119e266ae53f3ec4a425d897be02e846e3a186c511d7cd6f6866f4769

[root@3965c7b6ce1c ~]# curl http://localhost:8080
Built httpd image from scratch...

Later when we want to create a new container or containers from this image, we simply need need to do buildah from localhost/buildah-httpd:1. This will create a new containers based on this image for you.

Removing images or containers

To remove the scratch container called working-container, we can run below commands:

# buildah rm $newcontainer

Or 

# buildah rm working-container

To remove the image, we can use buildah rmi. However, do note that like other container engines, it will need you to first remove all referencing containers to image you are removing:

# list images present
[cloud_user@3965c7b6ce1c ~]$ buildah images
REPOSITORY                  TAG      IMAGE ID       CREATED       SIZE
localhost/buildah-hello     1        1e452678c915   8 hours ago   1.46 MB
docker.io/library/busybox   latest   a9d583973f65   2 weeks ago   1.45 MB

# lets try to remove buildah-hello image                              
[cloud_user@3965c7b6ce1c ~]$ buildah rmi localhost/buildah-hello:1
Could not remove image "localhost/buildah-hello:1" (must force) - container "3c7c2586f5f87e43772d0442b628880b3d7bd2efc24be7427bffb6a39a5f529d" is using its reference image: image is in use by a container
ERRO exit status 125    
                          

# verify that we have containers referencing this image
[cloud_user@3965c7b6ce1c ~]$ podman ps -a
CONTAINER ID  IMAGE                      COMMAND  CREATED      STATUS                  PORTS   NAMES
3c7c2586f5f8  localhost/buildah-hello:1           8 hours ago  Exited (0) 8 hours ago          charming_herschel


# remove containers
[cloud_user@3965c7b6ce1c ~]$ podman rm charming_herschel
3c7c2586f5f87e43772d0442b628880b3d7bd2efc24be7427bffb6a39a5f529d


# remove the image with buildah
[cloud_user@3965c7b6ce1c ~]$ buildah rmi localhost/buildah-hello:1
untagged: localhost/buildah-hello:1
1e452678c9154629c2a63bc512895666f070b979ea45279eb8ce2c8c4c588d64

# verify that image is removed
[cloud_user@3965c7b6ce1c ~]$ buildah images
REPOSITORY                  TAG      IMAGE ID       CREATED       SIZE
docker.io/library/busybox   latest   a9d583973f65   2 weeks ago   1.45 MB

Push and Pull Images from Container Registries

We can also use buildah to push and pull container images from private or public container registries. For the purpose of this blog post, we are connected to local registry on our machine. If you want to setup a local registry, see steps here. Let’s review the current state of our registry:

[root@3965c7b6ce1c ~]# curl -k -u user:password https://localhost:5000/v2/_catalog
{"repositories":[]}

We can see that we do not have any images in our local registry as of now. Lets try to push image localhost/buildah-hello:1 to it. Do note that by default, buildah is set up to expect secure connections to a registry. Since we are using self-signed cert for local instance of registry, we need to tell buildah to turn the TLS verification off using the --tls-verify flag. e also need to tell Buildah that the registry is on this local host ( i.e. localhost) and listening on port 5000. Similar to what you’d expect to do on multi-tenant Docker hub, we will explicitly specify that the registry is to store the image under the demo repository – so as not to clash with other users’ similarly named images:

# push image to registry
[cloud_user@3965c7b6ce1c privreg]$ buildah push --tls-verify=false --creds user:password buildah-hello:1 docker://localhost:5000/demo/buildah-hello:1
Getting image source signatures
Copying blob 81def7be1c69 done  
Copying blob 2983725f2649 done  
Copying config b91cc779e9 done  
Writing manifest to image destination
Storing signatures


# verify the images present in the registry
[cloud_user@3965c7b6ce1c privreg]$ curl -k -u user:password https://localhost:5000/v2/_catalog{"repositories":["demo/buildah-hello"]}

We can use buildah to pull down the image using the buildah from command. But before that, we need to clean up our local image so that we do not have an existing localhost/buildah-hello:1 – otherwise buildah will know it already exists and not bother pulling it down:

# verify the images already present
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY                   TAG      IMAGE ID       CREATED        SIZE
localhost/buildah-hello      1        b91cc779e908   36 hours ago   1.46 MB
docker.io/library/registry   latest   eefcac9e3856   5 days ago     26.8 MB
docker.io/library/busybox    latest   a9d583973f65   3 weeks ago    1.45 MB

# remove the image localhost/buildah-hello:1                             
                           
[cloud_user@3965c7b6ce1c privreg]$ buildah rmi localhost/buildah-hello:1
untagged: localhost/buildah-hello:1
b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0


# verify that image is removed
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY                   TAG      IMAGE ID       CREATED       SIZE
docker.io/library/registry   latest   eefcac9e3856   5 days ago    26.8 MB
docker.io/library/busybox    latest   a9d583973f65   3 weeks ago   1.45 MB


# pull image from container repository
[cloud_user@3965c7b6ce1c privreg]$ buildah from --tls-verify=false --creds user:password docker://localhost:5000/demo/buildah-hello:1Getting image source signatures
Copying blob 62bad172f512 skipped: already exists  
Copying blob 26f86aeede30 done  
Copying config b91cc779e9 done  
Writing manifest to image destination
Storing signatures
localhost-working-container


# verify that image is present now
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY                          TAG      IMAGE ID       CREATED        SIZE
localhost:5000/demo/buildah-hello   1        b91cc779e908   36 hours ago   1.46 MB
docker.io/library/registry          latest   eefcac9e3856   5 days ago     26.8 MB
docker.io/library/busybox           latest   a9d583973f65   3 weeks ago    1.45 MB

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s