buildah is a open source tool which can be used to build OCI compliant container images without using docker engine. It can also be run in a rootless mode, thereby reducing the attack surface area and also inside a container image itself.
You can use buildah to built container images from existing container images, from Dockerfiles and from scratch (read empty images) as well. OCI images built using buildah are portable and can be run on different hosts as well with different container engines such as CRI-O, Podman, Docker Engine, etc.
Since buildah does not include build tools within the image itself, as a result the images produced using buildah are comparatively smaller in size. It also make container images more secure by not having the software used to build the container (like gcc, make, etc) within the resulting images. By default, buildah stores images in an area identified as containers-storage (/var/lib/containers).
Installing and Configuring Buildah
As of writing of this blog post, buildah is not supported on platform other than linux. However, it is supported on a number of linux distributions. See buildah’s installation instructions for a full list of prerequisites, and the buildah installation section in the Red Hat documentation for RHEL-specific instructions. Most of the time, it is as simple as instructing underlying package manager to grab and install it.
There are couple of extra steps involved with a rootless mode configuration. The setup required for this is listed on the Podman GitHub site here. Buildah has the same setup and configuration requirements that podman does for rootless users.
Verify the Buildah Installation
To check this, we can run some basic commands like buildah version (to show buildah version installed), buildah images (to show existing OCI images on the system), buildah containers (to show containers on the system):
# show buildah version installed [cloud_user@3965c7b6ce1c ~]$ buildah version Version: 1.16.7 Go Version: go1.14.7 Image Spec: 1.0.1-dev Runtime Spec: 1.0.2-dev CNI Spec: 0.4.0 libcni Version: image Version: 5.6.0 Git Commit: Built: Thu Jan 1 00:00:00 1970 OS/Arch: linux/amd64 # show existing images on the system [cloud_user@3965c7b6ce1c ~]$ buildah images REPOSITORY TAG IMAGE ID CREATED SIZE # show existing containers on the system [cloud_user@3965c7b6ce1c ~]$ buildah containers CONTAINER ID BUILDER IMAGE ID IMAGE NAME CONTAINER NAME
Building an image from Dockerfile
You can use buildah to build images from Dockerfiles, if you have them handy. The build-using-dockerfile, or bud for short, takes a Dockerfile as input and produces an OCI image.
TFor example, we have a Dockerfile in our current directory which builds on the busybox image. Here, we are copying a simple script which echoes "Hello from buildah..." to the container image and marking it as entrypoint:
[cloud_user@3965c7b6ce1c buildah]$ ./echo.sh Hello from buildah... [cloud_user@3965c7b6ce1c buildah]$ cat Dockerfile # Based on the busybox image FROM busybox:latest MAINTAINER Mohit Goyal noreply@mohitgoyal.co WORKDIR /app COPY echo.sh . RUN chmod +x echo.sh ENTRYPOINT ["/app/echo.sh"]
We can now build image with buildah as buildah bud -f Dockerfile, which is very similar to Docker CLI:
[cloud_user@3965c7b6ce1c buildah]$ buildah bud -f Dockerfile -t buildah-hello:1 . STEP 1: FROM busybox:latest Getting image source signatures Copying blob 8b3d7e226fab done Copying config a9d583973f done Writing manifest to image destination Storing signatures STEP 2: MAINTAINER Mohit Goyal noreply@mohitgoyal.co STEP 3: WORKDIR /app STEP 4: COPY echo.sh . STEP 5: RUN chmod +x echo.sh STEP 6: ENTRYPOINT ["/app/echo.sh"] STEP 7: COMMIT buildah-hello:1 Getting image source signatures Copying blob 2983725f2649 skipped: already exists Copying blob b4f9e4e12143 done Copying config 1e452678c9 done Writing manifest to image destination Storing signatures --> 1e452678c91 1e452678c9154629c2a63bc512895666f070b979ea45279eb8ce2c8c4c588d64
Once the image is build, we can use buildah images to list container images:
[cloud_user@3965c7b6ce1c buildah]$ buildah images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/buildah-hello 1 1e452678c915 10 seconds ago 1.46 MB docker.io/library/busybox latest a9d583973f65 2 weeks ago 1.45 MB
Now we can use container engine of our choice to run this image. In our case, we’ll be using podman:
[cloud_user@3965c7b6ce1c buildah]$ podman run localhost/buildah-hello:1 Hello from buildah... [cloud_user@3965c7b6ce1c buildah]$
Similar to docker cli, buildah bud defaults to Dockerfile in absence of -f parameter.
Using buildah to inspect image and containers
We can use buildah inspect to show the metadata about an container image or a container:
# inspecting an image with buildah
[cloud_user@3965c7b6ce1c buildah]$ buildah inspect localhost/buildah-hello:1
{
"Type": "buildah 0.0.1",
"FromImage": "localhost/buildah-hello:1",
"FromImageID": "b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0",
"FromImageDigest": "sha256:8b14f44f9d9f4ec6d820c1d9caa1b373665b5a7dbecdf3bd880cb17da60b1b33",
"Config": "{\"created\":\"2021-03-29T19:36:41.354041447Z\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\",\"architecture\":\"amd64\",\"os\":\"linux\",\"config\":{\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Entrypoint\":[\"/app/echo.sh\"],\"WorkingDir\":\"/app\",\"Labels\":{\"io.buildah.version\":\"1.16.7\"}},\"rootfs\":{\"type\":\"layers\",\"diff_ids\":[\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\"]},\"history\":[{\"created\":\"2021-03-09T21:26:43.567167128Z\",\"created_by\":\"/bin/sh -c #(nop) ADD file:b49bf6359240618f2ada9386537eabc4474a5be15d02912da968dcd0417d1294 in / \"},{\"created\":\"2021-03-09T21:26:43.742604734Z\",\"created_by\":\"/bin/sh -c #(nop) CMD [\\\"sh\\\"]\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.144240542Z\",\"created_by\":\"/bin/sh -c #(nop) MAINTAINER Mohit Goyal noreply@mohitgoyal.co\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.255503941Z\",\"created_by\":\"/bin/sh -c #(nop) WORKDIR /app\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.710428711Z\",\"created_by\":\"/bin/sh -c #(nop) COPY file:3ac720fd9e26244587213fe6733e2f2e357ec54358eacedd19c2c47b6d9dcdb8 in . \",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:41.259850684Z\",\"created_by\":\"/bin/sh -c chmod +x echo.sh\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:42.000249645Z\",\"created_by\":\"/bin/sh -c #(nop) ENTRYPOINT [\\\"/app/echo.sh\\\"]\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\"}]}",
"Container": "",
"ContainerID": "",
"MountPoint": "",
"ProcessLabel": "",
"MountLabel": "",
...
...
# inspecting a container with buildah
[cloud_user@3965c7b6ce1c buildah]$ podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e44067c14035 localhost/buildah-hello:1 About a minute ago Exited (0) About a minute ago sleepy_galois
[cloud_user@3965c7b6ce1c buildah]$
[cloud_user@3965c7b6ce1c buildah]$ buildah inspect sleepy_galois
{
"Type": "buildah 0.0.1",
"FromImage": "localhost/buildah-hello:1",
"FromImageID": "b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0",
"FromImageDigest": "sha256:8b14f44f9d9f4ec6d820c1d9caa1b373665b5a7dbecdf3bd880cb17da60b1b33",
"Config": "{\"created\":\"2021-03-29T19:36:41.354041447Z\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\",\"architecture\":\"amd64\",\"os\":\"linux\",\"config\":{\"Env\":[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\"],\"Entrypoint\":[\"/app/echo.sh\"],\"WorkingDir\":\"/app\",\"Labels\":{\"io.buildah.version\":\"1.16.7\"}},\"rootfs\":{\"type\":\"layers\",\"diff_ids\":[\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\"]},\"history\":[{\"created\":\"2021-03-09T21:26:43.567167128Z\",\"created_by\":\"/bin/sh -c #(nop) ADD file:b49bf6359240618f2ada9386537eabc4474a5be15d02912da968dcd0417d1294 in / \"},{\"created\":\"2021-03-09T21:26:43.742604734Z\",\"created_by\":\"/bin/sh -c #(nop) CMD [\\\"sh\\\"]\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.144240542Z\",\"created_by\":\"/bin/sh -c #(nop) MAINTAINER Mohit Goyal noreply@mohitgoyal.co\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.255503941Z\",\"created_by\":\"/bin/sh -c #(nop) WORKDIR /app\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:39.710428711Z\",\"created_by\":\"/bin/sh -c #(nop) COPY file:3ac720fd9e26244587213fe6733e2f2e357ec54358eacedd19c2c47b6d9dcdb8 in . \",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:41.259850684Z\",\"created_by\":\"/bin/sh -c chmod +x echo.sh\",\"empty_layer\":true},{\"created\":\"2021-03-29T19:36:42.000249645Z\",\"created_by\":\"/bin/sh -c #(nop) ENTRYPOINT [\\\"/app/echo.sh\\\"]\",\"author\":\"Mohit Goyal noreply@mohitgoyal.co\"}]}",
"Manifest": "{\"schemaVersion\":2,\"config\":{\"mediaType\":\"application/vnd.oci.image.config.v1+json\",\"digest\":\"sha256:b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0\",\"size\":1462},\"layers\":[{\"mediaType\":\"application/vnd.oci.image.layer.v1.tar\",\"digest\":\"sha256:2983725f2649f8847244cbb73ff9cb0b041bd319144816dfdee904adfd18bd1f\",\"size\":1449984},{\"mediaType\":\"application/vnd.oci.image.layer.v1.tar\",\"digest\":\"sha256:81def7be1c69e6a2d3bba74a5b0a594a13f5da5e8fe468327a0021e119398129\",\"size\":6656}]}",
"Container": "sleepy_galois",
"ContainerID": "e44067c1403593562083f7257072926e6fcb06cb8c340d9270b7fa4d2855d161",
"MountPoint": "",
"ProcessLabel": "",
"MountLabel": "",
"ImageAnnotations": null,
"ImageCreatedBy": "",
Building an image from scratch
In order to meet certain requirements, you may want to start from scratch rather than starting from another base image. In such scenarios, instead of starting with a base image, you can create a new container that holds no content and only a small amount of container metadata. This is referred to as a scratch container.
Building images and containers from scratch are very minimal in size and ideal for production usage since they are devoid of any utilities that can be used to alter the application inside container. This works fine when you are copying application binaries into the image from local system. However, this becomes an issue if you want to install some dependencies inside image such as using package managers like yum or dnf or apt etc. or if you want to compile code inside a container created using such image.
The special image name scratch tells buildah to create an empty container. The container has a small amount of metadata about the container but no real Linux content:
[cloud_user@3965c7b6ce1c buildah]$ newcontainer=$(buildah from scratch)
We can see this new empty container by running buildah containers:
[cloud_user@3965c7b6ce1c buildah]$ buildah containers CONTAINER ID BUILDER IMAGE ID IMAGE NAME CONTAINER NAME 2236fca85b74 * scratch working-container
However, if we run buildah images, we would not see it listed:
[cloud_user@3965c7b6ce1c buildah]$ buildah images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/buildah-hello 1 1e452678c915 4 hours ago 1.46 MB docker.io/library/busybox latest a9d583973f65 2 weeks ago 1.45 MB
This is fine as it just indicates that there is no real image yet. i.e. It is containers/storage but there is no representation in containers/image. It has nothing in it yet. It is essentially an empty layer on top of the kernel. So how do we modify it? We need to first mount it on our filesystem and then enter the mount namespace by executing the buildah unshare command (for this we need to work as root user):
[root@3965c7b6ce1c ~]# scratchmnt=$(buildah mount working-container) [root@3965c7b6ce1c ~]# echo $scratchmnt /var/lib/containers/storage/overlay/e3f02c79964083eccc68ab49476a2e26abc09836018c90dee5a5648f71b423c8/merged
Note that our image layer is an overlay on top of /var/lib/containers. Now we can initialize an RPM database within the scratch image and add the redhat-release package:
[root@3965c7b6ce1c ~]# yum install -y --releasever=8 --installroot=$scratchmnt redhat-release Red Hat Update Infrastructure 3 Client Configuration Server 8 23 kB/s | 3.8 kB 00:00 Red Hat Enterprise Linux 8 for x86_64 - AppStream from RHUI (RPMs) 62 MB/s | 26 MB 00:00 Red Hat Enterprise Linux 8 for x86_64 - BaseOS from RHUI (RPMs) 77 MB/s | 28 MB 00:00 Last metadata expiration check: 0:00:01 ago on Mon 29 Mar 2021 03:28:01 PM UTC. Detection of Platform Module failed: No valid Platform ID detected ... ... Running transaction Preparing : 1/1 Installing : redhat-release-eula-8.3-1.0.el8.x86_64 1/2 Installing : redhat-release-8.3-1.0.el8.x86_64 2/2 Verifying : redhat-release-8.3-1.0.el8.x86_64 1/2 Verifying : redhat-release-eula-8.3-1.0.el8.x86_64 2/2 Installed: redhat-release-8.3-1.0.el8.x86_64 redhat-release-eula-8.3-1.0.el8.x86_64 Complete!
Now we can install the httpd service to the scratch directory:
[root@3965c7b6ce1c ~]# yum install -y --setopt=reposdir=/etc/yum.repos.d \ > --installroot=$scratchmnt \ > --setopt=cachedir=/var/cache/dnf httpd Last metadata expiration check: 2:04:04 ago on Mon 29 Mar 2021 01:25:40 PM UTC. Dependencies resolved. ======================================================================================================================================================================== Package Architecture Version Repository Size ======================================================================================================================================================================== Installing: httpd x86_64 2.4.37-30.module+el8.3.0+7001+0766b9e7 rhel-8-appstream-rhui-rpms 1.4 M Installing dependencies: acl x86_64 2.2.53-1.el8 rhel-8-baseos-rhui-rpms 81 k apr x86_64 1.6.3-11.el8 rhel-8-appstream-rhui-rpms 125 k apr-util x86_64 1.6.1-6.el8 rhel-8-appstream-rhui-rpms 105 k audit-libs x86_64 3.0-0.17.20191104git1c2f876.el8 rhel-8-baseos-rhui-rpms 116 k basesystem noarch 11-5.el8 rhel-8-baseos-rhui-rpms 11 k bash x86_64 4.4.19-12.el8 rhel-8-baseos-rhui-rpms 1.5 M ... ... util-linux-2.32.1-24.el8.x86_64 which-2.21-12.el8.x86_64 xkeyboard-config-2.28-1.el8.noarch xz-5.2.4-3.el8.x86_64 xz-libs-5.2.4-3.el8.x86_64 zlib-1.2.11-16.el8_2.x86_64 Complete!
Finally, we can add some httpd specific configuration to the image and save the container with buildah commit:
[root@3965c7b6ce1c ~]# echo "Built httpd image from scratch..." > $scratchmnt/var/www/html/index.html [root@3965c7b6ce1c ~]# buildah config --cmd "/usr/sbin/httpd -DFOREGROUND" working-container [root@3965c7b6ce1c ~]# buildah config --port 80/tcp working-container [root@3965c7b6ce1c ~]# buildah commit working-container localhost/buildah-httpd:1 Getting image source signatures Copying blob 0c30c4933dd5 done Copying config 52427f76ae done Writing manifest to image destination Storing signatures 52427f76aec07d575be4eea374340b786f211ae58f8ccfc7d6730cf0a0abe558 [root@3965c7b6ce1c ~]# [root@3965c7b6ce1c ~]# buildah images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/buildah-httpd 1 52427f76aec0 About a minute ago 637 MB
As we can see above, buildah images list the image localhost/buildah-httpd:1 we just created from scratch. We can now use podman to create a container based on this image and then see it in action:
[root@3965c7b6ce1c ~]# podman run -d -p 8080:80 localhost/buildah-httpd:1 51334e0119e266ae53f3ec4a425d897be02e846e3a186c511d7cd6f6866f4769 [root@3965c7b6ce1c ~]# curl http://localhost:8080 Built httpd image from scratch...
Later when we want to create a new container or containers from this image, we simply need need to do buildah from localhost/buildah-httpd:1. This will create a new containers based on this image for you.
Removing images or containers
To remove the scratch container called working-container, we can run below commands:
# buildah rm $newcontainer Or # buildah rm working-container
To remove the image, we can use buildah rmi. However, do note that like other container engines, it will need you to first remove all referencing containers to image you are removing:
# list images present
[cloud_user@3965c7b6ce1c ~]$ buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/buildah-hello 1 1e452678c915 8 hours ago 1.46 MB
docker.io/library/busybox latest a9d583973f65 2 weeks ago 1.45 MB
# lets try to remove buildah-hello image
[cloud_user@3965c7b6ce1c ~]$ buildah rmi localhost/buildah-hello:1
Could not remove image "localhost/buildah-hello:1" (must force) - container "3c7c2586f5f87e43772d0442b628880b3d7bd2efc24be7427bffb6a39a5f529d" is using its reference image: image is in use by a container
ERRO exit status 125
# verify that we have containers referencing this image
[cloud_user@3965c7b6ce1c ~]$ podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3c7c2586f5f8 localhost/buildah-hello:1 8 hours ago Exited (0) 8 hours ago charming_herschel
# remove containers
[cloud_user@3965c7b6ce1c ~]$ podman rm charming_herschel
3c7c2586f5f87e43772d0442b628880b3d7bd2efc24be7427bffb6a39a5f529d
# remove the image with buildah
[cloud_user@3965c7b6ce1c ~]$ buildah rmi localhost/buildah-hello:1
untagged: localhost/buildah-hello:1
1e452678c9154629c2a63bc512895666f070b979ea45279eb8ce2c8c4c588d64
# verify that image is removed
[cloud_user@3965c7b6ce1c ~]$ buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/library/busybox latest a9d583973f65 2 weeks ago 1.45 MB
Push and Pull Images from Container Registries
We can also use buildah to push and pull container images from private or public container registries. For the purpose of this blog post, we are connected to local registry on our machine. If you want to setup a local registry, see steps here. Let’s review the current state of our registry:
[root@3965c7b6ce1c ~]# curl -k -u user:password https://localhost:5000/v2/_catalog
{"repositories":[]}
We can see that we do not have any images in our local registry as of now. Lets try to push image localhost/buildah-hello:1 to it. Do note that by default, buildah is set up to expect secure connections to a registry. Since we are using self-signed cert for local instance of registry, we need to tell buildah to turn the TLS verification off using the --tls-verify flag. e also need to tell Buildah that the registry is on this local host ( i.e. localhost) and listening on port 5000. Similar to what you’d expect to do on multi-tenant Docker hub, we will explicitly specify that the registry is to store the image under the demo repository – so as not to clash with other users’ similarly named images:
# push image to registry
[cloud_user@3965c7b6ce1c privreg]$ buildah push --tls-verify=false --creds user:password buildah-hello:1 docker://localhost:5000/demo/buildah-hello:1
Getting image source signatures
Copying blob 81def7be1c69 done
Copying blob 2983725f2649 done
Copying config b91cc779e9 done
Writing manifest to image destination
Storing signatures
# verify the images present in the registry
[cloud_user@3965c7b6ce1c privreg]$ curl -k -u user:password https://localhost:5000/v2/_catalog{"repositories":["demo/buildah-hello"]}
We can use buildah to pull down the image using the buildah from command. But before that, we need to clean up our local image so that we do not have an existing localhost/buildah-hello:1 – otherwise buildah will know it already exists and not bother pulling it down:
# verify the images already present
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost/buildah-hello 1 b91cc779e908 36 hours ago 1.46 MB
docker.io/library/registry latest eefcac9e3856 5 days ago 26.8 MB
docker.io/library/busybox latest a9d583973f65 3 weeks ago 1.45 MB
# remove the image localhost/buildah-hello:1
[cloud_user@3965c7b6ce1c privreg]$ buildah rmi localhost/buildah-hello:1
untagged: localhost/buildah-hello:1
b91cc779e90807a373c558a4e29e3311abf356947b511a1b2e31097c2e6825b0
# verify that image is removed
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/library/registry latest eefcac9e3856 5 days ago 26.8 MB
docker.io/library/busybox latest a9d583973f65 3 weeks ago 1.45 MB
# pull image from container repository
[cloud_user@3965c7b6ce1c privreg]$ buildah from --tls-verify=false --creds user:password docker://localhost:5000/demo/buildah-hello:1Getting image source signatures
Copying blob 62bad172f512 skipped: already exists
Copying blob 26f86aeede30 done
Copying config b91cc779e9 done
Writing manifest to image destination
Storing signatures
localhost-working-container
# verify that image is present now
[cloud_user@3965c7b6ce1c privreg]$ buildah images
REPOSITORY TAG IMAGE ID CREATED SIZE
localhost:5000/demo/buildah-hello 1 b91cc779e908 36 hours ago 1.46 MB
docker.io/library/registry latest eefcac9e3856 5 days ago 26.8 MB
docker.io/library/busybox latest a9d583973f65 3 weeks ago 1.45 MB
mohitgoyal.co 