This blog post continues from where we left in our earlier blog post, where we discussed how systemd and podman fits in together to run and manage containers as systemd services. We discussed how we can do the same for a specific containers and learned to create generic systemd unit files. We also discussed few use cases where this integration is useful. As we know, pods in podman are a way to group and manage multiple application containers as one. So we can start them together, manage them together and then remove them together once done. If we do need, we can manage them individually as well. Basics of pods are covered here.
However, when we have multiple containers, we often want to start them in a specific order. For example, one of the containers may be app container and another is db container, so you would want to start first the db container and then only start the app container. You may also want the rule to not start app container at all if db container fails to come up. Lets see how we can do that with a wordpress and mariadb container as an example.
Create a Pod with Multiple Containers
Lets first create an empty pod named as wpapp_pod and verify it comes up fine:
[cloud_user@96dd25ebf51c ~]$ podman pod create -p 8080:80 --name wpapp_pod bdd858cbdfcd7de51a2e0097a46ee10bc983a5c4d6edfb725cea8bce2e5a2263 [cloud_user@96dd25ebf51c ~]$ [cloud_user@96dd25ebf51c ~]$ podman pod list POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS bdd858cbdfcd wpapp_pod Created 6 seconds ago cb3d1c6f32d1 1 [cloud_user@96dd25ebf51c ~]$ podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES cb3d1c6f32d1 k8s.gcr.io/pause:3.2 10 seconds ago Created 0.0.0.0:8080->80/tcp bdd858cbdfcd-infra
Now we’ll add mariadb container and wordpress container to above pod:
[cloud_user@96dd25ebf51c ~]$ podman run \ > -d --restart=always --pod wpapp_pod \ > -e MYSQL_ROOT_PASSWORD="myrootpass" \ > -e MYSQL_DATABASE="wp-db" \ > -e MYSQL_USER="wp-user" \ > -e MYSQL_PASSWORD="w0rdpr3ss" \ > --name=wptest-db mariadb Completed short name "mariadb" with unqualified-search registries (origin: /etc/containers/registries.conf) Trying to pull registry.access.redhat.com/mariadb:latest... name unknown: Repo not found Trying to pull registry.redhat.io/mariadb:latest... unable to retrieve auth token: invalid username/password: unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication Trying to pull docker.io/library/mariadb:latest... Getting image source signatures Copying blob a70d879fa598 done Copying blob 10e6159c56c0 done Copying blob 75bac9accf15 done Copying blob 6ee25b525863 done Copying blob 5553d41e82b1 done Copying blob c4394a92d1f8 done Copying blob 0df5d11e9502 done Copying blob b61f99a83efe done Copying blob 6fd69eaf8690 done Copying blob 03f7bd4f1dda done Copying blob 6ce5fa137d06 done Copying blob 689a8c518900 done Copying config e76a4b2ed1 done Writing manifest to image destination Storing signatures 8e4813b3218fe8a40f4262e3cfcd2ad646b94fa4e66a01898fe2b6bea074f941 [cloud_user@96dd25ebf51c ~]$ podman run \ > -d --restart=always --pod=wpapp_pod \ > -e WORDPRESS_DB_NAME="wp-db" \ > -e WORDPRESS_DB_USER="wp-user" \ > -e WORDPRESS_DB_PASSWORD="w0rdpr3ss" \ > -e WORDPRESS_DB_HOST="127.0.0.1" \ > --name wptest-web wordpress Completed short name "wordpress" with unqualified-search registries (origin: /etc/containers/registries.conf) Trying to pull registry.access.redhat.com/wordpress:latest... name unknown: Repo not found Trying to pull registry.redhat.io/wordpress:latest... unable to retrieve auth token: invalid username/password: unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication Trying to pull docker.io/library/wordpress:latest... Getting image source signatures Copying blob 6da3e75ee2ca done Copying blob 038e5b090752 done Copying blob 56671971dcc6 done Copying blob 854fb08fe050 done Copying blob 75646c2fb410 done Copying blob d099f6707d86 done Copying blob f33a657f956e done Copying blob 249520ff71af done Copying blob 88fd46807e1d done Copying blob 4213c3e42364 done Copying blob 4915809df15f done Copying blob 2faa4b167ab4 done Copying blob 78435232ad8f done Copying blob 662883b7bb15 done Copying blob bf62eea5448f done Copying blob 92a1afd88c46 done Copying blob e0f9cda83bc3 done Copying blob a01ecf9f410a done Copying blob 608ccbf945cb done Copying blob fd4a2a57c3c7 done Copying blob 0ca288048117 done Copying config bfcb597091 done Writing manifest to image destination Storing signatures 41f21efffcf4d2a6839f9a1ac519f20f248bcf2407b1a63e63bbb70604c0ed4b
Verify that all 3 containers are up and running now:
[cloud_user@96dd25ebf51c ~]$ podman pod list POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS bdd858cbdfcd wpapp_pod Running 6 minutes ago cb3d1c6f32d1 3 [cloud_user@96dd25ebf51c ~]$ podman ps -a --pod CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES POD ID PODNAME 41f21efffcf4 docker.io/library/wordpress:latest apache2-foregroun... 4 minutes ago Up 4 minutes ago 0.0.0.0:8080->80/tcp wptest-web bdd858cbdfcd wpapp_pod 8e4813b3218f docker.io/library/mariadb:latest mysqld 5 minutes ago Up 5 minutes ago 0.0.0.0:8080->80/tcp wptest-db bdd858cbdfcd wpapp_pod cb3d1c6f32d1 k8s.gcr.io/pause:3.2 6 minutes ago Up 5 minutes ago 0.0.0.0:8080->80/tcp bdd858cbdfcd-infra bdd858cbdfcd wpapp_pod
Generate and Review systemd unit files for pod
We can generate systemd unit files with podman generate systemd command. We’ll also add --files parameter to create generic files and use --new parameter to supply pod name:
[cloud_user@96dd25ebf51c ~]$ podman generate systemd --files --name wpapp_pod /home/cloud_user/container-wptest-db.service /home/cloud_user/container-wptest-web.service /home/cloud_user/pod-wpapp_pod.service
We can see that it created 3 files for 3 containers. Now lets review the systemd unit file for pod i.e. /home/cloud_user/pod-wpapp_pod.service:
[cloud_user@96dd25ebf51c ~]$ cat pod-wpapp_pod.service # pod-wpapp_pod.service # autogenerated by Podman 2.2.1 # Mon Apr 5 09:21:31 UTC 2021 [Unit] Description=Podman pod-wpapp_pod.service Documentation=man:podman-generate-systemd(1) Wants=network.target After=network-online.target Requires=container-wptest-db.service container-wptest-web.service Before=container-wptest-db.service container-wptest-web.service [Service] Environment=PODMAN_SYSTEMD_UNIT=%n Restart=on-failure ExecStart=/usr/bin/podman start bdd858cbdfcd-infra ExecStop=/usr/bin/podman stop -t 10 bdd858cbdfcd-infra ExecStopPost=/usr/bin/podman stop -t 10 bdd858cbdfcd-infra PIDFile=/run/user/1001/containers/overlay-containers/cb3d1c6f32d1f8eab036a35e5a4030515654934fc957572bc618fff65d0bbcf5/userdata/conmon.pid KillMode=none Type=forking [Install] WantedBy=multi-user.target default.target
Few properties are of interest here in the unit section:
Requires=: This directive lists any units upon which this unit essentially depends. If the current unit is activated, the units listed here must successfully activate as well, else this unit will fail. These units are started in parallel with the current unit by default.Wants=: This directive is similar to Requires=, but less strict. Systemd will attempt to start any units listed here when this unit is activated. If these units are not found or fail to start, the current unit will continue to function. This is the recommended way to configure most dependency relationships. Again, this implies a parallel activation unless modified by other directives.Before=: The units listed in this directive will not be started until the current unit is marked as started if they are activated at the same time. This does not imply a dependency relationship and must be used in conjunction with one of the above directives if this is desired.After=: The units listed in this directive will be started before starting the current unit. This does not imply a dependency relationship and one must be established through the above directives if this is required.
Another interesting property is BindsTo=, which is similar to Requires=, but also causes the current unit to stop when the associated unit terminates.
Now, we’ll review and modify our unit files as per our requirements. In the pod-specific unit file, we do not want to start wptest-web and wptest-db containers in parallel. However we may want to mark it as fail if these are not running. Similarly, we want to start wptest-web after wptest-db container. Let’s modify our files and configure them as below:
[cloud_user@96dd25ebf51c ~]$ cat pod-wpapp_pod.service # pod-wpapp_pod.service # autogenerated by Podman 2.2.1 # Mon Apr 5 09:21:31 UTC 2021 [Unit] Description=Podman pod-wpapp_pod.service Documentation=man:podman-generate-systemd(1) Wants=network.target After=network-online.target BindsTo=container-wptest-db.service container-wptest-web.service Before=container-wptest-db.service container-wptest-web.service [Service] Environment=PODMAN_SYSTEMD_UNIT=%n Restart=on-failure ExecStart=/usr/bin/podman start bdd858cbdfcd-infra ExecStop=/usr/bin/podman stop -t 10 bdd858cbdfcd-infra ExecStopPost=/usr/bin/podman stop -t 10 bdd858cbdfcd-infra PIDFile=/run/user/1001/containers/overlay-containers/cb3d1c6f32d1f8eab036a35e5a4030515654934fc957572bc618fff65d0bbcf5/userdata/conmon.pid KillMode=none Type=forking [Install] WantedBy=multi-user.target default.target [cloud_user@96dd25ebf51c ~]$ cat container-wptest-db.service # container-wptest-db.service # autogenerated by Podman 2.2.1 # Mon Apr 5 09:21:31 UTC 2021 [Unit] Description=Podman container-wptest-db.service Documentation=man:podman-generate-systemd(1) Wants=network.target After=network-online.target BindsTo=pod-wpapp_pod.service After=pod-wpapp_pod.service Before=container-wptest-web.service [Service] Environment=PODMAN_SYSTEMD_UNIT=%n Restart=on-failure ExecStart=/usr/bin/podman start wptest-db ExecStop=/usr/bin/podman stop -t 10 wptest-db ExecStopPost=/usr/bin/podman stop -t 10 wptest-db PIDFile=/run/user/1001/containers/overlay-containers/8e4813b3218fe8a40f4262e3cfcd2ad646b94fa4e66a01898fe2b6bea074f941/userdata/conmon.pid KillMode=none Type=forking [Install] WantedBy=multi-user.target default.target [cloud_user@96dd25ebf51c ~]$ cat container-wptest-web.service # container-wptest-web.service # autogenerated by Podman 2.2.1 # Mon Apr 5 09:21:31 UTC 2021 [Unit] Description=Podman container-wptest-web.service Documentation=man:podman-generate-systemd(1) Wants=network.target After=network-online.target BindsTo=pod-wpapp_pod.service After=pod-wpapp_pod.service container-wptest-db.service [Service] Environment=PODMAN_SYSTEMD_UNIT=%n Restart=on-failure ExecStart=/usr/bin/podman start wptest-web ExecStop=/usr/bin/podman stop -t 10 wptest-web ExecStopPost=/usr/bin/podman stop -t 10 wptest-web PIDFile=/run/user/1001/containers/overlay-containers/41f21efffcf4d2a6839f9a1ac519f20f248bcf2407b1a63e63bbb70604c0ed4b/userdata/conmon.pid KillMode=none Type=forking [Install] WantedBy=multi-user.target default.target [cloud_user@96dd25ebf51c ~]$
Enables systemd Services for Pod
First we need to copy all the generated files to $HOME/.config/systemd/user for installing as a non-root user:
[cloud_user@96dd25ebf51c ~]$ cp container-wptest-web.service container-wptest-db.service pod-wpapp_pod.service $HOME/.config/systemd/user [cloud_user@96dd25ebf51c ~]$ ls $HOME/.config/systemd/user container-wptest-db.service container-wptest-web.service pod-wpapp_pod.service
Now we can enable systemd for current user:
[cloud_user@96dd25ebf51c ~]$ systemctl enable --user pod-wpapp_pod.service Created symlink /home/cloud_user/.config/systemd/user/multi-user.target.wants/pod-wpapp_pod.service → /home/cloud_user/.config/systemd/user/pod-wpapp_pod.service. Created symlink /home/cloud_user/.config/systemd/user/default.target.wants/pod-wpapp_pod.service → /home/cloud_user/.config/systemd/user/pod-wpapp_pod.service.
Note that we have not created any services for the dependent application containers. We can verify if service is enabled with below command:
[cloud_user@96dd25ebf51c user]$ systemctl is-enabled --user pod-wpapp_pod.service enabled
Verify with systemd in action
Before that, lets stop our pod and verify that all 3 containers are stopped:
[cloud_user@96dd25ebf51c ~]$ podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [cloud_user@96dd25ebf51c ~]$ podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 14297e417b1f docker.io/library/wordpress:latest apache2-foregroun... 10 minutes ago Exited (0) 5 minutes ago 0.0.0.0:8080->80/tcp wptest-web 62b438173484 docker.io/library/mariadb:latest mysqld 10 minutes ago Exited (0) 5 minutes ago 0.0.0.0:8080->80/tcp wptest-db cf349f0c3d97 k8s.gcr.io/pause:3.2 11 minutes ago Exited (0) 5 minutes ago 0.0.0.0:8080->80/tcp ddcb4333bec6-infra
Now we can start our application with systemctl enable --user command:
[cloud_user@96dd25ebf51c user]$ systemctl start --user pod-wpapp_pod.service
[cloud_user@96dd25ebf51c user]$ systemctl status --user pod-wpapp_pod.service
● pod-wpapp_pod.service - Podman pod-wpapp_pod.service
Loaded: loaded (/home/cloud_user/.config/systemd/user/pod-wpapp_pod.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2021-04-05 10:16:50 UTC; 8s ago
Docs: man:podman-generate-systemd(1)
Process: 25874 ExecStart=/usr/bin/podman start ddcb4333bec6-infra (code=exited, status=0/SUCCESS)
Main PID: 25917 (conmon)
CGroup: /user.slice/user-1001.slice/user@1001.service/pod-wpapp_pod.service
├─25892 /usr/bin/fuse-overlayfs -o lowerdir=/home/cloud_user/.local/share/containers/storage/overlay/l/PC6CJP3JJJJLUZCLTJBR7RUHBL,upperdir=/home/cloud_user/>
├─25897 /usr/bin/slirp4netns --disable-host-loopback --mtu 65520 --enable-sandbox --enable-seccomp -c -e 3 -r 4 --netns-type=path /run/user/1001/netns/cni-4>
├─25899 containers-rootlessport
├─25906 containers-rootlessport-child
├─25917 /usr/bin/conmon --api-version 1 -c cf349f0c3d9723ecb8e1c54c4646fbb15e117cf72b3a6bad1a4ed7f010e0b1d9 -u cf349f0c3d9723ecb8e1c54c4646fbb15e117cf72b3a6>
└─cf349f0c3d9723ecb8e1c54c4646fbb15e117cf72b3a6bad1a4ed7f010e0b1d9
└─25928 /pause
Apr 05 10:16:50 96dd25ebf51c.mylabserver.com systemd[1550]: Starting Podman pod-wpapp_pod.service...
Apr 05 10:16:50 96dd25ebf51c.mylabserver.com podman[25874]: ddcb4333bec6-infra
Apr 05 10:16:50 96dd25ebf51c.mylabserver.com systemd[1550]: Started Podman pod-wpapp_pod.service.
# verify that now our containers come up fine
[cloud_user@96dd25ebf51c user]$ podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
14297e417b1f docker.io/library/wordpress:latest apache2-foregroun... 15 minutes ago Up 23 seconds ago 0.0.0.0:8080->80/tcp wptest-web
62b438173484 docker.io/library/mariadb:latest mysqld 15 minutes ago Up 24 seconds ago 0.0.0.0:8080->80/tcp wptest-db
cf349f0c3d97 k8s.gcr.io/pause:3.2 16 minutes ago Up 25 seconds ago 0.0.0.0:8080->80/tcp ddcb4333bec6-infra
Do note that that mariadb container came up before wordpress container, as indicated by the difference in the start time status. Since we have instructed systemd to stop at user logout and start at user login, it will do so. However we cannot show it here due to obvious limits of text based posts :).
mohitgoyal.co 