As we discussed in previous posts, that Docker team re-factored, extracted and donated containerd as container runtime to the CNCF in Mar’2017. As of docker engine 1.11, docker engine switch to containerd as default runtime as well and it has been there since. containerd implements downloading images, managing them, and running containers from images. When it needs to run a container it unpacks the image into an OCI runtime bundle and shells out to runc to run it. Also, containerd is platform agnostic, so you can run it on pretty much any OS.
containerd Plugins
Initially, when containerd was started, it had single purpose – to manage and run containers. However, over the time, more and more functionality was added such as downloading images, networking interfaces and management etc. One of the reasons for adding more functionality is to make it easier to use in other projects. For example, in projects like Kubernetes, it was beneficial for containerd to do additional things like push and pull images. However, all the extra functionality is modular and optional and is available in the form of plugins such as cri, namespaces, cgroups, images, tasks, etc, meaning you can pick and choose which bits you want. So, it’s possible to include containerd in projects such as Kubernetes, but only to take the pieces your project needs.
To see all the plugins containerd has, use ctr plugins list. Here are some of the typical plugins which are generally available:
cloud_user@3b36d84d6d1c:~$ sudo ctr plugins list [sudo] password for cloud_user: TYPE ID PLATFORMS STATUS io.containerd.content.v1 content - ok io.containerd.snapshotter.v1 aufs linux/amd64 ok io.containerd.snapshotter.v1 btrfs linux/amd64 error io.containerd.snapshotter.v1 devmapper linux/amd64 error io.containerd.snapshotter.v1 native linux/amd64 ok io.containerd.snapshotter.v1 overlayfs linux/amd64 ok io.containerd.snapshotter.v1 zfs linux/amd64 error io.containerd.metadata.v1 bolt - ok io.containerd.differ.v1 walking linux/amd64 ok io.containerd.gc.v1 scheduler - ok io.containerd.service.v1 introspection-service - ok io.containerd.service.v1 containers-service - ok io.containerd.service.v1 content-service - ok io.containerd.service.v1 diff-service - ok io.containerd.service.v1 images-service - ok io.containerd.service.v1 leases-service - ok io.containerd.service.v1 namespaces-service - ok io.containerd.service.v1 snapshots-service - ok io.containerd.runtime.v1 linux linux/amd64 ok io.containerd.runtime.v2 task linux/amd64 ok io.containerd.monitor.v1 cgroups linux/amd64 ok io.containerd.service.v1 tasks-service - ok io.containerd.internal.v1 restart - ok io.containerd.grpc.v1 containers - ok io.containerd.grpc.v1 content - ok io.containerd.grpc.v1 diff - ok io.containerd.grpc.v1 events - ok io.containerd.grpc.v1 healthcheck - ok io.containerd.grpc.v1 images - ok io.containerd.grpc.v1 leases - ok io.containerd.grpc.v1 namespaces - ok io.containerd.internal.v1 opt - ok io.containerd.grpc.v1 snapshots - ok io.containerd.grpc.v1 tasks - ok io.containerd.grpc.v1 version - ok cloud_user@3b36d84d6d1c:~$
You can see more information about a particular plugin’s load status using option -d, where -d is for detailed information:
cloud_user@3b36d84d6d1c:~$ sudo ctr plugins list -d id==namespaces id==btrfs
Type: io.containerd.snapshotter.v1
ID: btrfs
Platforms: linux/amd64
Exports:
root /var/lib/containerd/io.containerd.snapshotter.v1.btrfs
Error:
Code: Unknown
Message: path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin
Type: io.containerd.grpc.v1
ID: namespaces
Requires:
io.containerd.service.v1
More information on plugins is available at https://github.com/containerd/containerd/blob/master/docs/PLUGINS.md.
containerd config
The configuration for containerd can be seen using containerd config command. Its custom configuration is saved in the /etc/containerd/config.toml file and can be customized using the same. Below is the default configuration and it contains some of the settings worth pondering over.
cloud_user@3b36d84d6d1c:~$ containerd config
NAME:
containerd config - information on the containerd config
USAGE:
containerd config command [command options] [arguments...]
COMMANDS:
default see the output of the default config
dump see the output of the final main config with imported in subconfig files
OPTIONS:
--help, -h show help
cloud_user@3b36d84d6d1c:~$ containerd config default
version = 2
root = "/var/lib/containerd"
state = "/run/containerd"
plugin_dir = ""
disabled_plugins = []
required_plugins = []
oom_score = 0
[grpc]
address = "/run/containerd/containerd.sock"
tcp_address = ""
tcp_tls_cert = ""
tcp_tls_key = ""
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[ttrpc]
address = ""
uid = 0
gid = 0
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[timeouts]
"io.containerd.timeout.shim.cleanup" = "5s"
"io.containerd.timeout.shim.load" = "5s"
"io.containerd.timeout.shim.shutdown" = "3s"
"io.containerd.timeout.task.state" = "2s"
[plugins]
[plugins."io.containerd.gc.v1.scheduler"]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
[plugins."io.containerd.grpc.v1.cri"]
disable_tcp_service = true
stream_server_address = "127.0.0.1"
stream_server_port = "0"
stream_idle_timeout = "4h0m0s"
enable_selinux = false
selinux_category_range = 1024
sandbox_image = "k8s.gcr.io/pause:3.2"
stats_collect_period = 10
systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
disable_cgroup = false
disable_apparmor = false
restrict_oom_score_adj = false
max_concurrent_downloads = 3
disable_proc_mount = false
unset_seccomp_profile = ""
tolerate_missing_hugetlb_controller = true
disable_hugetlb_controller = true
ignore_image_defined_volumes = false
[plugins."io.containerd.grpc.v1.cri".containerd]
snapshotter = "overlayfs"
default_runtime_name = "runc"
no_pivot = false
disable_snapshot_annotations = true
discard_unpacked_layers = false
[plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
runtime_engine = ""
runtime_root = ""
privileged_without_host_devices = false
base_runtime_spec = ""
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
[plugins."io.containerd.grpc.v1.cri".cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
max_conf_num = 1
conf_template = ""
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins."io.containerd.grpc.v1.cri".image_decryption]
key_model = ""
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins."io.containerd.internal.v1.opt"]
path = "/opt/containerd"
[plugins."io.containerd.internal.v1.restart"]
interval = "10s"
[plugins."io.containerd.metadata.v1.bolt"]
content_sharing_policy = "shared"
[plugins."io.containerd.monitor.v1.cgroups"]
no_prometheus = false
[plugins."io.containerd.runtime.v1.linux"]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins."io.containerd.runtime.v2.task"]
platforms = ["linux/amd64"]
[plugins."io.containerd.service.v1.diff-service"]
default = ["walking"]
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = ""
pool_name = ""
base_image_size = ""
async_remove = false
cloud_user@3b36d84d6d1c:~$
Linux Runtime’s Plugin
If we carefully go through above output, we can see the following configuration for linux runtime plugin:
[plugins."io.containerd.runtime.v1.linux"]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
Here we can see some of the important configuration like which runtime is spin up to create containers, which is runc in this case. We can also see the shim used to attach with the runc, which is containerd-shim in this case.
containerd’s Role in Container Ecosystem
Here’s one of the old reports but its still significant and upholds today. As we can see that containerd is the primary component to deal with the OCI images. Other container engines such as docker, docker-swarm, AKS, EKS, Mesos, DC/OS etc depends on it.
Container Runtime Interface or CRI
cri is a containerd plugin implementation of the Kubernetes container runtime interface (CRI). With it, we are able to use containerd as the container runtime for a Kubernetes cluster:
Since containerd 1.1, the cri plugin is built into the release binaries and enabled by default. There are many other Kubernetes CRI compatible runtimes available such as rktlet, frakti, virtlet, cri-o, dockershim, etc.
containerd CLI
containerd also provides an API and client application that can be used to interact with it. The containerd command line client is ctr. We can view the options available using ctr --help:
cloud_user@3b36d84d6d1c:~$ ctr --help
NAME:
ctr -
__
_____/ /______
/ ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/
containerd CLI
USAGE:
ctr [global options] command [command options] [arguments...]
VERSION:
1.4.3
DESCRIPTION:
ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.
COMMANDS:
plugins, plugin provides information about containerd plugins
version print the client and server versions
containers, c, container manage containers
content manage content
events, event display containerd events
images, image, i manage images
leases manage leases
namespaces, namespace, ns manage namespaces
pprof provide golang pprof outputs for containerd
run run a container
snapshots, snapshot manage snapshots
tasks, t, task manage tasks
install install a new package
oci OCI tools
shim interact with a shim directly
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--debug enable debug output in logs
--address value, -a value address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
--timeout value total timeout for ctr commands (default: 0s)
--connect-timeout value timeout for connecting to containerd (default: 0s)
--namespace value, -n value namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
--help, -h show help
--version, -v print the version
For example, we can pull images using containerd with sudo ctr images pull <full image path>. Most of these commands would need to be run with root privileges:
And list images we have using sudo ctr images list:
Same set of operations can be performed with API provided by containerd. You can read more on the same with official docs. Again as we said previously, you may or may not need to know none/some/all of the details, depending on your job role, interests and time available at hand.
