使用nerdctl替代docker-ce

nerdctl是一个兼容Docker CLI的命令行工具, 用于管理containerd容器.

nerdctl

使用rootfull模式.

mkdir nerdctl
wget -c https://github.com/containerd/nerdctl/releases/download/v1.7.7/nerdctl-1.7.7-linux-amd64.tar.gz
tar -xzvf nerdctl-1.7.7-linux-amd64.tar.gz -C nerdctl
sudo cp -a nerdctl/nerdctl /usr/local/bin/

配置别名docker命令, 指向nerdctl.

# sudo vim /usr/local/bin/docker


#!/bin/bash


COMMAND="nerdctl"
if [[ $EUID -ne 0 ]]; then
    sudo $COMMAND "$@"
else
    $COMMAND "$@"
fi

containerd

复用k3s的containerd, 配置nerdctl.toml指向containerd sock.

mkdir /etc/nerdctl


cat << EOF > /etc/nerdctl/nerdctl.toml
debug          = false
debug_full     = false
address        = "unix:///run/k3s/containerd/containerd.sock"
namespace      = "k8s.io"
cni_path       = "/var/lib/nerdctl/cni/bin"
cni_netconfpath = "/var/lib/nerdctl/cni/net.d"
EOF

docker ps/exec/image 相关命令就可以使用了, 单纯管理k8s/k3s配置到这里就可以了.

如果需要完整的docker体验，docker run/build支持, 需要添加cni pulgin和buildkit支持.

cni plugin

使用docker run 创建启动容器, 需要配置cni plugin, 让nerdctl在创建容器的使用告诉containerd调用哪里的cni配置容器网络.

上面的nerdctl.toml已经配置了路径, 需要安装cni plugin到对应路径.

cni_path       = "/var/lib/nerdctl/cni/bin"
cni_netconfpath = "/var/lib/nerdctl/cni/net.d"

mkdir cni
wget -c https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz
tar -xzvf cni-plugins-linux-amd64-v1.5.1.tgz -C cni
sudo cp cni/* /var/lib/nerdctl/cni/bin/

# docker run --rm -ti alpine sh
# docker network ls


NETWORK ID      NAME      FILE
17f29b073143    bridge    /var/lib/nerdctl/cni/net.d/nerdctl-bridge.conflist
                host
                none

会发现不做配置, nerdctl在执行docker run的时候回创建默认的网桥网络.

# ifconfig
nerdctl0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.4.0.1  netmask 255.255.255.0  broadcast 10.4.0.255
        inet6 fe80::a096:c4ff:fe7e:a72f  prefixlen 64  scopeid 0x20<link>
        ether a2:96:c4:7e:a7:2f  txqueuelen 1000  (Ethernet)
        RX packets 15  bytes 1000 (1000.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13  bytes 1494 (1.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

主机查看网络, 多了一个nerdctl0网桥, 和docker的docker0体验是一样的.
注意这里在配置nerdctl.toml的时候故意将cni路径配置到了/var/lib/nerdctl, 因为默认会使用/opt/cni/bin和/etc/cni/net.d, 如果主机安装的k8s使用相同路径, 可能产生影响.

buildkit

安装buildctl和buildkitd.

mkdir buildkit
wget -c https://github.com/moby/buildkit/releases/download/v0.15.2/buildkit-v0.15.2.linux-amd64.tar.gz
tar -xzvf buildkit-v0.15.2.linux-amd64.tar.gz -C buildkit
sudo cp buildkit/bin/buildctl /usr/local/bin/
sudo cp buildkit/bin/buildkitd /usr/local/bin

配置buildkitd的worker使用k3s的containerd.

# mkdir /etc/buildkit
# vim /etc/buildkit/buildkitd.toml


[worker.oci]
  enabled = false


[worker.containerd]
  enabled = true
  namespace = "k8s.io"
  address = "/run/k3s/containerd/containerd.sock"

配置buildkitd守护启动.

# vim /etc/systemd/system/buildkit.service
# systemctl start buildkit.service
# systemctl enable buildkit.service


[Unit]



After=network.target local-fs.target


[Service]
#uncomment to enable the experimental sbservice (sandboxed) version of containerd/cri integration
#Environment="ENABLE_CRI_SANDBOXES=sandboxed"
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/buildkitd


Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999


[Install]
WantedBy=multi-user.target

nerdctl的rootfull模式和buildkitd通信会使用/run/buildkit/buildkitd.sock.

multi-platform

添加docker run/build/image的跨平台支持, 首先要再主机上注册binfmt_misc使用qemu, 可以使用这个项目tonistiigi/binfmt.

主机在遇到binfmt是其他平台的指令时候, 会自动调用qemu(容器进程也是主机进程, 所以也可以).

# docker run --privileged --rm tonistiigi/binfmt:master --install all


# ls -1 /proc/sys/fs/binfmt_misc/qemu*
/proc/sys/fs/binfmt_misc/qemu-aarch64
/proc/sys/fs/binfmt_misc/qemu-arm
/proc/sys/fs/binfmt_misc/qemu-mips64
/proc/sys/fs/binfmt_misc/qemu-mips64el
/proc/sys/fs/binfmt_misc/qemu-ppc64le
/proc/sys/fs/binfmt_misc/qemu-riscv64
/proc/sys/fs/binfmt_misc/qemu-s390x


# docker run --rm --platform=arm64 alpine uname -a


Linux 20910835f3ab 4.18.0-553.16.1.el8_10.x86_64 #1 SMP Thu Aug 8 17:47:08 UTC 2024 aarch64 Linux

summary

以上组合nerdctl/containerd/cni-plugin/buildkit, 实现docker-ce的体验. 在k8s/k3s开发调试场景, 共用k8s的containerd, 还是会很方便的.

文章目录