LXC containers with static IP in config not working anymore after Debian Bullseye 11.3 update

Written by Claudio Kuenzler - 0 comments

Published on April 12th 2022 - Listed in LXC Systemd Linux

---

Network configurations of LXC containers can happen outside and inside the container. In most of my setups I prefer to use static addressing of the containers using their config file at /var/lib/lxc/containername/config.

But since the latest Debian Bullseye release 11.3 this has stopped working.

Creating a new Bullseye container

By using the lxc-download template (which is installed by default), a new Debian Bullseye container can be created:

root@host:~# lxc-create -n bullseye -t download -- -d debian -r bullseye -a amd64
[...]
---
You just created a Debian bullseye amd64 (20220411_05:24) container.

To enable SSH, run: apt install openssh-server
No default root or user password are set by LXC.

You could now either configure the IP address inside the container's rootfs or by using the config file. In this case I used the config file /var/lib/lxc/bullseye/config:

root@host:~# cat /var/lib/lxc/bullseye/config | grep net
lxc.net.0.type = veth
lxc.net.0.flags = up
lxc.net.0.link = virbr1
lxc.net.0.ipv4.address = 192.168.1.111/24
lxc.net.0.ipv4.gateway = 192.168.1.1

After starting the container, the container should be listed as started and show the configured static IP address - but for this newly created bullseye container the IP address does not show up:

root@host:~# lxc-start -n bullseye
root@host:~# lxc-ls -f
NAME       STATE   AUTOSTART GROUPS IPV4           IPV6 UNPRIVILEGED      
container1 RUNNING 1         -      192.168.1.110  -    false        
bullseye   RUNNING 1         -      -              -    false

Another LXC (container1) with Debian Bullseye 11.2 is working just fine, only the newly created Bullseye container with Debian 11.3 is not working.

Note: Actually while pinging the IP (192.168.1.111) the IP responded 1-2 seconds during lxc-start.

You can still attach into the container and everything seems to be running - but no IP address is configured:

root@host:~# lxc-attach -n bullseye
root@bullseye:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 54:52:00:00:12:38 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::5652:ff:fe00:1238/64 scope link
       valid_lft forever preferred_lft forever

Change (regression?) in Systemd

Taking a look at the Debian 11.3 release notes shows the following change in the systemd package:

     Fix uncontrolled recursion in systemd-tmpfiles [CVE-2021-3997]; demote systemd-timesyncd from Depends to Recommends, removing a dependency cycle; fix failure to bind mount a directory into a container using machinectl; fix regression in udev resulting in long delays when processing partitions with the same label; fix a regression when using systemd-networkd in an unprivileged LXD container

LXD is relying on LXC in the background, so this change might have caused a major regression when configuring static IP addresses in the container's config.

A bug report (Debian #1009351) has been created to address this regression.

Taking a closer look at a Debian 11.2 and a Debian 11.3 LXC container, a difference in the Systemd units related to network can be seen:

Inside a LXC container with Debian 11.2:

 

root@112:~# cat /etc/debian_version
11.2

root@112:~# systemctl list-units|grep network
  networking.service                  loaded active exited    Raise network interfaces
  network-online.target               loaded active active    Network is Online
  network.target                      loaded active active    Network

Inside a LXC container with Debian 11.3:

root@113:~# cat /etc/debian_version
11.3

root@113:~# systemctl list-units|grep network
  systemd-networkd.service            loaded active running   Network Service
  systemd-networkd.socket             loaded active running   Network Service Netlink Socket
  network.target                      loaded active active    Network

Workaround: Disable systemd-networkd

A workaround to get the static IP address configuration outside of the container working again, is to disable the systemd-networkd systemd unit inside the container. 

root@host:~# lxc-attach -n bullseye

root@bullseye:~# systemctl list-units|grep network
  systemd-networkd.socket             loaded active listening Network Service Netlink Socket
  network.target                      loaded active active    Network

root@bullseye:~# systemctl stop systemd-networkd
Warning: Stopping systemd-networkd.service, but it can still be activated by:
  systemd-networkd.socket

root@bullseye:~# systemctl disable systemd-networkd
Removed /etc/systemd/system/sockets.target.wants/systemd-networkd.socket.

root@bullseye:~# reboot

A few seconds later, the container is now reachable on the configured static IP address and lxc-ls -f also shows the IP address in the output.

root@host:~# lxc-ls -f
NAME       STATE   AUTOSTART GROUPS IPV4           IPV6 UNPRIVILEGED      
container1 RUNNING 1         -      192.168.1.110  -    false        
bullseye   RUNNING 1         -      192.168.1.111  -    false

Similarities in Ubuntu

A similar but not the exact same issue also happens with Ubuntu containers. As Ubuntu has introduced netplan as the default network manager, this causes the same problem with missing IP address in LXC containers. A workaround here is to remove the netplan.io package as described here.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.