Unable to install any packages on jenkins-inbound-agent container using apt since the Debian 12 upgrade

Using Jenkins
,
1

We have been using the jenkins-inboud-agent:latest-jdk11 docker image with some additional packages installed in it. Since last 5 days our docker build is failing due to not being able to run apt update to install additional packages. I looked at the last update to the jenkins-inbound-agent image and looks like the Debian version was updated to Debian 12.

Here is the error I’m seeing,

nowalekar@admin-bastion-i-068e98ca8e70b7c5d:~$ docker run -it -u root jenkins/inbound-agent:latest-jdk11 bash
root@3e562058a639:/home/jenkins# apt update
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Err:1 http://deb.debian.org/debian bookworm InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY F8D2585B8783D481
Err:2 http://deb.debian.org/debian bookworm-updates InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
Err:3 http://deb.debian.org/debian-security bookworm-security InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54404762BBB6E853 NO_PUBKEY BDE6D2B9216EC7A8
Reading package lists... Done
W: GPG error: http://deb.debian.org/debian bookworm InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY F8D2585B8783D481
E: The repository 'http://deb.debian.org/debian bookworm InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://deb.debian.org/debian bookworm-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY 6ED0E7B82643E131
E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://deb.debian.org/debian-security bookworm-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54404762BBB6E853 NO_PUBKEY BDE6D2B9216EC7A8
E: The repository 'http://deb.debian.org/debian-security bookworm-security InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true'
E: Sub-process returned an error code

Also, I tried to add the missing keys manually. But they require gnupg package to be installed.

root@3e562058a639:/home/jenkins# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 0E98404D386FA1D9
E: gnupg, gnupg2 and gnupg1 do not seem to be installed, but one of them is required for this operation

Please help resolve this :pray:

2

I can’t duplicate the issue that you’re reporting. When I run those commands from my Ubuntu 22.04 computer with Docker version 24.0.6, it reports:

$ docker run -it -u root jenkins/inbound-agent:latest-jdk11 bash
root@10b286c2475c:/home/jenkins# apt update
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [52.1 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8906 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [6432 B]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [63.2 kB]
Fetched 9227 kB in 2s (5975 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
root@10b286c2475c:/home/jenkins# exit

Maybe you have something that is acting as a barrier between your container runtime and the public internet?

Maybe your container runtime is not Docker and is refusing to allow the operation as root?

Maybe your container runtime is outdated and needs to be updated to a recent release?

3

Hi Mark,

Thanks for your response. I did some more investigation and observed that only the debian:12 image has this issue, that too only in our AWS infrastructure. I was able to run apt update in the debian:12 image on my local computer.
Also, the debian:11 or debian:10 images work perfectly fine on both my local and the AWS infrastructure.

root@prod-ecs-i-0376cc9a6ba6479c3:~# docker run -it -u root jenkins/inbound-agent:latest-jdk11 bash
root@9ff3e53cd2f0:/home/jenkins# curl www.google.com
curl: (6) getaddrinfo() thread failed to start

I found a similar problem reported by someone, and one of the possible solutions was to use this flag in the docker run command. --security-opt seccomp=unconfined
It worked!
So I believe there is some syscall that debian:12 is using which is blocked by default in the docker-ce version we are running. And it isn’t blocked by the docker for desktop I’m running locally.

This is not at all a Jenkins related issue, but a debian:12 or Docker related one.
I hope someone will find this information useful.