Server rejected the 1 private key(s) for jenkins (credentialId:jenkins/method:publickey)

Hi, I am getting the error Server rejected the 1 private key(s) for jenkins (credentialId:jenkins/method:publickey) When I attempt to connect an agent. The agent and Jenkins are hosted on the same machine. I followed this guide for creating the agent in a docker container. Jenkins is not in a docker container. I used the ssh-keygen command to make the key, both as RSA and as ED25519. The command to create the docker:

docker run -d --rm --name=agent1 -p 4444:22 \
-e "[PUBLIC KEY]" \
jenkins/ssh-agent:alpine-jdk17

I provided the SSH credentials with the username jenkins. I used the /home/ec2-user as I am hosting all of this on AWS EC2. I used host localhost. I have tried opening port 22 and port 4444. I am using port 4444 as I am SSHing into the instance on port 22 myself. I did find a stack overflow post for the error, but to no avail. Any ideas on what may be the issue?