Almost. If you want to stay on JDK 8 in order to reduce the size of each step of the upgrade, then you would use jenkins/jenkins:2.319.3-jdk8 as the Docker container for the first step. After that first step of the upgrade is complete, you would upgrade to Java 11 by using the jenkins/jenkins:2.319.3 container or its more precisely labeled alias jenkins/jenkins:2.319.3-jdk11.
Yes, if you don’t mind making a larger upgrade step, then you can go directly to JDK 11.
Even if you don’t need JDK 8 any longer, you should prefer to have no executors on the controller and configure agents to execute the jobs. See the article on controller isolation for the list of reasons
Yes, Jenkins allows builds to run with many different JDK versions. We use Java 8, Java 11, and Java 17 regularly on ci.jenkins.io with the controller running Java 11. There are many users that run multiple JDK versions with Jenkins jobs. It works very well.
This video includes an example that configures an Apache Maven tool to use a different JDK
