I split this off of Java 11 as default in Docker images - #2 as its not really about the change to java 11 inside agent images.
I am having a hard time finding all the proper configuration settings.
What do you mean by proper? Is something not working that you want fixed?
Maybe @dduportal can share what they did for jenkins infra and windows vms (though I think its mostly via azure)