Prevent default checkout when checking pipeline file from SCM when two repositories are configured

When using the option to fetch the pipeline file from SCM (as opposed to having it inline in the job itself) and defining two repositories (for example to use with the Git Branch Parameter, to have two branch parameter one from each repo), Jenkins will fetch & checkout from both repositories with the last defined one being the last one to be checked out, this is sometime undesired behavior since it:

a. causes issues when the pipeline is actually in the first repo, so you have to maintain order

b. adds to build time in case for example you intend to have advanced cloning settings (such as reference repo and shallow clone to speed up the build time, which might not apply to both repos) or you need a different configuration or time at which you intend to checkout the second repo via pipeline code

I’m aware that in the pipeline itself I can control how each repo is configured but for the default checkout of the pipeline itself this isn’t enforced.
Is there any workaround for this?

I found an issue which discusses freestyle jobs which was closed as a “Won’t fix”, I’m wondering if this applies to this case as well