Do you have access to the repo via http? Then you could use the http_request plugin in combination with the readYaml step.
That would at least avoid that you have to clone the repo just to read a single file. You would still need your prepare stage in each pipeline but the approach doesn’t require a workspace.