The Windows agent will have a file agent.jar installed on it somewhere. A new copy of that file needs to be downloaded from the controller page that is used to configure the agent. Replace the old copy of agent.jar with the new copy and restart the agent.
As an alternative that would avoid the need to download that update yourself, you could switch from the inbound agent configuration you are running now to instead use an ssh (outbound) agent. Rough instructions are available in Launch Windows agents with OpenSSH.