If I am not using any new feature of Jenkins Server , should I upgrade Jenkins server frequently.
If no then when that particular upgrade need to consider?
Yes you should upgrade regularly. Even when you don’t need new features it ensures that you use a version that has no known security vulnerabilities.
Upgrading also involves updating all plugins to the latest version.
And having a recent Jenkins increases the chances to get answers from others in case you have a problem or questions.
And if you don’t need new features often, stick to the LTS release line.