Hello everyone 
This issue began with 2.332.X LTS version of jenkins.war which is deployed as an application on Oracle WebLogic Server 12.2.21.4.0 version and OpenJDK 8 on PROD env.
As I mentioned before, in our PROD env is installed Oracle WebLogic Server 12.2.21.4.0 and OpenJDK 8, but also I tried this upgrade (from 2.319.X to 2.332.X or 2.346.1) on a TEST enviroment (TEST VM has the same distro and the same resources and prerequisites installed as in PROD), with WebLogic Server 14.1.1.0.0 and OpenJDK 11, and this issue persists.
- As I said before, the issue is on Production environment.
The VM resources:
~ cat /etc/os-release
NAME=“Red Hat Enterprise Linux Server”
VERSION=“7.9 (Maipo)”
ID=“rhel”
ID_LIKE=“fedora”
VARIANT=“Server”
VARIANT_ID=“server”
VERSION_ID=“7.9”
PRETTY_NAME=“Red Hat Enterprise Linux”
ANSI_COLOR=“0;31”
CPE_NAME=“cpe:/o:redhat:enterprise_linux:7.9:GA:server”
HOME_URL=“https://www.redhat.com/”
BUG_REPORT_URL=“https://bugzilla.redhat.com/”
REDHAT_BUGZILLA_PRODUCT=“Red Hat Enterprise Linux 7”
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT=“Red Hat Enterprise Linux”
REDHAT_SUPPORT_PRODUCT_VERSION=“7.9”
~ lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Model name: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
Stepping: 2
CPU MHz: 2294.608
BogoMIPS: 4589.21
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 25344K
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx hypervisor lahf_lm ssbd ibrs ibpb stibp tsc_adjust arat md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
~ free -h
total used free shared buff/cache available
Mem: 15G 7.0G 523M 39M 8.0G 8.2G
Swap: 3.9G 48M 3.8G
- Jenkins Arhitecture:
Admin Server (WebLogic Console) - 2GB Xmx
Jenkins Server - 10GB Xmx
The issue concerns ugprade procedure for Jenkins. Until jenkins.war version 2.319.X, jenkins behaviour was normal, everything worked good, all the plugins and so on. After the upgrade procedure to jenkins.war 2.332.3, jenkins behaviour was completely unexpected, such as when navigating in Jenkins UI, everything loads very slowly and all installed plugins are inoperable. This behaviour started with 2.332.X versions of jenkins.war, and is the same issue on 2.346.1.
I have attached below 3 screenshots of the weblogic process performance for these versions of jenkins.war:
-
jenkins.war 2.346.1 - jenkins deplyoment as an application on WebLogic Server 14.1.1.0.0 with OpenJDK 11, and the CPU has increased after I run 3 jobs and I navigating trough dashbord. The same behaviour is on WebLogic Server 12.2.1.4.0 and OpenJDK 8.
-
jenkins.war 2.332.4 - jenkins deplyoment as an application on WebLogic Server 14.1.1.0.0 with OpenJDK 11, and the CPU has increased after I run 3 jobs and I navigating trough dashbord. The same behaviour is on WebLogic Server 12.2.1.4.0 and OpenJDK 8.
CPU is the same as the above picture on 2.346.1 (I can attached only screenshot)
- jenkins.war 2.319.3 - jenkins deplyoment as an application on WebLogic Server 14.1.1.0.0 and OpenJDK 11. All stuff works good, all the plugins. The same behaviour is on WebLogic Server 12.2.1.4.0 and OpenJDK 8.
CPU is 1.7% in Jenkins when more users run many jobs/pipelines. (I can attached only screenshot)
2.319.X LTS version of jenkins.war is the latest that is working normally deployed as an application on Oracle WebLogic Server 12.2.1.4.0 version and OpenJDK 8 / 14.1.1.0.0 version and OpenJDK 11.
If anyone could help me with any ideas to fix this issue, please feel free to do so in the comment section.
Thank you!
Andrei R.