Strong architectural thinking, Pratham! The agent+ gateway pattern makes sense
for large-scale K8s environments.
One scoping question for mentors: Given the project goal is “to help enhance
observability of Jenkins jobs on ci.jenkins.io” within a 175-hour medium project,
should we prioritize:
a) Full multi-tier architecture (agent DaemonSets+ gateway+ HPA/KEDA) from day one
b) Simpler gateway pattern first, with agent layer as future scaling enhancement
In my PoC, the gateway-only approach achieved 80%+ data reduction with tail-based sampling while keeping deployment straightforward.
Would be worth hearing from the infrastructure team on their Kubernetes setup
and whether node-level collectors are needed from day one.