Is there documentation somewhere that details what DNS records are required for using the ingress setup by the helm chart?
ingress is a pretty core k8s technique. Can you give examples of what information you’d need? Your hostname that ingress is listening to, must point to your ingress controller.
I provide this info to the jenkins helm chart helm-charts/charts/jenkins at main · jenkinsci/helm-charts · GitHub
ingress:
enabled: true
ingressClassName: nginx
paths: []
annotations:
apiVersion: "extensions/v1beta1"
hostName: ci.my.domain.com
tls:
- secretName: ci.my.domain.com
hosts:
- my-ci-jenkins
It creates
k -n my-ci-production get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-ci-jenkins ClusterIP 10.43.212.78 <none> 8080/TCP 13d
my-ci-jenkins-agent ClusterIP 10.43.36.219 <none> 50000/TCP 13d
k -n my-ci-production get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
my-ci-jenkins nginx ci.my.domain.com 10.193.61.195,10.193.61.39,10.193.62.213 80, 443 13d
Those IP addresses are my worker node addresses
My question is, what is the appropriate way to configure my non-k8s, external DNS server so users can resolve ci.my.domain.com. Right now I’m using multiple A records, one for each node IP address but I’m just guessing. I’ve mostly have used the load balancer option to the chart in the past
I provide this info to the jenkins helm chart helm-charts/charts/jenkins at main · jenkinsci/helm-charts · GitHub
ingress:
enabled: true
ingressClassName: nginx
paths: []
annotations:
apiVersion: "extensions/v1beta1"
hostName: ci.my.domain.com
tls:
- secretName: ci.my.domain.com
hosts:
- my-ci-jenkins
It creates
k -n my-ci-production get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
my-ci-jenkins ClusterIP 10.43.212.78 <none> 8080/TCP 13d
my-ci-jenkins-agent ClusterIP 10.43.36.219 <none> 50000/TCP 13d
k -n my-ci-production get ing
NAME CLASS HOSTS ADDRESS PORTS AGE
my-ci-jenkins nginx ci.my.domain.com 10.193.61.195,10.193.61.39,10.193.62.213 80, 443 13d
Those IP addresses are my worker node addresses
My question is, what is the appropriate way to configure my non-k8s, external DNS server so users can resolve ci.my.domain.com
I tried to reply several times and it said it blocked it
As in said before, you typically point things at your ingress controller’s public IP. You’ll want to grab the service for the nginx service.
It’s not really a Jenkins question and depends greatly on how you setup your cluster and ingress controller
I agree but the problem is the chart decides on how the ingress is setup with some control over the configuration so I was wondering if there was some assumption it’s making based on an expected environment. I think the answer for me is to walk their tpl code and see how it’s being put together.
Ultimately the reason I’m looking deeper into ingress and what is best practice is because my pods are going offline because of the following. I never had this issue when I was running a load balancer. It’s concerning that the chart would use node IPs in the ingress config because it seems like that would not account for nodes being swapped out.
2021-11-30 23:38:17.886+0000 [id=51116] WARNING h.r.AbstractByteBufferCommandTransport#processCommand: Failed to construct Command in channel JNLP4-connect connection from 10.42.3.115/10.42.3.115:43172
java.io.StreamCorruptedException: invalid type code: AC
at java.base/java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:3082)
at java.base/java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:3116)
at java.base/java.io.ObjectInputStream$BlockDataInputStream.skipBlockData(ObjectInputStream.java:3018)
at java.base/java.io.ObjectInputStream.skipCustomData(ObjectInputStream.java:2404)
at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2007)
at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1862)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2169)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679)
at java.base/java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2464)
at java.base/java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2358)
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2196)
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1679)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:493)
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:451)
at hudson.remoting.Command.readFromObjectStream(Command.java:155)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.AbstractByteBufferCommandTransport.processCommand(AbstractByteBufferCommandTransport.java:202)
at hudson.remoting.AbstractByteBufferCommandTransport.receive(AbstractByteBufferCommandTransport.java:189)
at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onRead(ChannelApplicationLayer.java:214)
at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecv(ApplicationLayer.java:206)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:677)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processRead(SSLEngineFilterLayer.java:367)
at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecv(SSLEngineFilterLayer.java:117)
at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecv(ProtocolStack.java:677)
at org.jenkinsci.remoting.protocol.NetworkLayer.onRead(NetworkLayer.java:136)
at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:160)
at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:790)
at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Ah found it in the remoting log. The problem is OOM
I still have no idea what you are asking for or what you are trying to suggest get changed.
Also the chart isn’t using node ips, charts don’t have runtime information. Typically in kubernetes you use a service hostname, and that hostname resolves to a node ip. Its at the k8s level, not helm chart level.
These are node addresses
my-ci-jenkins nginx ci.my.domain.com 10.193.61.195,10.193.61.39,10.193.62.213
When you deploy the chart they are blank and it looks like eventually nginx fills them in
Typically in kubernetes you use a service hostname, and that hostname resolves to a node ip.
Yes I have that. I have A records for ci.my.domain.com that resolve to each of the addresses above
Just seems kind if strange because if a node is removed or not answering and DNS round robins to that down node I’m not sure what would happen. If the client or agent’s DNS cache is too old and it resolves the service name again and DNS gives it a down node, what happens?
This is way deeper into running your own cluster then is appropriate for a jenkins forum. Your describing the differences in services between load balancer, node port, cluster IP. Also depends on if your ingress controller is a daemon set, or just a single running instance, etc.
None of this has to do with the jenkins helm chart.