Hello,
An hour ago I went to check our Jenkins instance[0] and all it currently shows is "Application is not available". The health probes are all failing as they're unable to connect to the Jenkins instance.
Going through the logs, the issue lies in /var/lib/jenkins/config.xml:
2020-09-28 19:17:40 SEVERE jenkins.InitReactorRunner$1 onTaskFailed Failed Loading global config java.io.IOException: Unable to read /var/lib/jenkins/config.xml at hudson.XmlFile.unmarshal(XmlFile.java:183) at hudson.XmlFile.unmarshal(XmlFile.java:163) at jenkins.model.Jenkins.loadConfig(Jenkins.java:3147) at jenkins.model.Jenkins.access$1200(Jenkins.java:318) at jenkins.model.Jenkins$13.run(Jenkins.java:3248) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1131) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 16 at com.thoughtworks.xstream.core.util.FastStack.pop(FastStack.java:42) at com.thoughtworks.xstream.io.xml.AbstractPullReader.move(AbstractPullReader.java:125) at com.thoughtworks.xstream.io.xml.AbstractPullReader.moveDown(AbstractPullReader.java:103) at com.thoughtworks.xstream.io.xml.XppReader.<init>(XppReader.java:63) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:54) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65) at hudson.XmlFile.unmarshal(XmlFile.java:180) ... 13 more
The interesting line is:
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 16
After looking at the config.xml file, it appears to be empty, which is, undoubtedly, not good:
sh-4.2$ stat /var/lib/jenkins/config.xml File: ‘/var/lib/jenkins/config.xml’ Size: 0 Blocks: 0 IO Block: 1048576 regular empty file Device: 10008bh/1048715d Inode: 78643377 Links: 1 Access: (0644/-rw-r--r--) Uid: (1000830000/1000830000) Gid: ( 0/ root) Access: 2020-09-28 19:17:12.043473571 +0000 Modify: 2020-09-28 19:17:12.043473571 +0000 Change: 2020-09-28 19:17:12.080473108 +0000 Birth: -
I'm not sure what happened, but according to the last email I got from the Jenkins (27.9.2020 11:14 PM) it must've happened in the past 24 hours.
[0] https://jenkins-systemd.apps.ocp.ci.centos.org/
Metadata Update from @siddharthvipul1: - Issue tagged with: low-trouble, medium-gain
jenkins was redeployed (no way found to fetch all the configurations as config.xml was empty)
Metadata Update from @siddharthvipul1: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Log in to comment on this ticket.