Plugin gets reinstalled and restarts Jenkins on Puppet daemon run but NOT manual run

[mjmccormac]

Setup

rtyler/jenkins 1.7.0
puppet/archive 1.1.2

Ok, so I've searched on this issue quite a bit the last couple of days and am running into what I believe is a unique situation that I'm hoping someone can help me with.

I have a list of plugins that I'm maintaining via the puppet-jenkins module; everything works fine when running puppet manually (puppet agent -t), however on the timed daemon runs puppet thinks there's a change with 2 plugins, reinstalls them, and restarts the service:

puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[gitlab-plugin]/Archive[gitlab-plugin.hpi]/ensure) replace archive: /var/lib/jenkins/plugins/gitlab-plugin.hpi from (sha1)8085eadd90eeb0adf06a833a9f636924e6abe853 to (sha1)
puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[gitlab-plugin]/File[/var/lib/jenkins/plugins/gitlab-plugin.hpi]/owner) owner changed 'root' to 'jenkins'
puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[gitlab-plugin]/File[/var/lib/jenkins/plugins/gitlab-plugin.hpi]/group) group changed 'root' to 'jenkins'
puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[groovy]/Archive[groovy.hpi]/ensure) replace archive: /var/lib/jenkins/plugins/groovy.hpi from (sha1)243329f6480354ac164abcd42db1b70f989b78f2 to (sha1)
puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[groovy]/File[/var/lib/jenkins/plugins/groovy.hpi]/owner) owner changed 'root' to 'jenkins'
puppet-agent[23599]: (/Stage[main]/Jenkins::Plugins/Jenkins::Plugin[groovy]/File[/var/lib/jenkins/plugins/groovy.hpi]/group) group changed 'root' to 'jenkins'
jenkins: jenkins: client (pid 21602) exited with 143 status
puppet-agent[23599]: (/Stage[main]/Jenkins::Service/Service[jenkins]) Triggered 'refresh' from 3 events

This led me to look at facter, and lo and behold those 2 plugins (gitlab-plugin and groovy) are missing:

ace-editor 1.1, active-directory 2.0, analysis-core 1.82, ansicolor 0.4.3, ant 1.4, antisamy-markup-formatter 1.5, branch-api 1.11.1, build-monitor-plugin 1.11+build.201701152243, build-pipeline-plugin 1.5.6, cloudbees-folder 5.16, conditional-buildstep 1.3.5, copyartifact 1.38.1, credentials 2.1.10, cucumber-testresult-plugin 0.9.7, cvs 2.13, display-url-api 0.5, durable-task 1.13, email-ext 2.54, envinject 1.93.1, extended-read-permission 1.0, extensible-choice-parameter 1.3.3, external-monitor-job 1.7, git 3.0.1, git-client 2.2.1, git-server 1.7, gradle 1.25, greenballs 1.15, handlebars 1.1.1, hipchat 2.0.0, icon-shim 2.0.3, javadoc 1.4, job-dsl 1.53, job-node-stalker 1.0.5, jquery 1.11.2-0, jquery-detached 1.2.1, junit 1.19, ldap 1.14, mailer 1.18, mapdb-api 1.0.9.0, matrix-auth 1.4, matrix-project 1.8, maven-plugin 2.14, momentjs 1.1.1, msbuild 1.26, nested-view 1.14, nuget 0.6, pam-auth 1.3, parameterized-trigger 2.32, performance 2.0, pipeline-build-step 2.4, pipeline-graph-analysis 1.3, pipeline-input-step 2.5, pipeline-milestone-step 1.3, pipeline-rest-api 2.4, pipeline-stage-step 2.2, pipeline-stage-view 2.4, plain-credentials 1.3, run-condition 1.0, scm-api 1.3, scoverage 1.3.1, script-security 1.25, ssh 2.4, ssh-credentials 1.12, ssh-slaves 1.12, structs 1.5, subversion 2.7.1, swarm 2.2, token-macro 2.0, translation 1.15, warnings 4.59, windows-slaves 1.2, workflow-aggregator 2.4, workflow-api 2.8, workflow-basic-steps 2.3, workflow-cps 2.24, workflow-cps-global-lib 2.5, workflow-durable-task-step 2.8, workflow-job 2.9, workflow-multibranch 2.9.2, workflow-scm-step 2.3, workflow-step-api 2.7, workflow-support 2.12, xvfb 1.1.3

While running puppet facts does in fact show the 2 plugins involved:

"jenkins_plugins": "ace-editor 1.1, active-directory 2.0, analysis-core 1.82, ansicolor 0.4.3, ant 1.4, antisamy-markup-formatter 1.5, branch-api 1.11.1, build-monitor-plugin 1.11+build.201701152243, build-pipeline-plugin 1.5.6, cloudbees-folder 5.16, conditional-buildstep 1.3.5, copyartifact 1.38.1, credentials 2.1.10, cucumber-testresult-plugin 0.9.7, cvs 2.13, display-url-api 0.5, durable-task 1.13, email-ext 2.54, envinject 1.93.1, extended-read-permission 1.0, extensible-choice-parameter 1.3.3, external-monitor-job 1.7, git 3.0.1, git-client 2.2.1, git-server 1.7, **gitlab-plugin 1.4.4**, gradle 1.25, greenballs 1.15, **groovy 1.30**, handlebars 1.1.1, hipchat 2.0.0, icon-shim 2.0.3, javadoc 1.4, job-dsl 1.53, job-node-stalker 1.0.5, jquery 1.11.2-0, jquery-detached 1.2.1, junit 1.19, ldap 1.14, mailer 1.18, mapdb-api 1.0.9.0, matrix-auth 1.4, matrix-project 1.8, maven-plugin 2.14, momentjs 1.1.1, msbuild 1.26, nested-view 1.14, nuget 0.6, pam-auth 1.3, parameterized-trigger 2.32, performance 2.0, pipeline-build-step 2.4, pipeline-graph-analysis 1.3, pipeline-input-step 2.5, pipeline-milestone-step 1.3, pipeline-rest-api 2.4, pipeline-stage-step 2.2, pipeline-stage-view 2.4, plain-credentials 1.3, run-condition 1.0, scm-api 1.3, scoverage 1.3.1, script-security 1.25, ssh 2.4, ssh-credentials 1.12, ssh-slaves 1.12, structs 1.5, subversion 2.7.1, swarm 2.2, token-macro 2.0, translation 1.15, warnings 4.59, windows-slaves 1.2, workflow-aggregator 2.4, workflow-api 2.8, workflow-basic-steps 2.3, workflow-cps 2.24, workflow-cps-global-lib 2.5, workflow-durable-task-step 2.8, workflow-job 2.9, workflow-multibranch 2.9.2, workflow-scm-step 2.3, workflow-step-api 2.7, workflow-support 2.12, xvfb 1.1.3"

The weird thing is that a manual puppet run works as I'd expect:

root@aws-nv-d-jenkins-2-master01:~# puppet agent -t -v
Info: Using configured environment 'development'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for aws-nv-d-jenkins-2-master01.valkyrie.net
Info: Applying configuration version '1492552615'
Notice: /Stage[main]/Splunk::Install/Exec[facter_variable]/returns: executed successfully
Notice: Applied catalog in 4.48 seconds

What could be happening to cause this issue?

[llowder]

I've been running into this myself - but with a different plugin.

It also happened on every run for me - manual or daemonized.

I was able to fix it by adding the digest_string param to the swarm plugin call in master.pp and also changing the define in plugin.pp to user the user and group parameters of archive.

[mjmccormac]

Well, I've upgraded Jenkins to the latest LTS release (2.46.2) and most of its plugins and the issue "fixed itself". Will close...