Jenkins

Links

• Learn X in Y minutes - Groovy
• Share a standard Pipeline across multiple projects with Shared Libraries
• Extending with Shared Libraries

Groovy reference

Strings

• https://www.tutorialspoint.com/groovy/groovy_strings.htm

s="hello"
println s             // hello
println "${s}"        // hello
println '${s}'        // ${s}
sh "echo s is ${s}"   // s is hello -- s is substituted by Groovy
sh 'echo s is ${s}'   // s is       -- s env var not defined

env.s="hello"
println "${env.s}"       // hello
println '${env.s}'       // ${env.s}
sh "echo s is ${env.s}"  // s is hello
sh 'echo s is ${env.s}'  // s is hello

env.BUILD_NUMBER0=String.format('%03d', BUILD_NUMBER as int)
println "${env.BUILD_NUMBER0}"  // 001

// https://stackoverflow.com/questions/50029296/extracting-part-of-a-string-on-jenkins-pipeline
def url = "git@github.com:project/access-server-pd.git"
final beforeColon = url.substring(0, url.indexOf(':'))  // git@github.com
final afterLastSlash = url.substring(url.lastIndexOf('/') + 1, url.length())  // access-server-pd.git
println beforeColon

// Note: def / final optional

String a = "Hello World Hello";
println(a.matches("Hello(.*)")); // true
println(a.replaceAll("^Hello","Bye"));  // Bye World Hell

// dirname using find operator
@NonCPS
def dir_name(path) {
  def dir = path =~ /(^.*)[\\\/]/
  dir ? dir[0][1] : '.'
}

@NonCPS
def dir_name_slash(path) {
  def dir = path =~ /(^.*[\\\/])/
  dir ? dir[0][1] : './'
}

println dir_name_slash("T'ar\\ta\\gueule\\jenkins\\de\\merde.exe")  // T'ar\ta\gueule\jenkins\de\
println dir_name_slash("T'ar/ta/gueule/jenkins/de/merde.exe")       // T'ar/ta/gueule/jenkins/de/
println dir_name_slash("merde.exe")                                 // ./
println dir_name_slash("c:\\merde.exe")                             // c:\

array

def arr=[]
arr += ['foo']
arr += ['bar']

arr.join(', ')  // 'foo, bar'
'foo' in arr    // true

Map

def map=[:]
map += ['foo': "FOO"]
map += ['bar': { println "BAR" }]
map += [baz: { println "BAZ" }] // Don't need quote on definition
println map['foo']     // 'FOO'
map['bar']()           // 'BAR'
map.bar()              // ... can use shorter notation
'foo' in map           // true

# Note that Groovy map maintains order (https://stackoverflow.com/questions/32811732/do-maps-in-groovy-maintain-order)
map.foreach { println "$it.key, $it.value" }
map.foreach { iter -> println "$iter.key, $iter.value" }

Tips

Cancel older builds if new one starting

Using milestone, we can cancel older builds if a new commit is pushed.

Here an example where all new build cancels the older ones, except on the master branch. This is useful to increase throughput of Jenkins slaves. This policy makes sense since non-master branches typically do not require thorough testing/analysis.

    stage('UTs')
    {
        // if not on master and older builds are ongoing, cancel them !
        if ( env.BRANCH_NAME != 'master' )
        {
            def buildNumber = env.BUILD_NUMBER as int
            if (buildNumber > 1) milestone(buildNumber - 1)
            milestone(buildNumber)
        }
        // ...
    }

Use 'Pipeline Syntax' to write Groovy script

Jenkins offers a button Pipeline Syntax to generate Groovy scripts. Very useful for adding new commands.

Custom git scm checkout with another submodule

The first line was created with the pipeline syntax tool in Jenkins. Then we simply checkout the branch we want in the submodule.

 def checkoutUsk() {
     checkout([$class: 'GitSCM', branches: name: '*/master', doGenerateSubmoduleConfigurations: false, extensions: $class: 'SubmoduleOption', disableSubmodules: false, parentCredentials: false, recursiveSubmodules: true, reference: '', trackingSubmodules: false, submoduleCfg: [], userRemoteConfigs: url: 'ssh://user@server.com/project.git'])
     sh 'cd my_submodule && git checkout origin/master && cd ..'
 }

Checkout git with submodule, not recursive

Say we have a repo with submodules, also with submodules, but only want to checkout first level.

The settings that work:

• Add Advanced clone behaviour, and select shallow clone with a depth 1.

• Optionally, select fetch tags or not (note that fetching tags may lead to failure if people create tag foo, then tag foo/bar)

• Select Advanced sub-modules behaviours, but don't click recursively update submodules.

• Also, DO NOT select shallow clone with submodules. It fails.

• Optionally, add Clean Before Checkout.
• Optionally, add Prune stale tags.

Increase perf on Windows

• Disable anti-virus, or move master/slave workspace in not-scanned directory

• Careful slave: on Windows, the slave uses the launch directory as base directory. Move that, or define user.dir in the slave environment variables (within Jenkins).

• Disable Windows search

Increase job number

First get a listing all available jobs in the script console (Jenkins → Manage Jenkins → Script Console):

Jenkins.instance.getAllItems(AbstractItem.class).each {
    println it.fullName + " - " + it.class
};

Then set the RC for the selected job with:

Jenkins.instance.getItemByFullName("your/job/name").updateNextBuildNumber(128)

Git push tag using a SSH key

This is the most difficult ever action.

On Windows in particular.

Really.

Sadly

The solution is to use SSH Agent Plugin.

• Install SSH Agent Plugin.

This requires that ssh-agent is available on Windows (eg. through Git for Windows).

• Add some SSH private key credentials in Jenkins.

This is the same credentials that we use for instance to clone a git repo

Then the magic setup:

stage('deliver') {
    sshagent(['b123c5ec-fb7c-6601-bd09-2e8d3be0aaf0']) { // Use your credential id here
        bat "git push -f origin dev/foo"

        // Some alternatives:
        // sh "git push -f origin dev/foo"   // This may use a different user root

        // sh "ssh -v username@server  // for debugging

        // To force to accept host key 
        // sh "GIT_SSH_COMMAND='ssh -o StrictHostKeyChecking=no' git push -f origin dev/foo"
    }
}

Possible issues:

• Wrong user being picked, hence the bad folder .ssh/ is used.
• Environment clashes / wrong PATH in either bat or sh shell.

Installing plugin offline

• Download the plugin .hpi file.
• Go to Jenkins, Manage Jenkins, Manage plugins, Advanced.
• Use the Upload plugin panel.

Move multibranch pipeline to a git submodule

See stackexchange (and also here).

Let's say we have following project tree:

build/
    Jenkinsfile
src/
    ...
common/           <-- a git submodule
    src/
        ...

Our Jenkinsfile is something like

// file: build/Jenkinsfile

def doThings()
{
    // ...
}

try {
    node {
        // ...
    }
    // ...
}
catch(e) {
    // ...
}
// ...

So basically some utility functions and some pipeline stages (here in a try-catch for instance).

Two actions are required to move this file to our common submodule:

• Move the file to submodule and adapt it slightly.
• Replace the original jenkinsfile with a small boilerplate version.

Our moved Jenkinsfile becomes

// file: common/build/Jenkinsfile

def doThings()
{
    // ...
}

def call() {
    try {
        // ...
    }
    // ...
}
  return this

So basically we surround the pipeline in a call function, and don't forget to return this at the end of the file.

The original Jenkinsfile is then replaced with this small boilerplate code as explained in the original post:

// file: build/Jenkinsfile

def myPipeline
node {
    checkout scm // mandatory
    myPipeline = load "common/build/Jenkinsfile"
}
myPipeline()

Of course this can be tuned further to pass parameters, use several jenkinsfile, etc. For instance reusing ideas from the documentation:

def call(Map params)
{
    println "Building for project " + params.projectName
    // ...
}

// ...
myPipeline(projectName: "MyProject")

Also note that this checkout the whole project, which might be inefficient. A better approach would be to checkout only the submodule that contains the Jenkinsfile.

Troubleshoot

igfx_ error

Doing this:

println new ProcessBuilder('sh','-c','ls').redirectErrorStream(true).start().text

we got a igfx_ crash message in Windows. PC not accessible through remote desktop, and had to reboot.

Multibranch failed because of tag/branch name conflict (some local refs could not be updated; try running)

This occurs when for instance a branch named dev/foo is deleted and a new branch dev/foo/bar is created afterwards.

error: cannot lock ref 'refs/tags/dev/delivery/FOO': 'refs/tags/dev/delivery' exists; cannot create 'refs/tags/dev/delivery/FOO'
 ! [new tag]         dev/delivery/FOO -> dev/delivery/FOO  (unable to update local ref)
error: some local refs could not be updated; try running
 'git remote prune ssh://git.server.com/project/project.git' to remove any old, conflicting branches

To fix:

• Go to multibranch log, and check for line like:

Creating git repository in C:\Jenkins\caches\git-1f7143a5b7a29bcbc3c47f31fc7a597c
 > git init C:\Jenkins\caches\git-1f7143a5b7a29bcbc3c47f31fc7a597c # timeout=10

• Go to the server, and delete that cache directory.

Unable to find project for artifact copy

We get the following error in a multibranch pipeline:

 ERROR: Unable to find project for artifact copy: Foo/bar/master
 This may be due to incorrect project name or permission settings; see help for project name in job configuration.

A FIRST problem is that the Permission to copy artifact cannot be set from the multibranch configuration, but must be set for each branch separatedly.

In script pipelines, do [1]:

properties([[$class: 'JiraProjectProperty'], copyArtifactPermission('*')])

In declarative pipelines, do:

options {
    copyArtifactPermission('my-downstream-project');
}

A SECOND problem is that the full path of the artifact must be given, so better use **/*.tgz or the like:

copyArtifacts filter: '**/*.wsp, **/*.tgz, **/*.exe', fingerprintArtifacts: true, projectName: '${JOB_NAME}', selector: specific(''+currentBuild.number)

Jenkins pipeline java.io.NotSerializableException

Yet more horror from Jenkins [2].

• Jenkins expect that all variables to be serializable.
• This excludes then objects which are null.
• In particular matcher objects (like foo =~ /abc/)

As a result:

• Use and abuse of def. Without it, all variables are defined in global scope.
• Use @NonCPS annotation on functions that are not serializable. Example:

@NonCPS
def version(text) {
  def matcher = text =~ '<version>(.+)</version>'   // This matcher is NOT serializable
  matcher ? matcher[0][1] : null
}