How We (almost) Reinvented Jenkins X on Our Own

By Sebastian Feduniak | December 4, 2018

How We (almost) Reinvented Jenkins X on Our Own

Introduction

I assume that almost everyone heard about the Jenkins. Many of you heard about the Kubernetes.

In the world of infrastructure as a code in many cases, we set up Jenkins server in an automated fashion. There are plenty of solutions which we can leverage - starting from Jenkins Job Builder, through job-dsl and ending with Jenkins Pipelines.

However, here comes a tricky question: how many of you automated Jenkins setup on Kubernetes?

In this post, I want to propose one of many approaches how to do that. Also, I want to describe a few things that helped me to automate the process entirely. This post is especially helpful for you if you are migrating an existing Jenkins server to the Kubernetes.

Finally, in the end, we summarize the whole approach and compare it with an opinionated solution called Jenkins X prepared by the Jenkins foundation itself.

Solution

Jenkins + Kubernetes + Groovy

So we need the following ingredients to build our solution:

  1. Helm chart for Jenkins.
  2. Official Docker images with Jenkins.
  3. Butler - a tool to import/export plugins and jobs to/from Jenkins.
  4. Groovy - as our workhorse to codify everything which is not available directly in the Jenkins configuration.

Configuration

For us, a starting point is a file named config.xml. It is a place where resides the most of the configuration for our server.

Please take a default template from a Helm chart and customize it up to your needs.

I highlight the changes essential for me below:

  • disabledAdministrativeMonitors - Disabling admin alerts that Jenkins should not notify.
  • authorizationStrategy - Configuring user authentication and authorization.
    • You can use hudson.security.AuthorizationStrategy$Unsecured if your domain is already secured.
  • workspaceDir / buildsDir - Setting directories where we keep current and historical data.
  • clouds - A configuration of your Kubernetes environment.
  • views - A description of how you group your jobs.

Some of the things above you can fill some directly in the config template, or you can use values file to load the data.

As a next step we tackle the following elements:

  • plugins,
  • Jenkins location URL,
  • script approvals,
  • global pipeline libraries,
  • credentials.

We cover those elements using Jenkins’s init scripts functionality.

Plugins

Configure list of extensions in the values file. Format as below:

  InstallPlugins:
    - kubernetes:1.13.6
    # Pipeline plugin
    - workflow-aggregator:2.6
    # Pipeline: Job plugin
    - workflow-job:2.25
    - branch-api:2.1.1
    - credentials-binding:1.17
    - git:3.9.1
    - blueocean:1.9.0

Initialization scripts

You can benefit from great integration between Jenkins and Groovy programming language by utilizing init scripts that help you codify the parts not available directly in the config.

Put your scripts here.

Examples:

Global pipeline libraries

If you use pipelines to organize your workflow, then you definitely should consider shared-libraries plugin to remove duplications in your code.

Having your libraries codified, you need to set up them in the Jenkins’s configuration to make it available for the jobs. Fortunately, you can create an init script for that - example is available on the plugin’s page

Jenkins location URL

A simple thing but not available in the config. It is especially important when you use Kubernetes as a platform for Jenkins slave nodes. The following snippet does the job:

locationConfig = JenkinsLocationConfiguration.get()
locationConfig.setUrl("http://jenkins:8080")

Script approvals

You have probably face it if you ever implemented Jenkinsfile for your pipeline. You can approve the signatures in an init script.

def signatures = [
        'method jenkins.model.CauseOfInterruption$UserInterruption getUser'
]
signatures.each {
    org.jenkinsci.plugins.scriptsecurity.scripts.ScriptApproval.get().approveSignature(it)
}

Credentials

That topic is a more complicated element and requires digging into the code of the credentials-plugin.

Finally, you can codify your credentials as well. Be careful about the formats and encoding while working with it.

import com.cloudbees.plugins.credentials.impl.*;
import com.cloudbees.plugins.credentials.*;
import com.cloudbees.plugins.credentials.domains.*;
import org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl;

// Username with password credentials.
someCreds = new UsernamePasswordCredentialsImpl(
        CredentialsScope.GLOBAL,
        "creds_id", "creds_description",
        "creds_user",
        "creds_pass"
)
SystemCredentialsProvider.getInstance().getStore().addCredentials(Domain.global(), someCreds)

// Secret files.
secretEncoded = "YOUR_SECRET_IN_BASE64"
secretBytes = SecretBytes.fromString(secretEncoded)
fileCredentials = new FileCredentialsImpl(
        CredentialsScope.GLOBAL,
        "secret_id", "secret_description",
        "file_name", secretBytes)
SystemCredentialsProvider.getInstance().getStore().addCredentials(Domain.global(), fileCredentials)

Importing/Exporting Jobs

Assuming that your build server is already set up and you have the significant amount of jobs there, you have to think about keeping it in the repo as well.

One option is to use job-dsl plugin, but I want to recommend Butler for that purpose. I consider it a far simpler tool that does the job. It supports plugins also, but I have not tested that functionality.

There are two ways to run butler in our setup:

I recommend the second option as some of the URLs may be internal to Kubernetes DNS and you may encounter errors while running the script from your localhost.

Persistent volume

Last, but not least - you should use a persistent volume for your Jenkins server. It helps you to avoid doing a new setup from scratch in case of any issues/crashes.

You can create persistent volume outside of your Kubernetes environment. We leverage AWS Elastic Block Storage (EBS). We attach such volume accordingly to the following example.

Testing

Yeah, I do not have to remind you to test it before the final deployment.

Init scripts

For that element, use Jenkins script console or jenkins-cli.

Helm charts

Just run your charts in dry-run mode so you can see prefilled templates without running it.

Summary

You saw: how to manage a new installation of Jenkins, how to codify and migrate your existing installation, a couple of good practices around these topics and a few possible ways to test it.

Do we need to do everything on our own?

It depends. No, I really mean it - everything depends on what your exact needs are. However, there are alternatives available - the most useful and famous is Jenkins X.

Enter the Jenkins X

Jenkins X

Jenkins X is a CI/CD solution for modern cloud applications on Kubernetes. This project is being proposed as a subproject of the Jenkins Foundation.

Sounds like what we need, right? Let’s evaluate the similarities and differences between this project and our approach.

Similarities

Architecture diagram for Jenkins X

It also uses Helm and leverages fully Kubernetes potential. It is a very similar way to adjust classic Jenkins to be an entirely cloud-native solution that can be deployed and integrated with a cluster.

Differences

First things first, that solution uses an additional tool called skaffold. How it differs from helm? Does it use helm charts?

helm lets you install/upgrade packages called charts which use one or more Docker images which are in some Docker registry along with some Kubernetes YAML to install/upgrade apps in a cluster.

skaffold is a tool for performing Docker builds and optionally redeploying apps via kubectl or helm - either inside a CI/CD pipeline or when developing locally.

Jenkins X uses skaffold in its CI/CD pipelines to create Docker images. It releases versioned docker images and helm charts on each merge to master. Then it promotes that to environments via helm.

More than that - Jenkins X exposes additional interface called JX API. That piece allows you to bootstrap a new service with full CI/CD. Everything quickly and predictably, with enforcing project standards via environments and pod templates. The same goes for deployment and releasing new versions.

Verdict

Your choice depends on how do you want to use Jenkins on Kubernetes and how you tackle the migration process.

If you would like to leverage additional benefits related with deployments or scaffolding new services with the use of JX API, a Jenkins foundation subproject is a way to go.

When your choice is to migrate an extensive existing Jenkins build server into Kubernetes you may consider migrating it without converting it into Jenkins X project. That was the case for us.

The choice is yours - the most important thing that it should be an informed decision, supported by the upfront research and discussion.

Build the Competetive Advantage with DevOps culture

Continuous Delivery is one of our core expertise. Partner with our experienced and pragmatic builders that help you innovate move quickly and be cost-effective with use of cloud platform and DevOps culture.

Schedule a call with our expert
comments powered by Disqus