34,463 views

Jenkins works perfectly well as a stand-alone open source tool but with the shift to Cloud native and Kubernetes, it invites challenges in terms of management and operation. In a recent past Jenkins X has emerged as a way to improve and automate continuous delivery pipelines to Kubernetes and cloud native environments. Jenkins X is a CI/CD solution for modern cloud applications on Kubernetes. It helps us create great pipelines for our projects and implement a full CI and CD. Jenkins X provides an automated CI/CD for your modern cloud applications on Kubernetes and enables developers to quickly establish continuous delivery best practices.

Container orchestration tools such as Kubernetes allow DevOps teams to manage these containers, leading to faster deployments and shorter time to market, while also satisfying heightened customer expectations and remaining competitive. 

Compelling Feature of Jenkins X 

How to Install Jenkins X on an Existing Kubernetes Cluster – The New Stack


Jenkins X is not just a CI/CD tool to run your builds and deployments, it is an attempt to automate the whole development process end to end for containerised applications based on Docker and Kubernetes. It is obviously open source, as are all best applications. Jenkins X builds upon the DevOps model of loosely-coupled architectures. It is designed to support deployment of large numbers of distributed microservices in a repeatable and manageable fashion, across multiple teams.

Today Developers shouldn’t spend time figuring out how to package software as docker images. They shouldn’t really create the Kubernetes YAML to run their application on Kubernetes, even shouldn’t care about  creating Preview environments or even learn how to implement CI/CD pipelines with declarative pipeline-as-code Jenkinsfiles. They should focus on their code and hence, on delivering value! Jenkins X is a project which rethinks how developers should interact with CI/CD in the cloud with a focus on making development teams productive through automation, tooling and DevOps best practices. What’s cool about Jenkins X is as a developer you can type one command jx create or jx import and get your source code, git repository and application created, automatically built and deployed to Kubernetes on each Pull Request or git push with full CI/CD complete with Environments and Promotion via GitOps!

Click on the image to redirect to Collabnix Slack


Jenkins X is designed to make it simple for developers to work to DevOps principles and best practices

Rather than having to have deep knowledge of the internals of Jenkins X Pipeline, Jenkins X will default awesome pipelines for your projects that implement fully CI and CD. Below are the few compelling features of Jenkins X .

  • Automated CI-CD Pipeline
  • Environmental Promotion using GitOps 
  • Preview Environments

Jenkins X Architecture

At the heart of the system is Kubernetes. Kubernetes hosts all services deployed by JX, including administrative ones (Jenkins, Chartmuseum, Monocular etc).  Let us talk about each of these components which falls under architecture:

 Source ~ https://jenkins-x.io/about/#architecture

  • Git: Git stores all the code and configurations, including environment setup. Serves as a source of truth for everything. Jenkins X manages remote Git repositories and follows the GitOps principles.
  • Helm: Deployment of the services (or applications) is coordinated via Helm. Helm’s Charts allow sharing of application templates and makes versioning easy. Helm also takes care of upgrade and rollback cases, which makes it quite useful.
  • Chartmuseum: It is basically a Helm Charts repository which helps to manage charts via rest api. 
  • Monocular: It is a web-based UI to Helm Charts repository
  • Nexus –  acts as a dependency cache for Node Js and Java applications to dramatically improve build times. After an initial build of a SpringBoot application the build time is reduced from 12 mins to 4. We have not yet but intend to demonstrate swapping this with Artifactory soon.
  • Docker Registry — an in cluster docker registry where our pipelines push application images.

Introducing JX CLI

jx is a command line tool for working with Jenkins X. It does all the magic of bringing all building blocks together and providing an entry point to a system management and orchestration. It is written in Go.The JX CLI is utilized by end users to manage resources (apps, environments, urls etc) as well as in Jenkins pipelines created by JX. Few of the notable commands includes:

CommandPurpose
$ jx installInstall JX on Kubernetes Cluster
$ jx createCreates JX resources and associated services like Kubernetes Namespace, Pods, Services etc
$ jx bootBoots up Jenkins X in a Kubernetes cluster using GitOps and a Jenkins X Pipeline
$ jx importImports a project code into JX
$ jx previewCreates a temporary Preview Environment for an application
$ jx promotePromotes an apps version to a specific environment

Under this blog post, we will set up Jenkins X on Google Cloud Platform in 5 Minutes.

Prerequisites:

Jenkins X requires the following supported services installed & configured properly prior to installation:

  • You will require GITHUB as a GIT provider and GITHUB user account.
  • You need to create a GitHub organization. You will also need to create a GitHub bot account/username( In my case, I created it by name “collabnix-bot” over GitHub)
  • Publicly accessible DockerHub account for creating and managing docker images.
  • A local Desktop machine(Linux/Mac) with the jx program.
  • The Kubernetes command-line tool, which can be installed to your local installation using the jx install command.
  • A Google Cloud Platform (GCP) account with the ability to provision kubernetes resources / create kubernetes clusters with API Enabled as shown below:

Setting up GITHUB User & Organization

First of all, let us  create a GitHub organisation which will have two members, a GitHub user account, eg ajeetraina, and a GitHub ‘Pipeline’ bot account, eg collabnix-bot.

Creating GITHUB Organization:

  • Click the “+” at the top right of GitHub’s top navigation bar or by clicking to the create an organization page. Choose free ‘Team for Open Source’ plan for GitHub organisation.. Name your organisation anything you like, eg jenkins-x-testproject.
  • It’s time to Invite your GitHub user account, e.g. ajeetraina to the organisation. 
  • This GitHub user account will create and manage development repositories.

Creating GITHUB Pipeline bot Account

  • Create a GitHub Pipeline bot account. This Pipeline bot will automate pull request notifications and create preview environments for quick validation and acceptance for code merging. 
  • Please note that your Pipeline bot should be created as a member of your GitHub organisation, e.g. jenkins-x-testproject.
  • I would suggest you create a new account that will be only for your bot. 
  • The bot account which you create must have a token created in your organization that authenticates the bot & allows it to perform various tasks on the repositories within your organization. You might need to generate a Git token for your Pipeline Bot with the correct permissions via this GitHub Link.

Installing JX 

jx version --short
Version 2.1.78+cjxd.11


Installing JX Dependencies

jx install dependencies -d kubectl
Installing Kubectl

Creating Kubernetes Cluster using JX

Before you run the below command, ensure that you have a valid GCE account. Run the below `jx create cluster gke` CLI which allows you to create a cluster on Google Kubernetes Engine (GKE), which you can initialise with a name. 

[Captains-Bay]🚩 >  jx create cluster gke --skip-installation -n mytestproject


This command will take 10-15 minutes and goes through the below process:

  • The program opens a web browser and you will be asked to choose the email address associated with your GCP account. 
  • It will allow the Google Cloud SDK access to your account. After confirming, you can close the browser page.
  • Back at the command-line, the jx create cluster program prompts you to choose your Google Cloud Project from the available list.
[Captains-Bay]🚩 >  jx create cluster gke --skip-installation -n mytestproject

This command will take 10-15 minutes and goes through the below process:

The program opens a web browser and you will be asked to choose the email address associated with your GCP account. 
It will allow the Google Cloud SDK access to your account. After confirming, you can close the browser page.
Back at the command-line, the jx create cluster program prompts you to choose your Google Cloud Project from the available list.


? Google Cloud Project: famous-hull-276807
Updated property [core/project].
? Configured cluster name: mytestproject
? Defaulting to cluster type: Zonal
? Google Cloud Zone: us-east1-b
? Defaulting to machine type: n1-standard-2
? Defaulting to minimum number of nodes: 3
? Defaulting to maximum number of nodes: 5
? Defaulting use of preemptible VMs: No
? Defaulting access to Google Cloud Storage / Google Container Registry: Yes
? Defaulting enabling Cloud Build, Container Registry & Container Analysis API's: Yes
? Defaulting enabling Kaniko for building container images: No
Creating cluster...
Creating cluster...
WARNING: Currently VPC-native is not the default mode during cluster creation. In the future, this will become the default mode and can be disabled using `--no-enable-ip-alias` flag. Use `--[no-]enable-ip-alias` flag to suppress this warning.
WARNING: Newly created clusters and node-pools will have node auto-upgrade enabled by default. This can be disabled using the `--no-enable-autoupgrade` flag.
WARNING: Starting with version 1.18, clusters will have shielded GKE nodes by default.
WARNING: Your Pod address range (`--cluster-ipv4-cidr`) can accommodate at most 1008 node(s).
This will enable the autorepair feature for nodes. Please see https://cloud.google.com/kubernetes-engine/docs/node-auto-repair for more information on node autorepairs.
Creating cluster mytestproject in us-east1-b...
................................done.
Created [https://container.googleapis.com/v1/projects/famous-hull-276807/zones/us-east1-b/clusters/mytestproject].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east1-b/mytestproject?project=famous-hull-276807
kubeconfig entry generated for mytestproject.
NAME           LOCATION    MASTER_VERSION  MASTER_IP     MACHINE_TYPE   NODE_VERSION    NUM_NODES  STATUS
mytestproject  us-east1-b  1.14.10-gke.36  34.73.89.190  n1-standard-2  1.14.10-gke.36  3          RUNNING
Initialising cluster ...
gke cluster created. Skipping Jenkins X installation.
Fetching cluster endpoint and auth data.
kubeconfig entry generated for mytestproject.
Context "gke_famous-hull-276807_us-east1-b_mytestproject" modified.

You can visit GCP console and click on “Workload” to see all Jenkins X specific workloads up and running:

Verifying Kubernetes Cluster

kubectl get nodes
NAME                                           STATUS   ROLES    AGE   VERSION
gke-mytestproject-default-pool-c87818e4-3vbd   Ready    <none>   50s   v1.14.10-gke.36
gke-mytestproject-default-pool-c87818e4-551t   Ready    <none>   50s   v1.14.10-gke.36
gke-mytestproject-default-pool-c87818e4-qg4b   Ready    <none>   50s   v1.14.10-gke.36

Clone the Jenkins X Boot configuration Repository 

Under this step, we will clone Jenkins X boot configuration repository. We will then open the jx-requirements.yml file of your newly cloned repo, eg. jenkins-x-boot-config/jx-requirements.yml. This specifies the requirements of your installation, including:

[Captains-Bay]🚩 >  git clone https://github.com/jenkins-x/jenkins-x-boot-config

Running JX Boot CLI

The ‘jx boot’ utility command allows you to boot up Jenkins X in a Kubernetes cluster using GitOps and a Jenkins X Pipeline. Jenkins X Boot uses the following approach:

jx boot

Then you will be asked a series of questions to ensure Jenkins X is installed properly on your cluster:

QuestionsWhat to respond?
You will be asked to input the Git Owner name for environment repositories:Type in the organisation you created: jenkins-x-testproject
You will be asked to provide Comma-separated git provider usernames of approvers for development environment repositoryType in the name of the GitHub account that is a member of the organisation you created, eg ajeetraina
You may receive a Warning that TLS is not enabled so your webhooks will be called using HTTP. You will be asked for confirmation to continue.
If you type ‘No’, the jx boot process will end with error: cannot continue because TLS is not enabled.If you type ‘Yes’, then namespace jx will be created in your cluster and Jenkins X booted in that namespace.
You may be asked if you wish to upgrade jx. It is recommended you say ‘Yes’ and then re-run jx boot.
There will be information logged on enabling storage on GKE. You do not need to enable storage for this walkthrough tutorial. 
You will be asked Jenkins X Admin UsernameType in a username or press return to have the default username of admin
You will then be asked for Jenkins X Admin PasswordType one in 
You will then be asked for the Pipeline bot Git username: Type in the name of the Pipeline Bot you created, eg jx-bot.Type in the name of the Pipeline Bot you created, eg collabnix-bot
You will then be asked for Pipeline bot Git email addressType in the email address you used when setting up your Pipeline Bot. 
You will then be asked for Pipeline bot Git token..Type in the token generated and saved previously.
You will be asked, Do you want to configure an external Docker Registry?: ‘No’ is sufficient for this tutorial‘No’ is sufficient for this tutorial. But if you say “yes” you might need to provide Docker Registry Details ( ie. DockerHub username and password)
  

Then you will see confirmation on the state of your installation process, such as “Installation is currently looking: GOOD” message.In the organisation you created, eg jenkins-x-testproject, there should now be 3 additional repositories for the dev, staging, and production environments, which map to the dev, staging, and production namespaces in your cluster.

verifying the Jenkins X installation in namespace jx
verifying pods
Checking pod statuses
POD                                                STATUS
jenkins-x-chartmuseum-d87cbb789-h4kzz              Running
jenkins-x-controllerbuild-847f4f4b79-t6g64         Running
jenkins-x-controllerrole-75d8d87d98-p5p2l          Running
jenkins-x-heapster-54bdffbc79-fwlds                Running
jenkins-x-nexus-b69b7745b-xnzrl                    Running
jx-vault-mytestproject-0                           Running
jx-vault-mytestproject-configurer-5547f59f5b-d2vjk Running
lighthouse-foghorn-dd76f6664-w9ql4                 Running
lighthouse-keeper-5f89b8b978-chkdp                 Running
lighthouse-webhooks-5cdf4b9f65-6mchq               Running
lighthouse-webhooks-5cdf4b9f65-nqjvw               Running
tekton-pipelines-controller-88c7cd9d5-4hr7z        Running
vault-operator-75d5446bb7-mnj6h                    Running
Verifying the git config
Verifying username collabnix-bot at git server github at https://github.com
Found 1 organisations in git server https://github.com: jenkins-x-testproject
Validated pipeline user collabnix-bot on git server https://github.com

Congratulations ! Jenkins X should be installed on your Kubernetes cluster by now.

Getting Jenkins X Environment

jx get environments
NAME       LABEL       KIND        PROMOTE NAMESPACE     ORDER CLUSTER SOURCE                                                                            REF    PR
dev        Development Development Never   jx            0             https://github.com/jenkins-x-testproject/environment-mytestproject-dev.git        master
staging    Staging     Permanent   Auto    jx-staging    100           https://github.com/jenkins-x-testproject/environment-mytestproject-staging.git    master
production Production  Permanent   Manual  jx-production 200           https://github.com/jenkins-x-testproject/environment-mytestproject-production.git master
[Captains-Bay]🚩 >

Bringing up GUI

To use the GUI, you must install and configure it for your CloudBees Jenkins X Distribution environment. Ensure that you have the proper namespace set by running jx from the command line to switch to the jx namespace:

jx ns jx
Using namespace 'jx' from context named 'gke_famous-hull-276807_us-east1-b_mytestproject' on server 'https://34.73.89.190'.


From a command-line, install the UI application by using the jx add app command. Please Note: If you have enabled GitOps mode in your CloudBees Jenkins X Distribution cluster, the jx add app command updates your development (dev) environment repository and automatically merges the UI app source code changes to your dev environment.


[Captains-Bay]🚩 >  jx add app jx-app-ui --version 0.1.211
WARNING: No secrets found on "helm/repos" due: reading the secret "helm/repos" from vault: no secret "helm/repos" not found in vault
Read credentials for http://chartmuseum.jenkins-x.io from vault helm/repos
Preparing questions to configure jx-app-ui. If this is the first time you have installed the app, this may take a couple of minutes.
Questions prepared.
Checking if TLS is enabled in the cluster
Created Pull Request: https://github.com/jenkins-x-testproject/environment-mytestproject-dev/pull/1
Added app via Pull Request https://github.com/jenkins-x-testproject/environment-mytestproject-dev/pull/1
[Captains-Bay]🚩 >  jx gadd app jx-app-ui --version 0.1.211kubectl get -n jx ingress jenkins-x-jxui

The ‘jx ui’ CLI opens the CloudBees Jenkins X UI app for Kubernetes for visualising CI/CD and your environments. Before running the above command, let us ensure that we are under “jx” namespace.


[Captains-Bay]🚩 >  jx ui -p 8080
UI not configured to run with TLS - The UI will open in read-only mode with port-forwarding only for the current user
Waiting for the UI to be ready on http://localhost:8080...
.
.
Jenkins X UI: http://localhost:8080
Opening the UI in the browser...

Opening up Jenkins X UI

Let’s go ahead and test drive a new app from a Quickstart and import the generated code into Git and Jenkins for CI/CD.

Search for Python-http and click “Continue”.

You have to enter the GITHUB Access token, organization name and name of the new repository which you want to create. Click “Finish” once you enter the details.

If you didn’t opt for GUI, you can perform the quickstart via CLI too as shown below:

jx create quickstart
? select the quickstart you wish to create python-http
Using Git provider github.com at https://github.com
? Git user name? collabnix-bot
? Who should be the owner of the repository? jenkins-x-testproject
? Enter the new repository name:  tp-alpha
Creating repository jenkins-x-testproject/tp-alpha
Generated quickstart at /Users/ajeetraina/july1/tp-alpha
Created project at /Users/ajeetraina/july1/tp-alpha
The directory /Users/ajeetraina/july1/tp-alpha is not yet using git
? Would you like to initialise git now? Yes
? Commit message:  Initial import

Git repository created
performing pack detection in folder /Users/ajeetraina/july1/tp-alpha
--> Draft detected Python (95.544554%)
selected pack: /Users/ajeetraina/.jx/draft/packs/github.com/jenkins-x-buildpacks/jenkins-x-kubernetes/packs/python
replacing placeholders in directory /Users/ajeetraina/july1/tp-alpha
app name: tp-alpha, git server: github.com, org: jenkins-x-testproject, Docker registry org: famous-hull-276807
skipping directory "/Users/ajeetraina/july1/tp-alpha/.git"
Draft pack python added
? Would you like to define a different preview namespace? Yes
? Enter the name for the preview namespace:  jx-previews
Pushed Git repository to https://github.com/jenkins-x-testproject/tp-alpha
Creating GitHub webhook for jenkins-x-testproject/tp-alpha for url http://hook-jx.35.237.205.64.nip.io/hook
Created Pull Request: https://github.com/jenkins-x-testproject/environment-mytestproject-dev/pull/5
Added label updatebot to Pull Request https://github.com/jenkins-x-testproject/environment-mytestproject-dev/pull/5
created pull request https://github.com/jenkins-x-testproject/environment-mytestproject-dev/pull/5 on the development git repository https://github.com/jenkins-x-testproject/environment-mytestproject-dev.git
regenerated Prow configuration
PipelineActivity for jenkins-x-testproject-tp-alpha-master-1
upserted PipelineResource meta-jenkins-x-testproject-tp-a-w9sxr for the git repository https://github.com/jenkins-x-testproject/tp-alpha.git
upserted Task meta-jenkins-x-testproject-tp-a-w9sxr-meta-pipeline-1
upserted Pipeline meta-jenkins-x-testproject-tp-a-w9sxr-1
created PipelineRun meta-jenkins-x-testproject-tp-a-w9sxr-1
created PipelineStructure meta-jenkins-x-testproject-tp-a-w9sxr-1

Watch pipeline activity via:    jx get activity -f tp-alpha -w
Browse the pipeline log via:    jx get build logs jenkins-x-testproject/tp-alpha/master
You can list the pipelines via: jx get pipelines
When the pipeline is complete:  jx get applications

For more help on available commands see: https://jenkins-x.io/developing/browsing/

Note that your first pipeline may take a few minutes to start while the necessary images get downloaded!

Conclusion

If you’re looking for a tool which can help you achieve CI-CD without any effort of assembling things together yourself, Jenkins X is a right tool for you.  It doesn’t aim to replace Jenkins but builds on it with best of breed open source tools. You do not directly install Jenkins to use Jenkins X, Jenkins is right embedded for you as a pipeline engine as part of the installation for a team.It is a complete CI/CD process with a Jenkins pipeline that builds and packages project code for deployment to Kubernetes and access to pipelines for promoting projects to staging and production environments.

Clap

Categories: Docker

Ajeet Raina

My name is Ajeet Singh Raina and I am an author of this blogging site. I am a Docker Captain, ARM Innovator & Docker Bangalore Community Leader. I bagged 2 special awards last year(2019) : Firstly, “The Tip of Captain’s Hat Award” at Dockercon 2019, San Francisco and secondly, “2019 Docker Community Award“. I run Collabnix Community Slack with over 4700+ audience as well as Redis Community Slack with close to 500+ members. I have built popular GITHUB repositories like DockerLabs, KubeLabs, Kubetools, RedisPlanet Terraform etc. with the support of Collabnix Community. Currently working as Developer Relations Manager at Redis Labs where I help customers and community members adopt Redis. With over 11,000+ followers over LinkedIn & close to 4800+ twitter followers, I like sharing Docker and Kubernetes related content . You can follow me on Twitter(@ajeetsraina) & GitHub(@ajeetraina)

0 Comments

Leave a Reply

Your e-mail address will not be published. Required fields are marked *