Guide To Setup CI/CD Tools Ci

User Manual:

Open the PDF directly: View PDF .
Page Count: 42

Guide to Setup CI/CD Tools

Table of Contents

Setup VM on VirtualBox 1

Setup VM Network 13

Install and setup GitLab on Ubuntu 18.04 server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê18

Install OpenJDK 8/11 in Ubuntu 18.04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê20

Install Jenkins Server on Ubuntu 18.04. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê21

Prerequiste . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê21

Installtion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê21

Installation and configuration of docker in Ubuntu 18.04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê23

Install docker package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê23

Enable docker command execution without sudo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê24

Install and setup NEXUS 3 on Ubuntu 18.04 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê24

Setup Maven Repository 26

Setup docker private registry 26

Install Helm Repository.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê33

Steps to Integrate Gitlab with Jenkins. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê36

Jenkins Server configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê36

Jenkins-to-Gitlab Clone AUthentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê37

Steps to Integrate Jenkins with Gitlab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ê39

Add NEXUS deployer credentials. 39

Add Environment variables related to docker registry. 39

Testing. 40

Setup VM on VirtualBox

The below steps guides through to create Virtual Machine in VirtualBox.

•In VirtualBox home screen go to Machine Menu and click on New (Machine→New), which

would open VM creation form window.

•In VM creation Window set VM name, location to store machine config and logs. and OS type

desired(In our case it will be Linux) and version (Ubuntu 18.04).

•Click on expert mode and set desired RAM(min 4GB)

•Select Option Create new Virtual Harddisk now and click create.

•Provide directory location to store VM image, set Hard disk space required(min 20GB) and set

Fixed Size Mode so that we get better performance. Let the image type be default VDI. Then

click Create Button(It would take couple of minutes to create Allocate Image)

•Now, Double click on the VM Name in VBox home page. which start the VM.

•A option will popup to provide OS Image. Browse to OS image location and provide OS image

(ubuntu-18.04.1.0-live-server-amd64.iso). Then click on button start.

•After image is loaded, you will be asked to choose the language, set the desired language.

•Next you will be prompted to select keyboard layout, let the default configuration be as it is.

Navigate to Done using tab and click enter.

•Next you will be prompted to select Network interface, let the default configuration be as it is

will change it later on. Navigate to Done using tab and click enter.

•Next setup hard drive by selecting option entire disk with LVM. CAUTION: When choosing LVM

make sure you’ve selected added entire disk to VG by default in 18.04 4GB is selected. You can

add the Disk at later point as well.

•Navigate through Configure Ubuntu Archive Mirror with default values.

•A summary window will shown describing the Disk Partitions, Navigate to Done option and

press Enter.

•Next Profile setup window comes up, fill all the details.

•Once you navigate from profile setup window, it would take sometime to setup OS. Once Setup

completes, would prompt to install grub, select yes to install grub, finally after installation click

on Reboot

Setup VM Network

We will add 3 Networks to VM, with first network being host only, second network being NAT and

final network being Bridged

•Shutdown the VM.

•Select the VM, Navigate to Machine→Settings and select tab Network.

•Select Adapter 1 tab and set attached-to Host-Only Adapter and fill the details as show below.

•Select Adapter 2 tab and set attached-to NAT and fill the details as show below.

•Select Adapter 3 tab and set attached-to Bridged and fill the details as show below.

•Start the VM, login to the VM, execute below command and note down list of interface names.

$ ifconfig -a

•Open netplan yaml file at /etc/netplan/50-cloud-init.yaml and set static IP for host-only

adapter and dhcp IP auto discovery for NAT and Bridged, For Host-only network before setting

up note down the subnet for Host only adapter from VirtualBox Network. You can also setup a

static IP from Bridge Network. A sample configuration is as shown below.

•Reboot the VM.

Install and setup GitLab on Ubuntu 18.04 server.

Install Gitlab

•Update packages

$ sudo apt-get update

•Install additional packages

$ sudo apt-get install -y curl openssh-server ca-certificates

•Add the GitLab package repository and install the package

Add the GitLab package repository.

$ curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh

Ê | sudo bash

•Next, install the GitLab package. Change https://gitlab.example.com to the URL at which you

want to access your GitLab instance. Installation will automatically configure and start GitLab

at that URL. For https:// URLs GitLab will automatically request a certificate with Let’s Encrypt,

which requires inbound HTTP access and a valid hostname. You can also use your own

certificate or just use http://.

$ sudo EXTERNAL_URL="https://gitlab.example.com" apt-get install gitlab-ce

•Browse to the hostname and login.

On your first visit, you’ll be redirected to a password reset screen. Provide the password for the

initial administrator account and you will be redirected back to the login screen. Use the default

account’s username root to login.

Setup Email Address using Gmail.

•Enable 2-way Authentication by following the steps here

•Generate App code.

◦Go to your Google Account.

◦On the left navigation panel, click Security.

◦On the "Signing in to Google" panel, click App passwords. If prompted, enter your password.

If you can’t get to the page, 2-Step Verification is:

▪Not set up for your account

▪Set up for security keys only

◦At the bottom, click Select app and choose the app you’re using.

◦Click Select device and choose the device you’re using.

◦Click Generate.

◦Follow the instructions to enter the App password (the 16 character code in the yellow bar)

on your device.

◦Copy the app password and click Done.

•Setup SMTP server on Gitlab

◦Open /etc/gitlab/gitlab.rb

$ sudo vim /etc/gitlab/gitlab.rb

◦Copy the below content and update email and password generated in previous step

gitlab_rails['smtp_enable'] = true

gitlab_rails['smtp_address'] = "smtp.gmail.com"

gitlab_rails['smtp_port'] = 587

gitlab_rails['smtp_user_name'] = "my.email@gmail.com"

gitlab_rails['smtp_password'] = "my-gmail-password"

gitlab_rails['smtp_domain'] = "smtp.gmail.com"

gitlab_rails['smtp_authentication'] = "login"

gitlab_rails['smtp_enable_starttls_auto'] = true

gitlab_rails['smtp_tls'] = false

gitlab_rails['smtp_openssl_verify_mode'] = 'peer'

gitlab_rails['gitlab_email_from'] = 'gitlab@example.com'

gitlab_rails['gitlab_email_reply_to'] = 'noreply@example.com'

•Reload the configuration be executing below command

$ sudo gitlab-ctl reconfigure

•Testing gitlab’s ability to send email

◦Open Ruby console

$ sudo gitlab-rails console

◦Execute below command after updating destination email address

> Notify.test_email('destination_email@address.com', 'Message Subject',

'<html><body>Message Body</body></html>').deliver_now

Install JDK-8 as Jenkins LTS does not support JDK-11

Install OpenJDK 8/11 in Ubuntu 18.04

Below are the steps to install openjdk 8/11 in ubuntu 18.04. The default jdk shipped in ubuntu 18.04

is JDK-10.

•Add Java PPA Repository.

$ sudo add-apt-repository ppa:webupd8team/java

•Update apt cache.

$ sudo apt update

•Now you can install openjdk-8 or openjdk-11 using below command.

$ sudo apt install openjdk-8-jdk

•Update Java alternatives.

$ sudo update-alternatives --install /usr/bin/java java usr/lib/jvm/java-8-openjdk-amd64/bin/java 1000

Install Jenkins Server on Ubuntu

18.04

Prerequiste

•JDK-8

Installtion

•Update Jenkins key cache.

$ wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -

•Add Repository.

$ sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ >

/etc/apt/sources.list.d/jenkins.list'

•Update package

$ sudo apt-get update

•Install Jenkins server

$ sudo apt-get install jenkins

This package installation will:

•Setup Jenkins as a daemon launched on start. See /etc/init.d/jenkins for more details.

•Create a jenkins user to run this service.

•Direct console log output to the file /var/log/jenkins/jenkins.log. Check this file if you are

troubleshooting Jenkins.

•Populate /etc/default/jenkins with configuration parameters for the launch, e.g JENKINS_HOME

•Set Jenkins to listen on port 8080. Access this port with your browser to start configuration.

If your /etc/init.d/jenkins file fails to start Jenkins, edit the /etc/default/jenkins

to replace the line ----HTTP_PORT=8080---- with ----HTTP_PORT=8081---- Here,

"8081" was chosen but you can put another port available.

•While configuring on UI select Install recommended plugins.

Configure Email Notification using Gmail.

•Go to Manager Jenkins -→ Configure System.

•Goto Extending Email Notification section. Fill details as shown in below screenshot with your

email address. SMTP Server is smtp.gmail.com, select adavanced option and set username and

password(password should be your app code) set USE SSL with Port 465

•Goto E-mail Notification section. Fill details as shown in below screenshot with your email

address. SMTP Server is smtp.gmail.com, select adavanced option and set username and

password(password should be your app code) set USE SSL with Port 465

Use jenkins user while adding user to docker group.

Installation and configuration of

docker in Ubuntu 18.04

Install docker package

Below are the steps to install docker in ubuntu 18.04

•First update existing packages.

$ sudo apt update ; sudo apt upgrade

•Install utility packages

$ sudo apt install apt-transport-https ca-certificates curl software-properties-

common

•Add GPG key

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

•Add docker repository to apt sources.

$ sudo add-apt-repository "deb [arch=amd64]

https://download.docker.com/linux/ubuntu bionic stable"

•Update apt sources and install docker

$ sudo apt update ; sudo apt install docker-ce

•Verify docker service is started.

$ sudo systemctl status docker

Enable docker command execution without sudo.

•We need to add user to docker group to enable docker command execution without sudo user.

◦To add current user

Ê$ sudo usermod -aG docker ${USER}

◦Alternatively to add any other user

Ê$ sudo usermod -aG docker <username>

•Verify if user is added to docker group by listing the groups the user is added to.

$ id -nG ${USER}

•Login to new session, now you can execute docker commands without need sudo access.

$ su - ${USER}

Install and setup NEXUS 3 on Ubuntu 18.04

•Download the NEXUS package

$ wget https://sonatype-

download.global.ssl.fastly.net/repository/repositoryManager/3/nexus-3.15.2-01-

unix.tar.gz

Checkout for the latest version instead of 3.15

•Extract the file

$ tar -xzvf nexus-3.15.2-01-unix.tar.gz

•Move the file to /opt directory.

$ sudo mv nexus-3.15.2-01 /opt/nexus-server/nexus-3.15.2-01.

•Create user nexus

$ sudo addUser nexus

•Give sudo permission to user nexus

$ sudo usermod -aG sudo nexus

•Change the ownersource,ship of the directory.

$ sudo chown nexus:nexus /opt/nexus-server

•In bin/nexus.rc assign the user between the quotes in the line below.

run_as_user="nexus"

•Create a file called nexus.service. Add the following contents, then save the file in the

/etc/systemd/system/ directory:

[Unit]

Description=nexus service

After=network.target

[Service]

Type=forking

LimitNOFILE=65536

ExecStart=/opt/nexus-server/nexus-3.15.2-01/bin/nexus start

ExecStop=/opt/nexus-server/nexus-3.15.2-01/bin/nexus stop

User=nexus

Restart=on-abort

[Install]

WantedBy=multi-user.target

•Activate the service with the following commands:

sudo systemctl daemon-reload

sudo systemctl enable nexus.service

sudo systemctl start nexus.service

•Default username and password is admin and admin123 respectively with port beign 8081

Setup Maven Repository

We will not create maven repository additionally, NEXUS comes with a default configuration for

Maven with repositories.

Following are the repositories created for maven.

•maven-releases, for hosting your release artifacts locally.

•maven-snapsource,shots, for hosting your snapsource,shot artifacts locally.

•maven-central, a repository of type proxy that connects to central maven repository.

•maven-public, a repository of type 'group` which holds the above repository, we connect to this

repository from dev machine, the order of searching for artifacts as defined above.

Setup docker private registry

By default, the Docker client communicates with the repo using HTTPS. Since I didn’t have the

certificate, will use HTTP instead of HTTPS.

The Docker repo requires 2 different ports. We are going to use 8184 for pull from

the proxy repo and 8185 for pull and pusource,sh to the private repo of release

and 8186 for pull and pusource,sh of snapsource,shots.

•Create a blob store.(optional)

Goto settings → repository → Blob stores → Create blob store, provide name of the location,

path also can be customized. image::docker_blob_store_1.png[]

•Create a private repository for releases

Goto settings → repository → repositories → Create repository, a repository recipe page pops

up, select docker(hosted)

◦Set name for the repository docker-release

◦Choose HTTP type and set port to 8185

◦Set anonymous docker pull (optional)

◦Enable docker V1 API.

◦Select previously created blob store from the dropdown.

◦Deployment policy Disable redeploy

◦Cleanup policy none.

•Create a cleanup policy

Goto settings → repository → cleanup policies → create cleanup policy. set name, format to

'docker` and criteria as Last Downloaded before.

•Create a private repository for snapsource,shots

Goto settings → repository → repositories → Create repository, a repository recipe page pops

up, select docker(hosted)

◦Set name for the repository docker-snapsource,shots

◦Choose HTTP type and set port to 8186

◦Set anonymous docker pull (optional)

◦Enable docker V1 API.

◦Select previously created blob store from the dropdown.

◦Deployment policy Allow redeploy

◦Cleanup policy to previously created policy.

[docker snapsource,shots repository 1] | docker_snapsource,shots_repository_1.png

•Create a proxy repository for docker hub

A repository that proxies everything you download from the official registry, Docker Hub. Next

time you download the same dependency, it will be cached in your Nexus.

Goto settings → repository → repositories → Create repository, a repository recipe page pops

up, select docker(proxy)

Set primarily these parameters, name, remote storage(https://registry-1.docker.io), docker

index to docker hub

•Create Group Repository.

This will group all the above repos and provide you a single URL to configure your clients to

download from to.

Goto settings → repository → repositories → Create repository, a repository recipe page pops

up, select docker(group)

◦Set name for the repository docker-public

◦Choose HTTP type and set port to 8184

◦Set anonymous docker pull (optional)

◦Enable docker V1 API.

◦Select previously created blob store from the dropdown.

◦Finally add docker-release, docker-snapsource,shots and 'docker-hub` in order as

source,shown below.

•To enable anonymous pull goto settings → security → realms, add docker Bearer token Realm.

Setup dev machine to use HTTP protocol.

To interact with your repo, the first thing is to configure the Docker daemon in your machine to

accept working with HTTP instead of HTTPS.

•If its ubuntu machine open/create /etc/docker/daemon.json add following details:

{

Ê "insecure-registries": [

Ê "kp-ci:8184",

Ê "kp-ci:8185",

Ê "kp-ci:8186"

Ê ],

Ê "disable-legacy-registry": true

}

•Restart docker daemon service.

$ sudo systemctl restart docker

Create user with deployment privileges.

•First we need to create custom role before creating a user, goto settings → security → Roles →

Create Role ⇒ New Role. Add role Id and role name. and also and below list of privileges.

◦nx-blobstores-all

◦nx-component-upload(most probably this source,should alone with view source,should

suffice, though I did not test)

◦nx-repository-admin---*

◦nx-repository-view---*

•Next goto settings → security → users → create user, a new popup comes prompting for user

details fill all the details and set status to Active and also add the role created at previous step

by moving the role from left hand side box to right hand side box.

Install Helm Repository.

Build from source code.

•Clone the project.

git clone https://github.com/sonatype-nexus-community/nexus-repository-helm.git

•change directory

cd nexus-repository-helm

•Build the source.

$ mvn clean package -DskipTests

If you want stable release checkout by last released version tag.

Enable Helm repository

•Copy the bundle from target to Copy the bundle into

<nexus_dir>/system/org/sonatype/nexus/plugins/nexus-repository-helm/0.0.7/nexus-repository-

helm-0.0.7.jar

Êsudo mkdir -p /opt/nexus-server/nexus-3.15.2-

01/system/org/sonatype/nexus/plugins/nexus-repository-helm/0.0.7/

Êsudo target/nexus-repository-helm-0.0.7.jar \

Ê /opt/nexus-server/nexus-3.15.2-01/system/org/sonatype/nexus/plugins/nexus-

repository-helm/0.0.7/

Êsudo chown nexus:nexus -R /opt/nexus-server/nexus-3.15.2-

01/system/org/sonatype/nexus/plugins/nexus-repository-helm/0.0.7/

•Update OSGi feature by updating features.xml at

<nexus_home>/system/org/sonatype/nexus/assemblies/nexus-core-feature/3.15.2-01/nexus-core-

feature-3.15.2-01-features.xml

◦Add entire highlighted line under nexus-core-feature section.

<feature name="nexus-core-feature"

description="org.sonatype.nexus.assemblies:nexus-core-feature"

version="3.15.2.01">

Ê <details>org.sonatype.nexus.assemblies:nexus-core-feature</details>