DevOps Archives - Page 85 of 91 - DEVOPS DONE RIGHT.

Setup Jenkins using Ansible

In this document I’ll walk you through how you can setup Jenkins using Ansible.

Prerequisites

OS – Ubuntu {at least two machine required in production}
First machine for Ansible installation
Second machine where we will install jenkins server
You should have basic understanding of ansible workflow.

Note : You should have password less login enabled in second machine. use this link
http://www.linuxproblem.org/art_9.html

Ansible Installation
Before starting with installing Jenkins using Ansible, you need to have Ansible installed in your system.

$ curl https://raw.githubusercontent.com/OpsTree/AnsiblePOC/alok/scripts/Setup/setup_ansible.sh | sudo bash

Setup jenkins using Ansible
Install jenkins ansible roles
Once we have ansible installed in our system, we can start installing the jenkins using ansible. To install we will use an already available ansible role to setup jenkins

$ ansible-galaxy install geerlingguy.jenkins

to know more about the jenkins role hit this link https://galaxy.ansible.com/detail#/role/440

Ansible roles default directory path is /etc/ansible/roles
Make ansible playbook file

Now the next step is to use the installed jenkins roles to install the jenkins. For this purpose we will create a playbook and hosts file with below content

$ cd ~/MyPlaybook/jenkins

create a file hosts and add below content

[jenkins_hosts]

192.168.33.15

Next create a file site.yml and add below content

—

– hosts: jenkins_hosts

roles:

– { role: geerlingguy.jenkins }

so configuration file is done, the next step is to run ansible playbook command

$ ansible-playbook -i hosts site.yml

Now that Jenkins is running, go to http://192.168.33.15:8080. You’ll be welcome by the default Jenkins screen.

Opstree SHOA Part 1: Build & Release

At Opstree we have started a new initiative called SHOA, Saturday Hands On Activity. Under this program we pick up a concept, tool or technology and do a hands on activity. At the end of the day whatever we do is followed by a blog or series of blog that we have understood during the day.

Since this is the first Hands On Activity so we are starting with Build & Release.

What we intend to do

Setup Build & Release for project under git repository https://github.com/OpsTree/ContinuousIntegration.

What all we will be doing to achieve it

Finalize a SCM tool that we are going to use puppet/chef/ansible.
Automated setup of Jenkins using SCM tool.
Automated setup of Nexus/Artifactory/Archiva using SCM tool.
Automated setup of Sonar using SCM tool.
Dev Environment setup using SCM tool: Since this is a web app project so our Devw443 environment will have Nginx & tomcat.
QA Environment setup using SCM tool: Since this is a web app project so our QA environment will have Nginx & tomcat.
Creation of various build jobs
- Code Stability Job.
- Code Quality Job.
- Code Coverage Job.
- Functional Test Job on dev environment.
Creation of release Job.
Creation of deployment job to do deployment on Dev & QA environment.

This activity is open for public as well so if you have any suggestion or you want to attend it you are most welcome

Marrying Nginx with ELB

Few weeks back I got a requirement to setup a highly available API server. I said not a big deal! I’ll have Nginx as a reverse proxy(Why not directly exposing API via ELB a different story) and my API auto scaled setup will sit behind an internal ELB and things would be in place TA DA.

Things worked perfectly fine for few days, but one day the API consumer reported that they are not getting response back, what? When I checked the API url was indeed returning a 502 error code. It was really strange for nginx to be sending 502 response back that meant the highly scalable setup was down? Well I was proven wrong ELB was working perfectly fine as the curl request to internal ELB was returning proper response, so yes the highly available API setup was in place. What next, yes! Nginx error logs. I did saw Nginx reporting connection timeout with 502 error code. The interesting thing to note that it was an IP(random IP assigned to ELB), when I tried to do curl hit on that IP for API request, it did failed EUREKA EUREKA!! I reproduced the problem.

Well now I’ve to collect all this information and infer what is the logical cause of the problem, and yes there are lot of smart people available who would have fond the solution to this problem so I’ve to ask right question in Google :). The question was “Nginx using Ip instead of domain name” and the answer was “Nginx caches the IP at the startup and obviously as ELB is Elastic in nature so it’s IP changes over the period of time”. That was the reason Nginx was trying to talk to the older un-associated IP’s of Internal ELB.

Finding solution was not a big task as it was just about making sure Nginx should talk to ELB not the IP’s associated with it, that’s why said marrying nginx with ELB :).

I’ll not go into the actual solution as there are already solutions available in web. I referred this really good blog as a solution.

http://ghost.thekindof.me/nginx-aws-elb-dns-resolution-nginx-resolver-directive-and-black-magic/

PERCONA STANDALONE SERVER

As a DevOps activist I am exploring Percona XtraDB. In a series of blogs I will share my learnings. This blog intends to capture step by step details of installation of Percona XtraDB in Standalone mode

Introduction:

Percona Server is an enhanced drop-in replacement for Mysql. It offers breakthrough performance, scalability, features, and instrumentation.

Percona focus on providing a solution for the most demanding applications, empowering users to get the best performance and lowest downtime possible.

The Percona XtraDB Storage Engine:

Percona XtraDB is an enhanced version of the InnoDB storage engine, designed to better scale on modern hardware, and including a variety of other features useful in high performance environments. It is fully backwards compatible, and so can be used as a drop-in replacement for standard InnoDB.
Percona XtraDB includes all of InnoDB’s robust, reliable ACID-compliant design and advanced MVCC architecture, and builds on that solid foundation with more features, more tunability. more metrics, and more scalability.
It is designed to scale better on many cores, to use memory more efficiently, and to be more convenient and useful.

Installation on ubuntu:

STEP 1: Add Percona Software Repositories

$ apt-key adv --keyserver keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A

STEP 2: Add this to /etc/apt/sources.list:

deb http://repo.percona.com/apt precise main
deb-src http://repo.percona.com/apt precise main

STEP 3: Update the local cache

$ apt-get update

STEP 4: Install the server and client packages

$ apt-get install percona-server-server-5.6 percona-server-client-5.6

STEP 5: Start Percona Server

$ service mysql start

Let me know if you have any suggestions.

Understanding Percona XtraDB cluster

As a DevOps activist I am exploring Percona XtraDB. In a series of blogs I will share my learnings. This blog intends to capture theoretical knowledge of Percona XtraDB Cluster.

Prerequisites

You should have basic knowledge of mysql.
OS – Ubuntu

What is Percona?

Percona XtraDB cluster is an open source, free MySql high availability and scalability software.

It provides:

Synchronous Replication: Transaction either committed on all nodes or none.
Multi-Master Replication: You can write to any node
Parallel applying events on slave. Real “parallel replication”.
Automatic node provisioning.
Data consistency. No more unsynchronized slaves.

Introduction

The cluster consists of nodes. The cluster’s recommended configuration is to have 3 nodes, however 2 nodes can be used as well.
Every node is a regular Mysql / Percona server setup. You can convert your existing MySQL / Percona Server into Node and roll Cluster using it as a base or you can detach Node from Cluster and use it as a regular server.
Each node will contain full copy of data.

Benefits of this approach:

Whenever you execute a query, it is executed locally. All data is available locally, so no remote access is required.
No central management. You can loose any node at any time, and cluster will continue functioning.
It is a good solution for scaling read workload. You can put read queries to any of the nodes.

Drawbacks:

Overhead of joining new node. New node will copy all data from an existing node. If it is 100 GB, it will copy 100 GB.
Not an effective write scaling solution. All writes have to go on all nodes.
Duplication of data. If you have 3 nodes, there will be 3 duplicates.

Difference between Percona XtraDB Cluster and MySQL Replication

For this we will have to look into the well known CAP theorem for distributed systems. According to this theorem, characteristics of Distributed systems are:

C – Consistency (all your data is consistent on all nodes),

A – Availability (your system is AVAILABLE to handle requests in case of failure of one or several nodes),

P – Partitioning tolerance (in case of inter-node connection failure, each node is still available to handle requests).

CAP theorem says that any Distributed system can have any two out of these three.

MySQL replication has: Availability and Partitioning tolerance.
Percona XtraDB Cluster has: Consistency and Availability.

So, MySql replication does not guarantee Consistency of data, while Percona XtraDB cluster provides consistency while it looses partitioning tolerance.

Components

Percona XtraDb Cluster is based on:

Percona Server with XtraDB and includes Write Set Replication patches.

It uses:

Galera Library: A generic synchronous Multi-Master replication plugin for transactional applications.
Galera supports:
- Incremental State Transfer (IST), useful in WAN deployments.
- RSU, Rolling Schema Update. Schema change does not block operations against table.

Percona XtraDB cluster limitations

Currently replication work only with InnoDB storage engine.

That means writes to table of other types, including (mysql.*) tables, are not replicated.

DDL statements are replicated in statement level and changes to mysql.* tables will get replicated that way.

So you can issue: CREATE USER …. , this will be replicated,

but issuing: INSERT INTO mysql.user …. , will not be replicated.

You can also enable experimental MyISAM replication support with wsrep_replicate_myisam.

Unsupported queries:
- LOCK/UNLOCK tables
- lock function (GET_LOCK(), RELEASE_LOCK()….)

Due to cluster level concurrency control, transaction issuing COMMIT may be aborted at that stage.

There can be two transactions writing to same rows and committing in separate Percona XtraDB Cluster nodes, and only one of the them can successfully commit. The failing one will be aborted. For cluster level aborts, Percona will give back deadlock error code.

The write throughput of whole cluster is limited by weakest node. If one node becomes slow, whole cluster will become slow.

FEATURES

High Availability

In a basic setup with 3 nodes, the Percona XtraDB cluster will continue to function if you take any of the nodes down. Even in a situation of node crash, or if node becomes unavailable over network, the cluster will continue to work, and queries can be issued on working nodes.

In case, when there are changes in data while node was down, there are two options that Node may use when it joins the cluster:

State Snapshot Transfer (SST): SST method performs full copy of data from one node to other. It’s used when a new node joins the cluster. One of the existing node will transfer data to it.
There are three available methods of SST:

- mysqldump
- rsync
- xtrabackup

Downside of “mysqldump” and “rsync” is that your cluster becomes READ-ONLY while data is copied from one node to other.

while

xtrabackup SST does not require this for entire syncing process.

Incremental State Transfer (IST): If a node is down for a short period of time, and then starts up, the node is able to fetch only those changes made during the period it was down.

This is done using caching mechanism on nodes. Each node contains a cache, ring-buffer of last N changes, and the node is able to transfer part of this cache. IST can be done only if the amount of changes needed to transfer is less than N. If it exceeds N, then the joining node has to perform SST.

Multi-Master Replication

Multi-Master replication stands for the ability to write to any node in the cluster, and not to worry that it will get out-of-sync situation, as it regularly happens with regular MySQL replication if you imprudently write to the wrong server.
With Percona XtraDB Cluster you can write to any node, and the Cluster guarantees consistency of writes. That is, the write is either committed on all the nodes or not committed at all.

All queries are executed locally on the node, and there is a special handling only on COMMIT. When the COMMIT is issued, the transaction has to pass certification on all the nodes. If it does not pass, you will receive ERROR as a response on that query. After that, transaction is applied on the local node.