Logstash Timestamp

Introduction

A few days back I encountered with a simple but painful issue. I am using ELK to parse my application logs  and generate some meaningful views. Here I met with an issue which is, logstash inserts my logs into elasticsearch as per the current timestamp, instead of the actual time of log generation.
This creates a mess to generate graphs with correct time value on Kibana.
So I had a dig around this and found a way to overcome this concern. I made some changes in my logstash configuration to replace default time-stamp of logstash with the actual timestamp of my logs.

Logstash Filter

Add following piece of code in your  filter plugin section of logstash’s configuration file, and it will make logstash to insert logs into elasticsearch with the actual timestamp of your logs, besides the timestamp of logstash (current timestamp).
 
date {
  locale => "en"
  timezone => "GMT"
  match => [ "timestamp", "yyyy-mm-dd HH:mm:ss +0000" ]
}
In my case, the timezone was GMT  for my logs. You need to change these entries  “yyyy-mm-dd HH:mm:ss +0000”  with the corresponding to the regex for actual timestamp of your logs.

Description

Date plugin will override the logstash’s timestamp with the timestamp of your logs. Now you can easily adjust timezone in kibana and it will show your logs on correct time.
(Note: Kibana adjust UTC time with you bowser’s timezone)

Gitolite

 

Requirement

We need private git repositories for internally use in our project so we use Gitolite for this requirement. Our client has a lot of consultants, partners and short term employees working with their code so they needed a good way of controlling access to the repos and preferably without giving each of them a unix user on the server where the repo is hosted.

What is Gitolite?

Gitolite is basically an access layer on top of Git. Users are granted access to repos via a simple config file and we as an admin only needs the users public SSH key and a username from the user. Gitolite uses this to grant or deny access to our Git repositories. And it does this via a git repository named gitolite-admin.

Installation

We need a public key and a Gitolite user through which we will setup the Gitolite.

In this case I have used my base machine(Ubuntu) public key so that only my machine can manage Gitolite.

Now we will copy this public key to a virtual machine

$ scp ~/.ssh/gitolite.pub git@192.168.0.20:/home/git

 

where vagrant is the user of my virtual machine & its IP is 192.168.0.20

Now we will install & create a gitolite user on remote machine which will be hosting gitolite.

root@git:~# apt-get install gitolite3
 
root@git:~# adduser gitolite
 
Now we need to remove password of gitolite user from below command
 
root@git:~# passwd -d gitolite
 
 
Let’s move & change the ownership of this public key.
root@git:~# mv gitolite.pub /home/gitolite/
root@git:~# chown gitolite:gitolite /home/gitolite/gitolite.pub
 
Become the gitolite user
 
root@git:~# su – gitolite
 
Now setup the gitolite with the public key
 
gitolite@git:~# gitolite setup -pk gitolite.pub
 
Now to manage the repositories, users and access-rights we will download the gitolite-admin(git repository) to our base machine.
 
$ git clone gitolite@192.168.0.20:gitolite-admin
$ cd gitolite-admin
$ ls -l
total
8
drwxr-xr-x
2 nitin nitin 4096 Jan 10 17:52 conf/
drwxr-xr-x
2 nitin nitin 4096 Jan  9 13:43 keydir/
 
where “keydir” is the directory where we store our user’s keys and that key name must be same as existing username on the system.
 
In conf directory there is a “gitolite.conf” file which controls which repositories are available on the system and who has which rights to those repositories.
We just need to add new repository name & users who will access it and this file will create the repo & grant the permission on it accordingly.
Let us explore my gitolite.conf file in which I have added a new repository called “opstreeblog”
$ cat conf/gitolite.conf

# Group name & members

@admin = nitin
@staff    = jatin james
 
# Gitolite admin repository

repo gitolite-admin
RW+   = gitolite @admin
 
# Read-Write permission to all the users on testing repo

repo testing
RW+    = @all
 
# Read-Write permission to user sandy & the admin group. And Read-Only access to staff group

repo opstreeblog
   RW+   = sandy @admin
   R         = @staff

where ‘@’ denotes the user group i.e @staff is a group & jatin, james are the users of this group and these names must be similar to the key name stored in keydir directory.
For example “jatin” user must have the public key named “jatin.pub”

Let’s have a quick test of our setup

$ git commit conf/gitolite.conf -m “added opstreeblog repo”
 
[master 357bbc8] added “opstreeblog” repo
 
1 files changed, 9 insertions(+), 1 deletions(-)
 
nitin@Latitude-3460:~/gitolite-admin$ git push origin master
 
Counting objects: 7, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 428 bytes, done.
Total
4 (delta 0), reused 0 (delta 0)
remote: Initialized empty Git repository in /home/gitolite/repositories/opstreeblog.git/
To gitbox:gitolite-admin d595439..357bbc8
master -> master
 
I hope that gives you a good overview of how to install and manage Gitolite.

Stunnel a Proxy to ship the log on SSL

Introduction

p { margin-bottom: 0.25cm; line-height: 120%; }a:link { }

Few days ago I got a task to create the SSL connection with logstash redis plug-in with Azure Redis. As we are shipping the logs form the several data center to the Azure Redis. So logs must be shipped on SSL connection. There is no provision to create SSL connection through logstash redis plug-in thats why logstash redis plug-in is not able to make SSL connection with Azure redis.To resolve this problem we have to setup the stunnel as proxy front of the logstash redis plug-in. Stunnel can create SSL connection with Azure redis and Stunnel provide non-ssl connection for the logstash redis plug-in.

p { margin-bottom: 0.25cm; line-height: 120%; }a:link { Azure redis provide two type of connections, SSL on 6380 port and non SSL on 6379 port and also provide primary and secondary key(password).

Installation

Install Stunnel on the ubuntu.

$ sudo apt-get install stunnel  

Configuration

Create a configuration for stunnel /etc/stunnel/stunnel.conf and put the following lines into the configure file.

setuid = root
setgid = root
pid = /var/run/stunnel-azureredis.pid
debug = 7
output = /var/log/stunnel4/azureredis.log
options = NO_SSLv2
options = NO_SSLv3
[azureredis]
accept=127.0.0.1:6379
connect=:6380
client=yes
Timeout idle = 30

p { margin-bottom: 0.25cm; line-height: 120%; }a:link { }

Restart the stunnel4
p { margin-bottom: 0.25cm; line-height: 120%; }a:link { }
$ service stunnel4 restart

Install Redis-cli to test the connection try to ping the redis azure it should reply as PONG

$ redis-cli -a
127.0.0.1:6379> ping
PONG

Now you can make SSL connection with azure redis on SSL.

Setup of Nginx Vhost

 

Introduction

This is an ancient way of setting up Vhost with nginx. As  we have Chef to automate everything. But before a kickoff with automation using chef, it’s crucial to interpret our problem statement by dealing with it manually.
 
“Choose Older Eggs For Hard Cooking Maria Simmons”

Problem Statement

NGINX is a free, open-source, high-performance HTTP server. Install nginx manually using package manager, and configure virtual host for opstree.com/blog/, chef.opstree.com.

Prerequisites

This exercise considers that you have a basic understanding of Git, and Vagrant. This blog deal with centos7.
 

Install Nginx

Clone our github repository and spin up a bare centos7 vagrant machine.
 
$ git clone git@github.com:OpsTree/Chef.git
 
Go to nginxVhost directory.
 
$ cd  Chef/centos/nginxVhost
 
  • This directory have a Vagrantfile. Which can initiate a centos7 vagrant box with 512mb ram.
 
$ cat Vagrantfile
 
 
This file update and install some basic tools in your vagrant machine using vagrant shell provisioning.
 
  • Launch new vagrant machine and login into it via ssh.
$ vagrant up
 
$ vagrant ssh
 
  • Add nginx repo
As nginx is not available in default list of centos7, we add nginx repo to it.
 
$ sudo yum install -y epel-release
 
  • Install nginx
Install nginx using package manager “yum”.
 
$ sudo yum install -y nginx
 
  • Start nginx
Nginx do not start on its own. Type following to start it.
 
$ sudo service nginx start

Setup Vhost

Let’s go ahead with our problem statement of setting up vhost with nginx. This leads some dull steps to serve our webpages with opstree.com/blog/ and chef.opstree.com.
 
  • Replace nginx.conf file with given nginx.conf file.
 
$ sudo cp /vagrant/nginx.conf /etc/nginx/nginx.conf
 
  • Copy opstree.com/blog/ and chef.opstree.com into the /etc/nginx/conf.d directory
$ sudo cp /vagrant/opstree.com/blog/ /etc/nginx/conf.d/opstree.com/blog/
 
$ sudo cp /vagrant/chef.opstree.com /etc/nginx/conf.d/chef.opstree.com
 
  • Create home directory for vhost.
$ sudo mkdir /usr/share/nginx/blog
$ sudo mkdir /usr/share/nginx/chef
 
  • Create index files.
$ sudo su -c “echo \”Welcome, this is opstree.com/blog/\” > /usr/share/nginx/blog/index.html”
$ sudo su -c “echo \”Welcome, this is chef.opstree.com\” > /usr/share/nginx/chef/index.html”
 
  • Make entry in /etc/hosts
$ sudo vim /etc/hosts
 
127.0.0.1 opstree.com/blog/
127.0.0.1 chef.opstree.com
 
  • Restart nginx server
 
$ sudo service nginx restart
 
  • Access and test your  Vhost
 
$ curl opstree.com/blog/
 
$ curl  chef.opstree.com
You have done all the tiring stuff to set up Nginx Vhost. 
 
“Don’t let a bad day make you feel like you have a bad life.”
 
We understand your hard labor, so in next blog we go ahead to automate all the stuff.   

jgit-flow maven plugin to Release Java Application

Introduction

As a DevOps I need a smooth way to release the java application, so I compared two maven plugin that are used to release the java application and in the end I found that Jgit-flow plugin is far better than maven-release plugin on the basis of following points:
  • Maven-release plugin creates .backup and release.properties files to your working directory which can be committed mistakenly, when they should not be. jgit-flow maven plugin doesn’t create these files or any other file in your working directory.
  • Maven-release plugin create two tags.
  • Maven-release plugin does a build in the prepare goal and a build in the perform goal causing tests to run 2 times but jgit-flow maven plugin builds project once so tests run only once.
  • If something goes wrong during the maven plugin execution, It become very tough to roll it back, on the other hand jgit-flow maven plugin makes all changes into the branch and if you want to roll back just delete that branch.
  • jgit-flow maven plugin doesn’t run site-deploy
  • jgit-flow maven plugin provides option to turn on/off maven deployment
  • jgit-flow maven plugin provides option to turn on/off remote pushes/tagging
  • jgit-flow maven plugin keeps the master branch always at latest release version.
Now let’s see how to integrate Jgit-flow maven plugin and use it

How to use Jgit-flow maven Plugin for Release

Follow the flowing steps
    1. Add the following lines in your pom.xml for source code management access
      
         scm:git:
        scm:git:git:
    2. Add these line to resolve the Jgit-flow maven plugin and put the other option that will be required during the build
      com.atlassian.maven.plugins
      maven-jgitflow-plugin
      1.0-m4.3
              
      true
      false
      true
      true
      true
      true
      true
      true
                
      master-test
      deploy-test       

Above code snippet will perform following steps:

    • Maven will resolve the jgitflow plug-in dependency
    • In the configuration section, we describe how jgit-flow plug-in will behave.
    • pushRelease XML tag to enable and disable jgit-flow from releasing the intermediate branches into the git or not.
    • keepBranch XML tag to enable and disable the plug-in for keep the intermediate branch or not.
    • noTag XMl tag to enable and disable the plug-in to create the that tag in git.
    • allowUntracked XML tag to whether allow untracked file during the checking.
    • flowInitContext XML tag is used to override the default and branch name of the jgit-flow plug-in
    • In above code snippet, there is only two branches, master from where that code will be pulled and a intermediate branch that will be used by the jgit-flow plug-in. as I have discussed that jgit-flow plug-in uses the branches to keep it records. so development branch will be created by the plug-in that resides in the local not remotely, to track the release version etc.
  1. To put your releases into the repository manager add these lines
    <distributionManagement>
      <repository>
        <id><auth id></id>
        <url><repo url of repository managers></url>
      </repository>
      <snapshotRepository>
        <id><auth id></id>
        <url><repo url of repository managers></url>
      </snapshotRepository>
    </distributionManagement>
  2. Put the following lines into your m2/settings.xml with your repository manager credentials
    <settings>
      <servers>
        <server>
            <id><PUT THE ID OF THE REPOSITORY OR SNAPSHOTS ID HERE></id>
           <username><USERNAME></username>
           <password><PASSWORD></password>
        </server>
      </servers>
    </settings>

Start Release jgit-flow maven plugin command

To start the new release execute jgitflow:release-start.

Finish Release jgit-flow maven plugin  command

To finish new release, execute mvn jgitflow:release-finish.
For a example I have created a repository in github.com. for testing and two branch master-test and deploy-test. It is assumed that you have configured maven and git your system.

In the deploy-test branch run following command
$ mvn clean -Dmaven.test.skip=true install jgitflow:release-start

This command will take input from you for release version and create a release branch with release/. then it will push this release branch into github repository for temporarily because we are not saving the intermediate branched

Now At the end run this command
$ mvn -Dmaven.test.skip=true jgitflow:release-finish
after finishing this command it will delete release/ from local and remote.

Now you can check the changes in pom file by jgitflow. in the above snapshot, it is master-test branch, you can see in the tag it has removed the snapshot and also increased the version.  It hold the current version of the application.

And in the deploy-test branch it show you new branch on which developers are working on