Linux Utility to manage login to systems

One of the problem I used to have as build & release engineer is to manage login to huge number of boxes through my Linux system. At the scale of 5-10 machine it’s a not a big problem but once you have close to 100+ boxes then it is not humanly possible to remember the ip’s of those boxes.
The usual approach for this problem is to maintain a reference file, from where you map machine name with the ip & find the ip of the box from this file, but again after some time this solution seems to be not that efficient. Another solution is to have a DNS server where you can store such mappings & then you can access these machines using their names only, this is the idle solution but what if you don’t have DNS server also still you have to execute the ssh command ‘ssh user@machine”.

I developed a simple solution for this problem, I created a utility script connect.sh, this script takes machine name as an argument & then we have multiple conditions statements which checks which ssh command to be executed for the machine name.

#!/bin/bash
if [ “mc1” == $1 ]; then
    ssh user@
elif [ “mc2” == $1 ]; then
    ssh user@
elif [ “mc3” == $1 ]; then
    ssh user@
.
.
.
fi

This solution worked really well for me as now I’m saved from executing whole ssh command, also for machine name I’ve followed a convention i.e _ for example the entry for a machine for release environment that is hosting an application catalog the machine name would be release_catalog, similarly dev_catalog, staging_catalog, pt_catalog.. so you don’t have to remember machine names as well :).

Automated DB Updater Release First Release

Initial version of Automated DB Updater Release ADU

With this blog I’m releasing the intial version of a python utility to provide automated db updates across various environments for different components.

The code for this utility is hosted on github
https://github.com/sandy724/ADU

You can clone the read only copy of this codebase by url given below
https://github.com/sandy724/ADU.git

To understand the basic idea about this utility go thorugh this blog
http://sandy4blogs.blogspot.in/2013/07/automated-db-updater.html

How to use this utility
Checkout the code at some directory, add the path of this directory in PYTHONPATH environment variable
Create a database with a script’s metadata table with given below ddl

CREATE TABLE `script_metadata` (
  `name` varchar(100) DEFAULT NOT NULL,
  `version` int(11) DEFAULT NOT NULL,
  `executed` tinyint(1) NOT NULL DEFAULT ‘0’,
  `env` varchar(30) DEFAULT NOT NULL,
  `releas` varchar(30) DEFAULT NOT NULL,
  `component` varchar(30) DEFAULT NOT NULL
)
Create a database.properties, containing connection properties of each environment database

[common_db]
dbHost=localhost
dbPort=3306
dbUser=root
dbPwd=root
db=test
 
 
[env1]
dbHost=localhost
dbPort=3306
dbUser=root
dbPwd=root
db=test

Here common_db represents connection to database which will contain metadata of scripts for monitoring

Now execute the pythong utility
Copy the client(updateDB.py) to directory of your choice, make sure that property configration file should also be at this directory
python updateDB.py -f -r –env

Automated DB Updater

In continuation with my blog series I’m finally introducing a automated db updater tool. You can read about the idea in previous blogs by going .

The short form of my tool is ADU(Automated DB Updater). Now some details about this tool

Each application will have database_script folder at the root level, this folder will contain folders corresponding to each release i.e release1, release2, release3…

A database release folder will contain

  • Meta file :sql_sequence.txt, this file will contain the sequence in which sql files will be executed, only files mentioned in this file will be entertained
  • SQL Files : A sql file must have a naming convention like this __.sql/__.sql

Process of automatic execution of scripts on an environment

  • Input
    • release_name : to figure out the folder from where scripts will be executed
    • environment : Environment on which scripts will be executed
  • Execution
    • sql_sequence.txt file will be read line by line having one sql file name in each line
    • The sql file will be verified whether it has been already executed or not
    • If the sql file is already executed then two conditions are verified
      • A new version of sql should be available
      • Undo version of last executed sql should be present
    • After execution of undo file the latest version of the sql file will be executed and the info is stored accordingly that it has been executed so that it will not be picked again
  • Validations & Boundary Conditions
    • All the files mentioned in sql_sequence.txt should exist.
    • Undo script should be present for all the versions of a sql file barring the latest version of sql file.
    • Undo script will only be executed if next version of script is available.

Very soon I’ll share the github url of this project keep waiting 🙂

Ubuntu Rest Assurer

Before I’ll start talking about this utility I would like to talk about the story behind the creation of this utility. Few days back we have a session ERGONOMICS about healthy lifestyle, one of the main thing was that a person should take a break after every 20 minutes. After the session I was having a chit chat about one of my colleague Rahul Narang & we were discussing about this 20:20:20 rule & then we thought about creating a utility that will force a person to leave his/her system after some stipulated time & that’s how the idea came about this utility.

So what this utility does
1.) It runs after every half hour or configured amount of time
2.) Prompts a snoozer dialog box which will allow a person to snooze the screen locking for some amount of time
3.) After that a lunching video will run for 10 seconds
4.) Once a lunching video completes the screen gets locked for a configured amount of time
5.) If user try to unlock the system before configured amount of time the utility will lock the screen again

This complete utility is created using shell script only, we have used couple of command to do that
vlc : To run a launcher video
zenity : To prompt a snooze dialogue box
gnome-screensaver-command : To operate on screen lock

You can find the source code of this utility at my github account
https://github.com/sandy724/REAS

Puppet module for setting up Multiple mongo’s with replication

In this blog I’ll be talking about a puppet module, that can be used to installing multiple mongo’s with replication on a single machine. Since I’m very new to puppet so you may find this module very crude, but it works :). Their were couple of puppet module already available but most of them are only for installing a single instance of mongo at a machine & I’ve a specific requirement of installing multiple instances of mongo having master slave replication between them. As I already said that this module may be quiet crude or basic so please bear with that & my approach may also seem a bit unconventional so please let me know what all can be improved in this module or how things could have been done in a better way.

So let’s start with the actual details first of this module is hosted on github(https://github.com/sandy724/Mongo), if you want to look at the source code you can clone it from github. For installing mongo you would be executing the command
puppet apply -e “class {mongo:port => , replSet => ,master => ,master_port => ,}”

Command for installing master
puppet apply -e “class {mongo:port => 27017, replSet => sdrepsetcommon,master => master, master_port => 27017,}”

Command for installing slave
puppet apply -e “class {mongo:port => 27018, replSet => sdrepsetcommon,master => slave,master_port => 27017,}”

Before going into the details what all this module is doing I will share some details of mongo

  • You can start mongo by executing mongod command
  • You can provide a configration file which contains details such as
    • log directory where mongo would be generating the logs
    • port at which mongo would be listening for requests
    • dbpath where mongo would be storing all the data
    • pidfilepath containg process id of mongo instance, that would be used to check whether mongo is running or not
    • replSet name of the replicaset
  • You need to have a mongo as a service installed in you system to start an instance of mongo
  • For replication you need to execute rs.initiate command on the master mongo
  • For adding another instance into replication you need to execute rs.add(“:”)  command on the master mongo
Now let’s go into more details what all this component does, I’ll be listing down all the steps in bulleted points
  • As you can figure out this module is expecting few parameters :
    • port : port at which mongo would be listening,
    • replSet : name of replicaset which would be used for managing replication
    • master : A string parameter which would signify whether the mongo setup is for master or slave
    • master_port : Port at which master instance of mongo would be listening
  • First of all we create a mongo user
  • Parent Log directory for the mongo instance is created if it doesn’t exists with mongo user as owner.
  • Mongo db directory is created under /data/mongo with a naming convention replSet_port, i.e if replSet parameter is sdrepsetcommon & port is 27017 then the data directory for this mongo instance would be  /data/mongo/sdrepsetcommon_27017. This directory would be owned by mongo user.
  • A mongo service would be installed if not already their.
  • A mongo restart shell script is also placed at the mongo db directory
  • A file is also placed under the mongo db directory that have a mongo command to setup replication, this file is created conditionally depending on whether we are setting up a master or slave instance.
  • Finally the replication command is executed on mongo server & restart script is also executed

This concludes the setting up of a mongo instance on a machine.

Just for more details to start mongo we are using mongod -f command, this configuration file is saved as a template & the mongo modules processes the template with the values passed & creates the desired mongod.conf. In our case we are evaluating following properties of mongod.conf : logpath, port, dbpath, pidfilepath, replSet