Chef Journey

I’m starting a blog series on chef where I would be taking you to a journey of managing my current infrastructure using Chef. To start with these are the high level tasks lists that I’ve in mind:

  • User Management : User’s creation or deletion on an environment(Dev/QA/Staging/Production) should be managed by chef, along with kind of access on the environment i.e read-only access, root access, or adding a user to some groups.
  • VPN Setup : Currently we are using openvpnas for managing secured access to our environment, it is manual right now so the vpn set-up will also be done by chef.
  • Apache Setup : We are using apache as web server that sits in front of our app server and also provides SSL.
  • Jar App : We have a SOA based set-up in which we have multiple micro java services, so we would be using chef to manage those jar app i.e deploying those jar app’s, starting/stopping those jar app’s.
  • Tomcat : Another major component type in our application are web apps that are hosted on tomcat server, the tomcat server is not managed as a service instead we create tomcat as an app user along with tomcat management scripts.
  • Mongo : We use replicated mongo as No SQL database in our application.
  • Logstash : For managing logs we are using log stash in a clustered set-up where all the log agents publish the logs to a central server and then served by Kibana, so this complete setup should also be managed by chef
  • ActiveMQ : We are using ActiveMQ for our queuing purpose

This list is not complete surely, I’ll be adding many more tasks in this list as I proceed in setting up my environment using chef as this is the first time I’ll be doing a set-up using Chef, but this list will be a good starting point.

Before jumping into creating the Chef cookbooks, runlists or data bags I’ve to setup the base infrastructure of Chef that is Chef Server to which all chef agents talk to, a chef workstation which would be updating the server with the configurations and a git repo to keep track of all my configuration as shown in the image given below.

In the next blog I’ll talk about how I’ll set-up a chef server. Let me know if you have any inputs for me or suggestion that how I should proceed with the chef set-up.

VPC per envrionvment versus Single VPC for all environments

This blog talks about the two possible ways of hosting your infrastructure in Cloud, though it will be more close to hosting on AWS as it is a real life example but this problem can be applied to any cloud infrastructure set-up. I’m just sharing my thoughts and pros & cons of both approaches but I would love to hear from the people reading this blog about their take as well what do they think.

Before jumping right away into the real talk I would like to give a bit of background on how I come up with this blog, I was working with a client in managing his cloud infrastructure where we had 4 environments dev, QA, Pre Production and Production and each environment had close to 20 instances, apart from applications instances there were some admin instances as well such as Icinga for monitoring, logstash for consolidating logs, Graphite Server to view the logs, VPN server to manage access of people.

 

At this point we got into a discussion that whether the current infrastructure set-up is the right one where we are having a separate VPC per environment or the ideal setup would have been a single VPC and the environments could have been separated by subnet’s i.e a pair of subnet(public private) for each environment

Both approaches had some pros & cons associated with them

Single VPC set-up

Pros:

  1. You only have a single VPC to manage
  2. You can consolidate your admin app’s such as Icinga, VPN server.

Cons:

  1. As you are separating your environments through subnets you need granular access control at your subnet level i.e instances in staging environment should not be allowed to talk to dev environment instances. Similarly you have to control access of people at granular level as well
  2. Scope of human error is high as all the instances will be on same VPC.

VPC per environment setup

Pros:

  1. You have a clear separation between your environments due to separate VPC’s.
  2. You will have finer access control on your environment as the access rules for VPC will effectively be access rules for your environments.
  3. As an admin it gives you a clear picture of your environments and you have an option to clone you complete environment very easily.

Cons:

  1. As mentioned in pros of Single VPC setup you are at some financial loss as you would be duplicating admin application’s across environments

In my opinion the decision of choosing a specific set-up largely depends on the scale of your environment if you have a small or even medium sized environment then you can have your infrastructure set-up as “All environments in single VPC”, in case of large set-up I strongly believe that VPC per environment set-up is the way to go.

Let me know your thoughts and also the points in favour or against of both of these approaches.

Chef Solo an Introduction

Introduction

Chef Solo is simple way to begin working with Chef. It is an open source version of the chef-client that allows using cookbooks with nodes without requiring access to a server. Chef Solo runs locally and requires that a cookbook (and any of its dependencies) be on the same physical disk as the node. It is a limited-functionality version of the chef-client and does not support the following:

  • Node data storage
  • Search indexes
  • Centralized distribution of cookbooks
  • A centralized API that interacts with and integrates infrastructure components
  • Authentication or authorization
  • Persistent attributes  

Installing chef-client  (Pre-requisite : curl )
Login to your box and run the following command to install the chef. Make sure that curl program is available on your box. 

 curl -L https://www.opscode.com/chef/install.sh | bash
cropinstall.jpg
To check if the installation was successful check the version of the installed chef-solo by:
 chef-solo -v

version.jpg                                  

Making Chef Repository
Next step is to setup a file structure that will help organize various Chef files. Opscode, the makers of Chef provide one sample structure. They call it simply the Chef Repository.

 wget http://github.com/opscode/chef-repo/tarball/master
structure.jpg 
 tar zxf master 
 mv opscode-chef-repo-**** chef-repo/
structure1.jpg
Assign cookbook’s path to the newly created cookbook directory inside the Chef Repository which will hold the cookbook

  mkdir .chef  
  echo "cookbook_path ['/root/chef-repo/cookbooks' ]" > .chef/knife.rb   
  knife cookbook site download apt
.Chef folder

For Chef Solo this directory generally contains only knife.rb file. A knife.rb file is used to specify the chef-repo-specific configuration details for Knife. This file is the default configuration file and is loaded every time this executable is run. The configuration file is located at: ~/.chef/knife.rb. If a
knife.rb file is present in the . chef/knife.rb directory in the chef-repo, the settings contained within that file will override the default configuration settings. Sample content of knife.rb file can be:
 cookbook_path [ '/root/chef-repo/cookbooks' ]  
 role_path [ '/root/chef-repo/roles' ]  
 environment_path [ ' /root/chef-repo/environments ' ]  
 data_bag_path [ ' /root/chef-repo/data_bags ' ]
Getting Started with Chef Solo
Before we’re able to run Chef Solo on our servers, we will need to add two files to our local Chef repository: solo.rb and node.json.
The solo.rb file tells Chef Solo where to find the cookbooks, roles, and data bags.

The node.json file sets the run list (and any other node-specific attributes if required).

     Create a solo.rb file inside our Chef repository with the following contents:
       current_dir = File.expand_path(File.dirname(__FILE__))  
 file_cache_path "#{current_dir}"  
 cookbook_path "#{current_dir}/cookbooks"  
 role_path "#{current_dir}/roles"  
 data_bag_path "#{current_dir}/data_bags"
      Create a file called node.json inside your Chef repository with the following contents:
       {  
      "run_list": [ "recipe[]" ]  
 }
                  

      Example:- 
      In this example i am going to install apt cookbook and the recipe which i am going to use is apt and here is my solo.rb and node.json files looks like
      solo1.jpg
      Our first Chef run  Goto chef-repo folder and execute following command

       chef-solo -c solo.rb -j node.json
      run1.jpg
       
      run_last1.jpg

      How it works:

      1. solo.rb configures Chef Solo to look for its cookbooks, roles, and data bags   inside the current directory: the Chef repository.
           2. Chef Solo takes its node configuration from a JSON file, in our example we simply        called it node.json. If we’re going to manage multiple servers, we’ll need a separate    &nbsp &nbsp &nbsp file.
      1. Then, Chef Solo just executes a Chef run based on the configuration data found in
                 solo.rb and node.json

      Revert a patch in most awesome way

      If you are a Release Engineer, System Admin or Support Engineer you have definitely come across a requirement where you have to apply patches to the target systems be it production or non-production. I’m assuming that you are using some automated system to manage the patches i.e applying them and reverting them. In this blog I would be discussing about the standard way of patch management and how you can have an out of the box solution to revert your patch in most simplistic way and without much fuss. At the end of the blog I would like to see an expression where you will say what the hell it’s so awesome yet so simple :).

      People usually use some tool to apply patch to a target system which in addition to applying a patch also manage the history the patches so that it can be reverted in case the patch goes wrong. The patch history usually contains below details:

      1. The new files that were added in the patch, while reverting the patch those files should be deleted.
      2. The files that were deleted by the patch, while reverting the patch the deleted files should be restored back.
      3. The files that were modified by the patch, while reverting the patch the modified files should be restored back.
      You can definitely create a tool that can revert the patch for you as the use cases are not much, but do you really need to put this much effort if you can have an out of the box solution for this. What if I tell you that we use git for managing our patch history and reverting them. As git comes with a local repository concept so we created a local git repository at our app server codebase location only. Git comes with all the file level tracking we map each patch with one git commit, so at the time of reverting a specific patch you can ask git to simply revert the commit for you.

      Extra steps to be done after applying patch:
      To make git track the changes done in patch, you just need to perform 2 extra commands

      git add . : This command will track all the files that have been modififed, added or deleted in the system.
      git commit -m “Applying Patch” : This command actually adds the files information tracked by previous command with a message in the git system

      Steps to be done in reverting changes done by a patch:
      Once you have all the information tracked in git it will become no-brainer to revert the patches.

      To view the details of all the patches: You can use git log command that will provide you the list of all the patches that you have applied or reverts that you have done

      sandy@sandy:~/test/app1$ git log
      commit f622f1f97fc44f6897f9edc25f9c6aab8e425049
      Author: sandy
      Date:   Thu Jun 19 15:19:53 2014 +0530

          Patch 1 on release2

      commit 9a1dd81c7799c2f83d897eed85914eecef304bf0
      Author: sandy
      Date:   Thu Jun 19 15:16:52 2014 +0530

          Release 2

      commit 135e04c00b3c3d5bc868f7774a5f284c3eb8cb29
      Author: sandy
      Date:   Thu Jun 19 15:16:28 2014 +0530

        Release 1

      Now Reverting a patch is as simple as executing a simple command git revert, with the commit id of the patch

      git revert f622f1f97fc44f6897f9edc25f9c6aab8e425049
      [master 0ba533f] q Revert "Patch 1 on release2"
       1 file changed, 1 deletion(-)

      If you run git log command, you will see the patch revert history as well

      sandy@sandy:~/test/app1$ git log
      commit 0ba533fda95ed4d7fcf0b7e6b23cd1a5589946a7
      Author: sandy
      Date:   Thu Jun 19 15:20:24 2014 +0530

          Revert "Patch 1 on release2"

          This reverts commit f622f1f97fc44f6897f9edc25f9c6aab8e425049.commit f622f1f97fc44f6897f9edc25f9c6aab8e425049
      Author: sandy
      Date:   Thu Jun 19 15:19:53 2014 +0530

          Patch 1 on release2

      commit 9a1dd81c7799c2f83d897eed85914eecef304bf0
      Author: sandy
      Date:   Thu Jun 19 15:16:52 2014 +0530

          Release 2

      commit 135e04c00b3c3d5bc868f7774a5f284c3eb8cb29
      Author: sandy
      Date:   Thu Jun 19 15:16:28 2014 +0530

          Release 1

      I hope this blog has given you a very different perspective of managing the patches, let me know your thoughts about this. Also if you have such more ideas do share with me.

      Win Free Ecopy of new book on ReviewBoard

      Readers would be pleased to know that I have teamed up with Packt Publishing  to organize a Giveaway of the Getting Started with Review Board

      And two lucky winners stand a chance to win ecopy of the book. Keep reading to find out how you can be one of the Lucky Winners.

      Overview of book:

      • Install and set up Reviewboard
      • Create a review request with the changes you have introduced
      • Publish or share the review request with the team/reviewer/reviewer groups
      • Integrate your code with code repositories
      • Close the code review request by providing a review comment
      • Understand how to search the user dashboard (limited and full text search)
      • Manage Reviewboard as an administrator
      • Acquire tips and tricks to optimize the usage and performance of Reviewboard

      How to Enter?

      All you need to do is head on over to the book page (Getting Started with Review Board) and look through the product description of the book and drop a line via the comments below this post to let us know what interests you the most about this book. It’s that simple.

      Deadline

      The contest will close on 5th of April 2014. Winners will be contacted by email, so be sure to use your real email address when you comment!