Tag: Release Process

Revert a patch in most awesome way

If you are a Release Engineer, System Admin or Support Engineer you have definitely come across a requirement where you have to apply patches to the target systems be it production or non-production. I’m assuming that you are using some automated system to manage the patches i.e applying them and reverting them. In this blog I would be discussing about the standard way of patch management and how you can have an out of the box solution to revert your patch in most simplistic way and without much fuss. At the end of the blog I would like to see an expression where you will say what the hell it’s so awesome yet so simple :).

People usually use some tool to apply patch to a target system which in addition to applying a patch also manage the history the patches so that it can be reverted in case the patch goes wrong. The patch history usually contains below details:

  1. The new files that were added in the patch, while reverting the patch those files should be deleted.
  2. The files that were deleted by the patch, while reverting the patch the deleted files should be restored back.
  3. The files that were modified by the patch, while reverting the patch the modified files should be restored back.
You can definitely create a tool that can revert the patch for you as the use cases are not much, but do you really need to put this much effort if you can have an out of the box solution for this. What if I tell you that we use git for managing our patch history and reverting them. As git comes with a local repository concept so we created a local git repository at our app server codebase location only. Git comes with all the file level tracking we map each patch with one git commit, so at the time of reverting a specific patch you can ask git to simply revert the commit for you.

Extra steps to be done after applying patch:
To make git track the changes done in patch, you just need to perform 2 extra commands

git add . : This command will track all the files that have been modififed, added or deleted in the system.
git commit -m “Applying Patch” : This command actually adds the files information tracked by previous command with a message in the git system

Steps to be done in reverting changes done by a patch:
Once you have all the information tracked in git it will become no-brainer to revert the patches.

To view the details of all the patches: You can use git log command that will provide you the list of all the patches that you have applied or reverts that you have done

sandy@sandy:~/test/app1$ git log
commit f622f1f97fc44f6897f9edc25f9c6aab8e425049
Author: sandy
Date:   Thu Jun 19 15:19:53 2014 +0530

    Patch 1 on release2

commit 9a1dd81c7799c2f83d897eed85914eecef304bf0
Author: sandy
Date:   Thu Jun 19 15:16:52 2014 +0530

    Release 2

commit 135e04c00b3c3d5bc868f7774a5f284c3eb8cb29
Author: sandy
Date:   Thu Jun 19 15:16:28 2014 +0530

  Release 1

Now Reverting a patch is as simple as executing a simple command git revert, with the commit id of the patch

git revert f622f1f97fc44f6897f9edc25f9c6aab8e425049
[master 0ba533f] q Revert "Patch 1 on release2"
 1 file changed, 1 deletion(-)

If you run git log command, you will see the patch revert history as well

sandy@sandy:~/test/app1$ git log
commit 0ba533fda95ed4d7fcf0b7e6b23cd1a5589946a7
Author: sandy
Date:   Thu Jun 19 15:20:24 2014 +0530

    Revert "Patch 1 on release2"

    This reverts commit f622f1f97fc44f6897f9edc25f9c6aab8e425049.commit f622f1f97fc44f6897f9edc25f9c6aab8e425049
Author: sandy
Date:   Thu Jun 19 15:19:53 2014 +0530

    Patch 1 on release2

commit 9a1dd81c7799c2f83d897eed85914eecef304bf0
Author: sandy
Date:   Thu Jun 19 15:16:52 2014 +0530

    Release 2

commit 135e04c00b3c3d5bc868f7774a5f284c3eb8cb29
Author: sandy
Date:   Thu Jun 19 15:16:28 2014 +0530

    Release 1

I hope this blog has given you a very different perspective of managing the patches, let me know your thoughts about this. Also if you have such more ideas do share with me.

Initial thoughts for a patch framework for a java based web project

Although this blog was not in the pipeline of feb month but I got a requirement to build a patch framework for a java based web project, so along with building this framework I thought of writing this blog as well so that I’ll get idea from other people as well.

First of all I will talk about what can be patched using this patch framework, majorly it will be three resources/things that can be patched

  • Class Files
  • JSP’s
  • Static Resources such as images, css, js …

I’m thinking of adding few other features in this patch framework as well

  • Sequence of patches should be maintained : Since we have a big team around 80 developers working on same code-base, their may be a scenario that we can have two or more patches which needs to be applied to a target system. Their may be a fair chance that those patches have to be executed in a sequence or you can say their could be dependency among patches.
  • Validation while applying patches : One of the validation that I can think of is that the resources that have to be patched will be either new or existing one & in case of existing resources the system should verify the location at which resources are patched should already have those resources
  • Rollback : The patch framework should have rollback capability

I’m planning to build this patch framework using

  • Shell scripting : For programming aspect
  • Git : As a version control system for storing patches
  • Jenkins : Provide a front-end to user for applying patches
  • Mysql : Not so sure about it yet but I’ve to store few information such as what all patches are applied, sequence of patches…. I can use file systems as well for storing this information

Let me know your thought about  this framework or any other feature that you can think of

Automated Database Update Or Rollback

One of the important step during release is doing database update and rollback in case something goes wrong, usually people perform this operation manually. In this blog I’ll talk about how we can automate this process by following some convention.

Here I’m taking mysql database as an example we can have same conventions for other databases also

Convention to manage rollback/updates of a release

  • Each project codebase at it’s root will have a folder database_scripts
  • The database_scripts folder will contain folder for each release i.e Release1_1, Release2_0…
  • The database scripts release folder will in turn contains two folders update & rollback which will contain updates & rollbacks scripts for a release.

Automating the rollback/update

  • The update folder will have a source input file FileSequencer.txt. This file will point to all the update scripts in correct order that needs to be executed for the release
  • In the similar manner rollback folder will have a source input file FileSequencer.txt. This file will point to all the rollback scripts in correct order that needs to be executed for the release
  • At last we will have a utility shell script, this script will take db details and execute all the scripts referred in FileSequencer.txt using mysql command

Release Strategy for Java Web based projects

In this post I’ll be discussing about the 2 strategies that  we can follow for releasing a Java based web project.

A project can be primarily released in two ways
    Incremental Release
    Full Release

Incremental Release is done in big projects which has multiple modules & usually few modules gets updated between two releases. It makes sense to include only updated modules in release archive and during deployment only update those modules in application server.

Full release is usually done in small projects where the release archive contains all the components and then this release archive can be deployed to the application server as a whole

Both incremental & full release strategy has their pros & cons, where full release strategy scores in simple release archive generation & deployment incremental release has upper hand in space usage by only having modified components in it, although it brings overhead when doing rollback.

Release Steps in Incremental Release Strategy: If you are following incremental strategy in general you need to perform following steps

1.) Checkout the latest code for the release
2.) Generate the list of components which needs to be deloyed for the release
3.) Generate the release archive based on the list of components
4.) Stop the server(If hot deployment of components is not available)
5.) Take the backup of existing application on application server as we may need to do rollback in case of any issues
6.) Replace the components in application server with the components in the release archive
7.) Start the server(If hot deployment of components is not available)

Release steps in Full Release Strategy: As explained earlier Full release strategy is fairly simple, steps involved are:
1.) Checkout the latest code for the release
3.) Generate the release archive for whole application
4.) Stop the server(If hot deployment of components is not available)
6.) Deploy the release archive to the application server
7.) Start the server(If hot deployment of components is not available)