{"id":2831,"date":"2020-05-19T12:48:45","date_gmt":"2020-05-19T07:18:45","guid":{"rendered":"https:\/\/opstree.com\/blog\/\/?p=2831"},"modified":"2020-06-09T19:09:27","modified_gmt":"2020-06-09T13:39:27","slug":"terraforming-the-better-way-part-i","status":"publish","type":"post","link":"https:\/\/opstree.com\/blog\/2020\/05\/19\/terraforming-the-better-way-part-i\/","title":{"rendered":"Terraforming The Better Way: Part-I"},"content":{"rendered":"\r\n<figure class=\"wp-block-gallery aligncenter columns-1 is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\r\n<figure><\/figure>\r\n<ul class=\"blocks-gallery-grid\">\r\n<li class=\"blocks-gallery-item\">\r\n<figure><img loading=\"lazy\" decoding=\"async\" width=\"960\" height=\"540\" class=\"wp-image-2998\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2020\/05\/terraforming-better-way.jpg?w=840\" alt=\"\" data-id=\"2998\" \/><\/figure>\r\n<\/li>\r\n<\/ul>\r\n<\/figure>\r\n\r\n\r\n\r\n<p>We often face complications after a certain point when we can not change the foundation layer of our code because we haven\u2019t thought it through and didn\u2019t plan or strategize the way of writing code in the beginning,\u00a0there are certain points which should be taken under consideration similarly there are some common mistakes which we should avoid.<\/p>\r\n<p><!--more--><\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>After managing numerous Infrastructures as a code with a plethora of terraform modules, We at <a href=\"https:\/\/opstree.com\/\">Opstree<\/a> have build up a decent maturity with Terraform and have a defined workflow with best practices. In this blog post, I will be sharing the learning which I acquired using Terraform for diverse Infrastructures.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\">Versioning with Git<\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>Version control is the essential requirement to keep track of every change to a file over time so early\u00a0<em>versions<\/em>\u00a0can be restored and are used by your teams for source\u00a0<em>code<\/em>.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>Ensure that you have a proper <code>.gitignore<\/code>, <code>.env<\/code>, <code>README.md<\/code> files in your repositories.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li><code>.<a href=\"https:\/\/git-scm.com\/docs\/gitignore\" target=\"_blank\" rel=\"noopener\">gitignore<\/a><\/code> : Mention the files which you want to ignore, in this case, it could be <code>.env<\/code>, <code>*.tfstate.*<\/code> files and, <code>.terraform<\/code> directory.<\/li>\r\n<li><code>.env<\/code> : In this file, you can manage your environment variables going to be used in the code.<\/li>\r\n<li><code><a href=\"https:\/\/en.wikipedia.org\/wiki\/README\" target=\"_blank\" rel=\"noopener\">README.md<\/a><\/code>: It is an essential part of the code which has the document of usage of the code, it helps the user to get a proper understanding of the code also it helps in easy onboarding of new members in the team.<\/li>\r\n<li><code><a href=\"https:\/\/keepachangelog.com\/en\/1.0.0\/\" target=\"_blank\" rel=\"noopener\">CHANGELOG.md<\/a><\/code>: In the case of Terraform modules one should maintain a proper release history with CHANGELOG.md.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\">Backend got your back<\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>Whenever you run terraform to create or update your infrastructure it stores the state of resource it has created in a readable JSON file. Which has a mapping of your resources present in the <code>.tf<\/code> files of your code.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>If you are using terraform individually then keeping the state file in local could be agreeable, but it has it&#8217;s own drawbacks like if your system underwent failure then your state will be lost and there is no way getting it back. Also if managing Terraform among large teams you want to keep it at a sharable space. That sharable space must have functionalities like:<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Regular update: We want the state files to be updated automatically whenever a change takes place to avoid any manual error.<\/li>\r\n<li>Locking: The state files need to be locked when two or more people are simultaneously running terraform on the same state file.<\/li>\r\n<li>Encryption: The state file consists of a lot of secrets related to the infrastructure which can not be exposed.<br \/><br \/>Clearly a VCS is not a wise choice to keep the state files considering the above requirements. Hence terraform has its own functionality called remote backends to store the state files. The below snippet of creating an s3 bucket will do the needful for using it as a backend.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">resource \"aws_s3_bucket\" \"terraform_state\" {\r\n  bucket = \"terraform-up-and-running-state\"\r\n  \r\n  # Enable versioning of state files\r\n  versioning {\r\n    enabled = true\r\n  }\r\n  # To prevent deletion of the S3 bucket\r\n  lifecycle {\r\n    prevent_destroy = true\r\n  }\r\n\r\n  # To enable server-side encryption by default\r\n  server_side_encryption_configuration {\r\n    rule {\r\n      apply_server_side_encryption_by_default {\r\n        sse_algorithm = \"AES256\"\r\n      }\r\n    }\r\n  }\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<p>&nbsp;<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\">Logical separation of components<\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>Let us consider an example in which we&#8217;ll be creating various components of an Infrastructure<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Networking Segment in which we&#8217;ll be creating various resources in our infrastructure like VPC, IGW, Subnets, NAT, NACL, Route tables, etc which we can entitle as a networking segment of our Infrastructure.<\/li>\r\n<li>A Highly available frontend application, for that we\u2019ll be creating resources like Route53 record, Load balancers, Launch templates, Autoscaling groups, EC2 instances, Security Groups, etc.<\/li>\r\n<li>Backend services, Databases, Middleware, and so on&#8230;<br \/><br \/>Considering the above examples if we want to cause a change in the frontend application then we need not worry about the networking segment because the code of both segments is already separated and are independent of the changes happening among them. So categorizing the code will be beneficial for hassle-free management of resources.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\">Hierarchy in module: child, parent !!<\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>It is an unsaid rule in development that if we are using a chunk of code repeatedly, then create a function of it and call it whenever needed. A similar approach is well-liked in terraform called Modules.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<figure class=\"wp-block-gallery columns-1 is-cropped wp-block-gallery-2 is-layout-flex wp-block-gallery-is-layout-flex\">\r\n<ul class=\"blocks-gallery-grid\">\r\n<li class=\"blocks-gallery-item\">\r\n<figure><img loading=\"lazy\" decoding=\"async\" width=\"960\" height=\"540\" class=\"wp-image-3011\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2020\/05\/terraforming-better-way-2-1.jpg?w=960\" alt=\"\" data-id=\"3011\" data-link=\"https:\/\/opstree.com\/blog\/\/terraforming-better-way-2-1\/\" \/><\/figure>\r\n<\/li>\r\n<\/ul>\r\n<\/figure>\r\n<p>\r\n\r\n<\/p>\r\n<p>In the above diagram you can see multiple layers used to create modules, let me explain the significance of each layer.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Module Layer-I: In this layer, we have modules of each and every resource which is going to be used in the bigger scheme of creating infra. You conclude is as the building block of our infrastructure as suggested in the below-mentioned snippet.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\"># Layer-I \r\n$  cat vpc.tf\r\nresource \"aws_vpc\" \"main\" {\r\n  ......\r\n}\r\n$  cat subnet.tf\r\nresource \"aws_subnet\" \"subnet_1\" {\r\n  ....\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Module Layer-II: In this layer, we have our customized modules with respect to the use-cases, for eg: to have networking in place, the layer-II networking component will be calling the modules from layer-I, the below code snippet will give better clarity.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">$  cat network_component.tf\r\n# Layer-II Network Component\r\nmodule \"vpc\" {\r\n  source = \"Layer-I\/vpc\"\r\n  .....  \r\n}\r\nmodule \"subnet\" {\r\n  source = \"Layer-I\/subnet\"\r\n  .....  \r\n}\r\n......\r\n<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\">Proper usage of variables<\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>Terraform has it&#8217;s own variable precedence as mentioned in below figure.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"656\" height=\"526\" class=\"wp-image-3020\" src=\"https:\/\/opstree.com\/blog\/\/wp-content\/uploads\/2020\/05\/image-7.png?w=656\" alt=\"\" \/><\/figure>\r\n<p>\r\n\r\n<\/p>\r\n<p>As you can see there are various ways to define variables in terraform depending upon the use cases, there are a couple of practices that can be followed to improve the way we define variables.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Locals: If there are some values whose values are kept to be constant try not putting them as variables define them as local, moreover you can create functions or ternary operators and use the resulting values in a couple of places, as shown in below snippet.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">locals {\r\n  public_dir_with_leading_slash = \"${length(var.public_dir) &gt; 0 ? \"\/${var.public_dir}\" : \"\"}\"\r\n  static_website_routing_rules = &lt;&lt;EOF\r\n[{\r\n    \"Condition\": {\r\n        \"KeyPrefixEquals\": \"${var.public_dir}\/${var.public_dir}\/\"\r\n    },\r\n    \"Redirect\": {\r\n        \"Protocol\": \"https\",\r\n        \"HostName\": \"${var.domain_name}\",\r\n        \"ReplaceKeyPrefixWith\": \"\",\r\n        \"HttpRedirectCode\": \"301\"\r\n    }\r\n}]\r\nEOF\r\n}\r\ndata \"aws_iam_policy_document\" \"static_website_read_with_secret\" {\r\n  statement {\r\n    sid       = \"1\"\r\n    actions   = [\"s3:GetObject\"]\r\n    resources = [\"${aws_s3_bucket.static_website.arn}${local.public_dir_with_leading_slash}\/*\"]\r\n.....\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Validation: A set of validation can be incorporated while defining the variable so that to enforce the user to inject the specific syntactical value for eg: we know the AMI id syntax in AWS, so to ensure the input value to be such we can use below snippet.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">variable \"image_id\" {\r\n  type        = string\r\n  description = \"The id of the machine image (AMI) to use for the server.\"\r\n  validation {\r\n    # regex(...) fails if it cannot find a match\r\n    condition     = can(regex(\"^ami-\", var.image_id))\r\n    error_message = \"The image_id value must be a valid AMI id, starting with \\\"ami-\\\".\"\r\n  }\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<h5 class=\"wp-block-heading\"><strong>Secrets<\/strong>\u00a0better kept as\u00a0<strong>secrets<\/strong><\/h5>\r\n<p>\r\n\r\n<\/p>\r\n<p>We are well aware that terraform makes API calls to the cloud provider, hence it needs authentication and authoriZation to make any changes. This is done by using secret access keys provided by the provider, these secret access keys supposed to be used in a way so that they should not be a part of your code. As we say DO NOT COMMIT SECRETS IN VCS.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>Hence there are a couple of ways you can manage your secrets access keys.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Environment file(<code>.env<\/code>): We can keep our secret access keys in <code>.env<\/code> and mention this file in <code>.gitignore<\/code> to avoid havoc.<\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">$.  cat .env\r\nunset \"${!TF_VAR_@}\"\r\nexport TF_VAR_aws_access_key_id=*****************\r\nexport TF_VAR_aws_secret_access_key=**************\r\n\r\n$.  cat main.tf\r\nprovider \"aws\" {\r\n  region     = \"us-west-2\"\r\n  access_key = var.aws_access_key_id\r\n  secret_key = var.aws_secret_access_key\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<ul>\r\n<li>Using Shared credentials file: The better way that the above to have secrets in a file outside the code directory The default location is\u00a0<code>$HOME\/.aws\/credentials<\/code>\u00a0on Linux and OS X, or\u00a0<code>\"%USERPROFILE%\\.aws\\credentials\"<\/code><\/li>\r\n<\/ul>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">$  cat \/Users\/tf_user\/.aws\/creds\r\n[customprofile]\r\naws_access_key_id = ******************\r\naws_secret_access_key = *****************\r\n\r\n$  cat main.tf \r\nprovider \"aws\" {\r\n  region                  = \"us-west-2\"\r\n  shared_credentials_file = \"\/Users\/tf_user\/.aws\/creds\"\r\n  profile                 = \"customprofile\"\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<p>Another secret could be of your <code>aws_db_instance<\/code> the password which can be randomly generated and it&#8217;s output can also be encrypted as shown in the below example.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<pre class=\"wp-block-syntaxhighlighter-code\">$  cat main.tf\r\n# Password generator\r\nresource \"random_password\" \"postgres_admin_password\" {\r\n  length  = 26\r\n  special = false\r\n}\r\nresource \"aws_db_instance\" \"default\" {\r\n  identifier = var.name\r\n  ...\r\n  ...\r\n  password = random_password.postgres_admin_password.result\r\n}\r\n\r\n$  cat output.tf\r\noutput \"db_password\" {\r\n  value       = aws_db_instance.db.password\r\n  description = \"The password for logging in to the database.\"\r\n  sensitive   = true\r\n}<\/pre>\r\n<p>\r\n\r\n<\/p>\r\n<p>The <code>sensitive<\/code> flag in the output code will encrypt the password and can be further used in the rest of the code.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>In the next part of this blog, I&#8217;ll be explaining how to use terraform in a <strong>team<\/strong> at a scale with appropriate <strong>automation<\/strong> and <strong>workflow<\/strong>. Stay tuned!!!<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>If you got any feedback or queries regarding the blog please leave your comment below I\u2019d really appreciate it.<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>Thank you for reading \ud83d\ude42<\/p>\r\n<p>&nbsp;<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\r\n<div class=\"wp-block-button\">\r\n<p>Opstree is an End to End DevOps solution provider<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\r\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link\" title=\"https:\/\/www.opstree.com\/contact-us\" href=\"https:\/\/www.opstree.com\/contact-us\" target=\"_blank\" rel=\"noopener\">contact us<\/a><\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<p>\r\n\r\n<\/p>\r\n<p>&nbsp;<\/p>\r\n<p>\r\n\r\n<\/p>\r\n<p>&nbsp;<\/p>\r\n<p><\/p>","protected":false},"excerpt":{"rendered":"<p>We often face complications after a certain point when we can not change the foundation layer of our code because we haven\u2019t thought it through and didn\u2019t plan or strategize the way of writing code in the beginning,\u00a0there are certain points which should be taken under consideration similarly there are some common mistakes which we &hellip; <a href=\"https:\/\/opstree.com\/blog\/2020\/05\/19\/terraforming-the-better-way-part-i\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Terraforming The Better Way: Part-I&#8221;<\/span><\/a><\/p>\n","protected":false},"author":159458168,"featured_media":29900,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[28070474],"tags":[768739294,2185,768739308,3021235,18187],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/opstree.com\/blog\/wp-content\/uploads\/2025\/11\/DevSecOps-1.jpg","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pfDBOm-JF","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/2831"}],"collection":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/users\/159458168"}],"replies":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/comments?post=2831"}],"version-history":[{"count":25,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/2831\/revisions"}],"predecessor-version":[{"id":3371,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/posts\/2831\/revisions\/3371"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media\/29900"}],"wp:attachment":[{"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/media?parent=2831"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/categories?post=2831"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/opstree.com\/blog\/wp-json\/wp\/v2\/tags?post=2831"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}