Introduction
This is a quick blog on how we can use the TruffleHog utility in our Jenkins pipeline to search for the secrets, passwords, sensitive keys which may have been accidentally committed in our repositories.
TruffleHog proves to be a great tool in helping us to fetch the sensitive data from our repositories which we do not want to expose at any cost.
Before moving further with this blog, I would like you all to take a look at the prerequisites that are mentioned below.
Prerequisite:
- Basic understanding of Docker, Jenkins
- Basic shell commands
What is TruffleHog?
In simple words, TruffleHog is a utility that searches through git repositories for secrets, private keys and credentials so that you can protect your data before a breach occurs.
It is effective at finding secrets accidentally committed.
TruffleHog previously functioned by running entropy checks on git diffs. This functionality still exists, but high signal regex (regular expressions) checks have been added, and the ability to suppress entropy checks have also been added.
How we can use TruffleHog utility in our Jenkins pipeline?
Use case: Let’s suppose there is a requirement from your client end to figure out all the secrets, sensitive API keys which are present in the code repositories.
The steps below will guide you on how to setup TruffleHog to meet the above requirement.
In the following section, we will setup a Jenkins pipeline in which we will launch a Docker container with the help of Dockerfile and run our TruffleHog utility in that Docker container.
Let’s get started.
Step 1: Go to your Jenkins server and start creating a Freestyle project.
Step 2: In the general section of your pipeline, give a brief description of your pipeline. (Optional)
Step 3: Give a log rotation strategy to your pipeline. (Optional)
Step 4: Here, we are leaving the rest of the options to default values. You can change and give options according to your need.
Step 5: Under the source code management section, pass the repository URL. Your git repo contains a Dockerfile that will build your docker container.
URL: https://github.com/lakshayarora476/truffleHog.git
Pass the credentials for your repository. (if required)
Step 6: Under the ‘build’ section, execute the below shell commands:
Explanation of the above shell command:
cd /var/lib/jenkins/workspace/Jenkins-TruffleHog: We are moving to the workspace directory of Jenkins
docker stop $(docker ps -a -q): It will stop all the Docker containers which are currently in execution.
docker rm $(docker ps -a -q): It will remove all the Docker containers.
NOTE: The aim of this blog is to help you understand the concept of TruffleHog through the Jenkins pipeline. Hence, to make things simple, I am stopping and removing all the docker containers (under running state) so that we can focus only on the container running our TruffleHog utility.
docker build -t mypersonalimage . : It will build a Docker image from Dockerfile which we have cloned in our Jenkins pipeline. The name of your image will be ‘ mypersonalimage ‘
docker run -dit –name personalcontainer mypersonalimage: It will start the Docker container in detached mode from the previously built image. The name of your container will be ‘ personalcontainer ‘.
docker ps -a: Here we are listing all the Docker containers.
docker exec personalcontainer trufflehog –entropy=NO –regex –json https://github.com/lakshayarora476/truffleHog.git > output.txt :
Here, we are executing the ‘TruffleHog’ utility inside the docker container along with the git URL. It will run the TruffleHog utility against the git URL in the container and will output the result in the output.txt file.
NOTE: Here, the reason, why I have included entropy and regex parameters, is that TruffleHog by default finds too many false positives: random-looking text which may not be a secret. So, there is an alternative search mode that uses regular expressions (regex) instead of entropy heuristic.
cat output.txt |grep -oE “\”stringsFound\”\:.[.\”]}”|sed -e “s/,\”.]//” -e “s/}//”|sed “s/\”stringsFound\”://”|grep -o “\”.\””|awk -F “,” ‘{ for(i=1;i<=NF;i++) print $i}’ :
This command is used to format the output from TruffleHog.
NOTE: Without the above command, your output will look something like this:
Step 7: After your pipeline is executed successfully, you can go to console output and see the pipeline result.
It will list the secrets, API keys which looks something like this:
Though, version control systems make it easy for developers to work collaboratively to create software. By storing a history of who made what changes and when, it makes it easy to combine edits made by different people without losing either person’s work.
However, all this stored history has a downside – it’s all too easy for developers to commit information to the repository that shouldn’t be visible to everyone who has access to the source.
This could be files containing database passwords, deploy scripts including server credentials, or even the private key files for SSH or HTTPS. Removing the secret data from the current version doesn’t help, because the previous version is stored in the history and is still accessible. TruffleHog is one tool that makes it easier to search through the history of a git repository to discover passwords and other secrets.
This was a basic example to help you understand how truffle hog works. You can tweak the commands according to your own needs and start experimenting.
Happy Learning 🙂
Blog Pundit: Naveen Verma
Opstree is an End to End DevOps solution provider
Connect Us