Managing infrastructure state using Git

Everything has its beginning

Working with a new environment has a very first stage which usually means that all configuration starts from the scratch. Putting this into the context of a cloud, the account setup phase may be also fully automated. The particular steps do not vary much from the post setup stage and one differentiator is state storage. Here, the state cannot be put into the cloud object storage due to the fact that it does not exist yet and is required to be created manually which is not wanted and breaks the chain of automation. One technique that is often practiced is to use a code repository.

State management using Git

In this case there is no out-of-the-box solution, however there are several ways to accomplish the task. One approach is to make use of Terraform backend defined as local, which translates into the usage of a local filesystem where all of the operations are performed, including state locking. As with other operations, this job of the infrastructure pipeline can also be fully automated. In most cases the state storage comes as a final step in the pipeline. For this purpose the git bot is utilized and often executed from the well known and containerized environment. It is worth mentioning that along the way zero trust security model shall be applied and all of the sensitive information must be safely stored in a vault and exchanged via secure mechanisms. Thus, the operations performed by the git bot may be fully automated and controllable by the respective permissions.

In an exemplary pipeline the infrastructure may be created in a fully automated manner keeping all good practices in place. This means that pipeline shall combine a set of jobs maintaining the quality of the configuration code and leads to automatic deployment to the non-production environments while keeping the control over the production environment which is deployed with human intervention. As shown in the picture below the last step of the entire sequence is state stage in git. This step is also fully automated and the updated information is checked-in to the remote code repository and selected branch. The operations leading to successful state storage are often executed as bash scripts from the automation environment.

Exemplary pipeline aimed for automation of two environments staging and production is shown below. Here, the execution environment is a GitLab Runner that controls and processes CI/CD jobs and sends back the results.

Conclusion

Infrastructure state management with a git seems to be a proven solution when bootstrapping new projects that require setup of infrastructure. This approach addresses the problem of managing all the components with a code without a need to manually create e.g. object storage buckets for persistence. It also may be beneficial in multi-cloud projects by providing portability. Note that when using Terraform there is one additional tool in the toolbox in the form of Terraform Cloud service which may be suitable in some cases. It results in the offloading of the infrastructure state manipulation and adding on top the possibility of collaboration within the team.

Looking for assistance with automation? Allow us to handle it using a DevOps approach.