How do I upgrade to Terraform 12 incrementally?
I am on a legacy project that uses Gruntwork with an old version of Terraform and Terragrunt (0.11.x and 0.18.x). I would like to upgrade to the latest version, but I have too much infrastructure to do it all at once. Is there a way I can incrementally update my projects?
There is a process we have identified for a partial migration that has worked well with a few of our customers. It involves upgrading to terragrunt 19+ first, and then taking on the terraform 12 upgrade. The reason for doing this is that the newer terragrunt features allow you to better manage cross module dependencies that are not bound by the terraform version (`terraform_remote_state` has limitations where you can't read state that is written by a different terraform version). Here is the rough process: **Incrementally upgrade to using terragrunt 0.19+** Terragrunt 0.19 has a bunch of features that support mixed terraform versions, so your first step is to upgrade to the latest terragrunt to take advantage of those features. Here is a process for how to do this incrementally in your repo: - Prerequisite: Install `tgswitch` and `tfswitch`. These tools have a bunch of nice features that make it easy to manage mixed version repos. See our blog post [How to manage multiple versions of terragrunt and terraform as a team](https://blog.gruntwork.io/how-to-manage-multiple-versions-of-terragrunt-and-terraform-as-a-team-in-your-iac-project-da5b59209f2d) for more info. - Prerequisite: Install terragrunt 0.18 and 0.23 using `tgswitch` - tgenv 0.18.7 - tgenv 0.23.31 - Prerequisite: Install terraform 0.11 and 0.12 using `tfswitch` - tfenv 0.11.15 - tfenv 0.12.31 - Prerequisite: Add .tgswitchrc and .tfswitchrc files at root of repo, pointing to `0.18.7` and `0.11.15` respectively. This ensures that the default version used for your modules will be tf11/tg18. Once the prerequisites are done, you are ready to incrementally switch to tg19+: - Make a copy of your root `terraform.tfvars` file, and convert it to `terragrunt.hcl`. - Update any `tfvars` in your directory structure (e.g. `region.tfvars`) to use yaml files. Replace the relevant `extra_arguments` subblocks in the root `terragrunt.hcl` with the yaml pattern as described in [our upgrade guide](https://github.com/gruntwork-io/docs/blob/legacy/content/guides/upgrading-to-tf12-tg19/index.md). - For each module: 1. Replace `terraform.tfvars` with `terragrunt.hcl` file. 1. Add `terraform_version_constraint = ">= 0.11"` to `terragrunt.hcl`. 1. Add `.tgswitchrc` in the upgraded module to point to 0.23.31. 1. Verify `terragrunt plan` works and no changes are reported 1. Open PR and merge to trunk. - Once all modules are upgraded to terragrunt 0.23, replace the root `.tgswitchrc` file to point to `0.23.31`. **Switch to using dependency blocks** Terragrunt 0.19 introduced a new feature for passing outputs across modules that doesn't depend on the terraform version and remote state: [dependency blocks](https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/#passing-outputs-between-modules). Using this feature, it is possible to hook up the outputs of a module using terraform 11 with a module using terraform 12. This is important because, as mentioned above, `terraform_remote_state` data sources are terraform version sensitive. However, this is a big change for the module so you will want to do this incrementally, in isolation of terraform 12 changes. - For each module that has a dependency, update the module to replace `terraform_remote_state` usage to instead take in variables of the ARN/ID of the resources. Use data sources to your advantage to limit the amount of dependency inputs. - Introduce `dependency` blocks in `terragrunt.hcl` to pull in the dependency information from other modules. Note that if a module doesn't export the output you need to feed to the data source, you will need to add this into the upstream modules. - Run `terragrunt plan` and `terragrunt apply` to ensure the changes are saved in state. - Open PR and merge to trunk. - Repeat for every module that has dependencies. **Upgrade modules to terraform 12** Once terragrunt is upgraded and the dependency blocks are introduced, you can start to have mixed terraform version modules. At this point, you can start to incrementally upgrade your modules to terraform 0.12, in a similar fashion to terragrunt: - Follow https://github.com/gruntwork-io/docs/blob/legacy/content/guides/upgrading-to-tf12-tg19/index.md to upgrade the module to terraform 12 - Add `.tfswitchrc` in the upgraded module to point to `0.12.29`. - Remove the `terraform_version_constraint` attribute in `terragrunt.hcl`. - Run `terragrunt plan` and `terragrunt apply` to ensure the changes are saved in state. - Open PR and merge to trunk. - Repeat for every module.