Generate providers for each project in Terragrunt

Question

i try to generate providers based on location where I am. Here is the folder structure:

src/
├── environments/
│   ├── all/
│   │   └── environment.hcl
│   ├── dev/
│   │   └── environment.hcl
│   ├── test/
│   │   └── environment.hcl
│   └── prod/
│       └── environment.hcl
└── terraform/
    └── infra/
        ├── project1/
        │   ├── main.tf
        │   ├── variables.tf
        │   └── terragrunt.hcl
        ├── project2/
        │   ├── main.tf
        │   ├── variables.tf
        │   └── terragrunt.hcl
        └── terragrunt.hcl

My code in terraform/infra terragrunt file looks like this:

locals {
  wrk_path = trimsuffix(path_relative_to_include(), "/")

  all_path   = "${get_parent_terragrunt_dir()}/../environments/all"
  all_config = try(read_terragrunt_config("${local.all_path}/environment.hcl").inputs, {})
  env_path   = "${get_parent_terragrunt_dir()}/../environments/${local.env}"
  env_config = try(read_terragrunt_config("${local.env_path}/environment.hcl").inputs, {})
  fin_config = merge(local.all_config, local.env_config)

  all_var_file = "${local.all_path}/${local.wrk_path}.tfvars"
  env_var_file = "${get_parent_terragrunt_dir()}/../environments/${local.env}/${local.wrk_path}.tfvars"
}

remote_state {
    // backend
}


terraform {
  // stuff
  }

  extra_arguments "apply_environment" {
    commands = get_terraform_commands_that_need_vars()
    optional_var_files = [
      local.all_var_file, local.env_var_file
    ]
  }
}

inputs = merge(local.fin_config)

and then, in terraform/infra/project1 i want to reference settings from master terragrunt hcl and i did a code:

include "root" {
  path = find_in_parent_folders()
}

generate "provider" {
  path      = "providers.tf"
  if_exists = "overwrite"
  contents = <<EOF
provider "azurerm" {
  features {}
  subscription_id = "abc"
  skip_provider_registration = true
  alias = "${local.fin_config.law}"
}

provider "azurerm" {
  features {}
  subscription_id = "${local.fin_config.sub}"
  skip_provider_registration = true
}
EOF
}

But it looks like that config from my master terragrunt.hcl is not moved to "local (project1) terragrunt.hcl.

My question is, how to read common inputs in master terragrunt hcl file and use it in project's terragrunt.hcl file? Is it possible? if not, what is the best approach? When i place generate provider block into my master terragrunthcl providers are created in project1 and 2 but i want to have different providers set per project

Answer 1

You want the local.fin_config from your master terragrunt.hcl to be accessible in each project's terragrunt.hcl file.
But, as far as I know, configurations defined in the master file are not automatically available in the child configurations: Terragrunt does not natively support the inheritance of locals across configurations.

However, inputs defined in an included terragrunt.hcl file can be accessed by the child configuration. That means you can try and use inputs for shared configuration rather than locals.

Modify your master terragrunt.hcl in terraform/infra to output fin_config as an input that can be inherited by child configurations:

# In master terragrunt.hcl
locals {
  # Your locals remain the same
}

inputs = {
  fin_config = local.fin_config
}

In your project-specific terragrunt.hcl (e.g., within project1), you should be able to access fin_config through the inputs of the included configuration:

include "root" {
  path = find_in_parent_folders()
}

generate "provider" {
  path      = "providers.tf"
  if_exists = "overwrite"
  contents = <<EOF
provider "azurerm" {
  features {}
  subscription_id = "abc"
  skip_provider_registration = true
  alias = "\${include.root.inputs.fin_config.law}"
}

provider "azurerm" {
  features {}
  subscription_id = "\${include.root.inputs.fin_config.sub}"
  skip_provider_registration = true
}
EOF
}

For project-specific configurations (e.g., different Azure subscription IDs), you should define these directly within each project's terragrunt.hcl as inputs or manage them through additional .tfvars files specified in the extra_arguments block.

---

Attempt to get attribute from null value

The error should mean that, when the child configurations try to access fin_config via include.root.inputs.fin_config, the fin_config is not properly initialized or passed down as expected.

Terragrunt's include mechanism allows child configurations to inherit inputs from parent configurations, but this inheritance does not extend to the locals in the same straightforward way. When you try to pass locals from the parent configuration to the child through inputs, it is important that these locals are fully resolved and available at the time the child configurations are parsed.

If fin_config is dependent on any dynamic values or other locals that are not resolved in time, it may result in the child configurations receiving a null value when they attempt to access fin_config through include.root.inputs.fin_config.

A workaround for this issue involves making sure that any values you wish to pass down from the master to the child configurations are not dependent on unresolved locals, or are resolved before they are assigned to inputs.

The master terragrunt.hcl would be (simplified example):

# Define a local that you intend to pass down
locals {
  common_config = {
    law = "some-value",
    sub = "another-value"
  }
}

# Directly assign the local value to inputs
inputs = {
  fin_config = local.common_config
}

The child terragrunt.hcl:

include "root" {
  path = find_in_parent_folders()
}

generate "provider" {
  path      = "providers.tf"
  if_exists = "overwrite"
  contents = <<EOF
provider "azurerm" {
  features {}
  subscription_id = "abc"
  skip_provider_registration = true
  alias = "\${include.root.inputs.fin_config["law"]}"
}

provider "azurerm" {
  features {}
  subscription_id = "\${include.root.inputs.fin_config["sub"]}"
  skip_provider_registration = true
}
EOF
}

So make sure any locals used within the inputs block of the master configuration are statically defined or resolved early enough to be fully available when the child configurations are processed.
When accessing map attributes in the contents of the generate block, use the ["key"] notation to make sure proper interpolation.
Double-check the structure and definitions in your configurations to avoid any timing issues with the evaluation of locals.

---

But, in my master terragrunt.hcl, I don't want to use hardcover vars, I want to read vars dynamically.
That's why I have so many hacks with fin_config read from environment.hcl files.

So you want to dynamically load configurations from environment.hcl files, and use these configurations to generate provider configurations across multiple environments. You need to avoid repetition and keep your Terragrunt configuration DRY (Do not Repeat Yourself).

The challenge becomes to dynamically include environment-specific settings in the Terragrunt configuration for each project without hardcoding variables or repeating code.

A strategy that might work involves leveraging the locals block within each project's terragrunt.hcl to dynamically read environment configurations while still maintaining a level of DRYness by abstracting common logic into a separate file whenever possible.
Since direct inheritance of locals across Terragrunt configurations is not supported, one approach to minimize repetition involves using the include block alongside a helper function or script.

So:

• Keep your environment.hcl files as they are, centralized under each environment. That will maintain clear, environment-specific configurations.

• Consider abstracting the logic for reading and processing environment.hcl files into a separate script or a Terragrunt before_hook. That script can be invoked to set environment variables or generate a .tfvars file that Terragrunt can read from.

  A load-env-config.sh script would be:

  #!/bin/bash
  
  # Load environment configurations based on the given environment and set them as environment variables
  # Alternatively, generate a .tfvars file that Terragrunt can consume
  
  ENV=$1
  CONFIG_PATH="./environments/${ENV}/environment.hcl"
  
  # Pseudocode: Read the CONFIG_PATH file and export configurations as environment variables
  # or generate a "generated-${ENV}.tfvars" file with the needed variables.
  
  echo "Variables set for environment: $ENV"
  

• In each project's terragrunt.hcl, use a combination of include to inherit global settings and locals to dynamically load environment-specific configurations. If necessary, invoke your abstraction to prepare the environment before Terragrunt runs.

  terraform {
  # Your Terraform configuration
  }
  
  before_hook "load_env_config" {
  commands     = ["init", "apply", "plan"]
  execute      = ["bash", "${get_terragrunt_dir()}/load-env-config.sh", "${local.env}"]
  }
  

• Dynamically Load Configurations in Project terragrunt.hcl:

  locals {
  # Assuming the environment name is passed via CLI or determined in some way
  env = "dev"
  }
  
  include "root" {
  path = find_in_parent_folders()
  }
  
  # Use the dynamically set environment variables or read from a generated .tfvars file
  

You will need a mechanism to determine the current environment (dev, test, prod, etc.). That could be done through CLI arguments, environment variables, or naming conventions.