Terragrunt support

[cdenneen]

Any chance the tf-controller can be used with terragrunt? Allows scaling the TF modules a bit more.

[ilithanos]

It's not that i dislike terragrunt, but i've seen this mentioned both in issues and here.

I don't personally see the fit between terragrunt and the tofu-controller (tf-controller). They basically solve the same problem but in different ways.

Putting technical difficulties aside when it comes to implementing it ( dosn't fit into the architecture of the tofu-controller and the way it interfaces with terraform ), i'm curious about what it is that people are missing from terragrunt when using the terraform controller.

So please, explain to me what terragrunt provides that would make it need to be supported by the controller. And saying it's needed to scale is to vague, i would like some more details than that.

[chanwit]

For the new tofu-controller, I would say that we shouldn't support this huge feature in it. However, considering the long-term goals of the Flux IaC organization, we aim to be a place that breeds a new kind of IaC controllers around Flux, using the same technology we've learned from TF-Controller.

Would you like to give a new controller a spin?

[ilithanos]

I don't have the time to do it sadly, and we've had this issue running for some time now.

#1184 that i just closed as a not planned since the discussion in there was just going around in circles. I've closed it with a note that terragrunt should be done a separate controller and not as part of the tofu-controller. And that any further discusion about how to make that happen should be moved here. Now theres a link to that issue from here so that we atleast have the information linked together.

[thejosephstevens]

If it's helpful, I have a specific use-case:
We keep our TF modules DRY by sym-linking common files (e.g. we store all identity provider resources in the same AWS account, so we re-use the same AWS provider declaration with account, region, role_arn, etc to ease maintenance). This works fine locally, but once I deploy to flux it looks like those symlinks aren't being handled as I expect (not exactly sure on the details, but when I replace the symlinks with real files the issue goes away). I think we would likely get 90+% of the functionality I would look for from Terragrunt if symlinks within the same source artifact are supported.

[ilithanos]

Hmm... Interesting way of doing it. I can see multiple ways around this though, but yeah terragrunt is an option for that. But never the less it's helpful to get real world use cases that could potentially help solve issues people have.

How we solve that use case at my day job using tofu-controller:

Our specific provider configuration blocks are completely omitted from the terraform code. Instead it's handled using namespaces runners where the service accounts are linked up with the Arn it needs.

Then we use a kustomize shared patch file to patch in the region etc to all the Terraform resources. That way we still only have to change that configuration in a single place.

That doesn't solve running the Terraform code locally without the controller though, but that is by choice, since we are only running Terraform using the controller when deploying Terraform. For testing of modules during module development we use terratest to automate module testing.

I'm not against making a seperate terragrunt controller based on the tofu-controller project, if there are enough people willing to put in the work, but I can't personally dedicate the time for it right now, due to the amount of work we already have on this project.

The thing is, terragrunt support would have to be a seperate controller due to the nature of the such a project, do we would need a team of community members willing to put in the work of maintaining such a project.