Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix: Fixing excessive warning for dependency input usage #3879

Merged
merged 16 commits into from
Feb 13, 2025

Conversation

yhakbar
Copy link
Collaborator

@yhakbar yhakbar commented Feb 12, 2025

Description

Fixes #3873

TODOs

Read the Gruntwork contribution guidelines.

  • Update the docs.
  • Run the relevant tests successfully, including pre-commit checks.
  • Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
  • Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Updated strict mode to allow for error suppression.

Summary by CodeRabbit

  • New Features

    • Updated environment variable usage now adopts the new names (TG_TF_PATH with a default value change and TG_PROVIDER_CACHE added) to streamline integration workflows.
    • Enhanced validation and clearer error/warning messages when using deprecated dependency inputs and legacy command flags.
  • Documentation

    • Revised dependency block docs with a new attribute for defining mock outputs merge strategy and deprecation notice for the previous attribute.
  • Tests

    • Updated integration tests to reflect simplified command-line flags and improved error messages for deprecated functionalities.
    • Added assertions to check for deprecation warnings related to reading inputs from dependencies.

Copy link

vercel bot commented Feb 12, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
terragrunt-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Feb 13, 2025 6:21pm

Copy link
Contributor

coderabbitai bot commented Feb 12, 2025

Walkthrough

This pull request introduces several updates across the project. The CircleCI configuration and various test files update deprecated environment variable names (e.g., replacing TERRAGRUNT_TFPATH with TG_TF_PATH and adjusting default values). In the core configuration packages, new functions and stricter error handling have been added to validate dependency input usage. Documentation for configuration blocks is updated with new attributes and deprecation notices. Additionally, the strict control mechanisms are enhanced by introducing a new warning suppression method and related adjustments in control evaluation logic.

Changes

File(s) Change Summary
.circleci/config.yml, test/fixtures/hooks/error-hooks/tf.sh, test/fixtures/render-json-metadata/attributes/terragrunt.hcl, test/helpers/package.go, test/integration_common_test.go Updated environment variable names from TERRAGRUNT_TFPATH to TG_TF_PATH and adjusted default values (e.g., from terraform to tofu).
config/config.go, config/config_partial.go Enhanced configuration parsing with a new function detectInputsCtyUsage to check for deprecated dependency inputs and improved error handling via stricter control evaluation.
docs/_docs/04_reference/04-config-blocks-and-attributes.md Updated documentation: added mock_outputs_merge_strategy_with_state attribute and deprecated mock_outputs_merge_with_state; clarified usage for dependency blocks.
internal/strict/control.go, internal/strict/controls/control.go, internal/strict/controls/controls.go, internal/strict/controls/deprecated_env_var.go, internal/strict/controls/deprecated_flag_name.go Introduced a new SuppressWarning method in the Control interface and struct; added a Suppress field; updated error and warning messages and logging conditions to consider warning suppression.
test/integration_json_test.go, test/integration_serial_test.go, test/integration_strict_test.go, test/integration_test.go Simplified command-line flag formats (removing --terragrunt- prefixes), updated error messages for deprecated commands and dependency inputs, and added new assertions for deprecation warnings.

Sequence Diagram(s)

sequenceDiagram
    participant U as User/CLI
    participant PC as ParseConfig()
    participant D as detectInputsCtyUsage()
    participant SC as StrictControl

    U->>PC: Start configuration parsing
    PC->>D: Check for dependency input usage
    D-->>PC: Return usage flag (found/not found)
    PC->>SC: Evaluate strict control settings
    SC-->>PC: Return error/warning if necessary
    PC-->>U: Return parsed config or error status
Loading

Possibly related PRs

Suggested labels

docs

Suggested reviewers

  • levkohimins
  • denis256

Poem

In the code where changes bloom,
Variables shift, dispelling gloom,
Tests now sing a simpler tune,
Strict controls dance in the room, 🚀
A tweak, a fix—our code's in full bloom!
Cheers to progress and cleaner zoom!

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@levkohimins
Copy link
Contributor

There is no need to change the interface. There are two options:

  1. Easy, just remove warn message:
	skipDependenciesInputsControl := &Control{
		// TODO: `ErrorFmt` and `WarnFmt` of this control are not displayed anywhere and needs to be reworked.
		Name:        SkipDependenciesInputs,
		Description: "Disable reading of dependency inputs to enhance dependency resolution performance by preventing recursively parsing Terragrunt inputs from dependencies.",
		Error:       errors.Errorf("Reading inputs from dependencies is no longer supported. To acquire values from dependencies, use outputs."),
		Category:    stageCategory,
	}

If ctrl.Warning is empty it does nothing:

if logger := log.LoggerFromContext(ctx); logger != nil && ctrl.Warning != "" {

  1. Flexible, creating a new struct, like DeprecatedEnvVar, with implementing your own Evaluate method
type ControlWithoutWarning struct {
	*Control
}

func (ctrl *ControlWithoutWarning) Evaluate(ctx context.Context) error {

	// Do what you want.

	return nil
}

Copy link
Contributor

@levkohimins levkohimins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, we can go with this, but I would recommend simplifying the code (I explaned in the comments).

Comment on lines 466 to 467
suppressable, err := skipDependenciesInputs.(*controls.SuppressableControl)
if err {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
suppressable, err := skipDependenciesInputs.(*controls.SuppressableControl)
if err {
suppressable, ok := skipDependenciesInputs.(*controls.SuppressableControl)
if ok {

Comment on lines 34 to 39
// Suppress suppresses the warning message of the control.
func (ctrl *SuppressableControl) Suppress() *SuppressableControl {
ctrl.suppress = true

return ctrl
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Suppress suppresses the warning message of the control.
func (ctrl *SuppressableControl) Suppress() *SuppressableControl {
ctrl.suppress = true
return ctrl
}
// Suppress suppresses the warning message of the control.
func (ctrl *Control) Suppress() *Control {
ctrl.suppress = true
return ctrl
}

I think we can add a suppress field to the already existing Control structure and use it where we need. The default value suppress: false will not affect the behavior of other controls in any way. And in this way you can greatly simplify the code, and if necessary use it in other controls, if such a need arises in the future.

@levkohimins
Copy link
Contributor

levkohimins commented Feb 13, 2025

The new structure approach is justified when you need to write different logic for Evaluate(ctx context.Context), but in a situation where logic is the same, it only complicates the future maintenance of the same logic in two places.

@levkohimins
Copy link
Contributor

Sorry if I confused you with the options initially.

Copy link
Contributor

@levkohimins levkohimins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job👍 the latest comments.

@levkohimins levkohimins self-requested a review February 13, 2025 16:51
levkohimins
levkohimins previously approved these changes Feb 13, 2025
Copy link
Contributor

@levkohimins levkohimins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already good. It's up to you, apply the latest suggestion or not.

@yhakbar yhakbar requested a review from levkohimins February 13, 2025 17:29
@yhakbar yhakbar marked this pull request as ready for review February 13, 2025 17:29
@yhakbar yhakbar requested a review from denis256 as a code owner February 13, 2025 17:29
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
config/config.go (1)

960-991: Looks logically consistent with a straightforward approach.

This helper method diligently hunts for dependency.*.inputs references in HCL attributes. In case you need deeper or more dynamic referencing detection beyond three segments, consider refining or generalizing further. Otherwise, this does the trick without overhead.

test/integration_test.go (1)

3030-3038: LGTM! Consider adding a comment explaining the warning removal.

The removal of the deprecation warning check makes sense if the functionality is no longer deprecated. However, it would be helpful to add a comment explaining why this check was removed to provide context for future maintainers.

 _, stderr, err := helpers.RunTerragruntCommandWithOutput(t, "terragrunt run-all apply -auto-approve --non-interactive --log-level trace --working-dir "+rootPath)
 require.NoError(t, err)

+// Note: Deprecation warning check removed because <reason>
 assert.NotContains( // Check that we're getting a warning for usage of deprecated functionality.
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a02986e and 2d5e986.

📒 Files selected for processing (17)
  • .circleci/config.yml (11 hunks)
  • config/config.go (3 hunks)
  • config/config_partial.go (1 hunks)
  • docs/_docs/04_reference/04-config-blocks-and-attributes.md (1 hunks)
  • internal/strict/control.go (1 hunks)
  • internal/strict/controls/control.go (3 hunks)
  • internal/strict/controls/controls.go (1 hunks)
  • internal/strict/controls/deprecated_env_var.go (1 hunks)
  • internal/strict/controls/deprecated_flag_name.go (1 hunks)
  • test/fixtures/hooks/error-hooks/tf.sh (1 hunks)
  • test/fixtures/render-json-metadata/attributes/terragrunt.hcl (1 hunks)
  • test/helpers/package.go (1 hunks)
  • test/integration_common_test.go (1 hunks)
  • test/integration_json_test.go (2 hunks)
  • test/integration_serial_test.go (3 hunks)
  • test/integration_strict_test.go (2 hunks)
  • test/integration_test.go (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: Pull Request has non-contributor approval
🔇 Additional comments (27)
config/config.go (3)

21-21: Nice addition of the controls import.

No issues spotted with bringing in internal/strict/controls. It's straightforward and aligns with the new usage below.


28-28: Importing hclsyntax looks good.

Now you can parse HCL files more precisely and detect references with better granularity. This is a solid addition.


873-886: Clarify default behavior if the control is not set.

This new block cleanly checks for the SkipDependenciesInputs control. However, an error is thrown if the control is absent. If that’s the intended behavior (strict enforcement), all good. Otherwise, consider a fallback or an optional approach if you’d like looser handling.

internal/strict/controls/deprecated_env_var.go (1)

88-88: Wise safeguard for suppressed warnings.

Including !ctrl.Suppress ensures logs remain quiet when the control is intentionally muted. This aligns neatly with your new suppression logic.

internal/strict/controls/deprecated_flag_name.go (1)

108-108: Good addition of the !ctrl.Suppress check.

Your warning flow now accurately respects the suppression toggle, preventing unnecessary messages when warnings should be silenced.

internal/strict/controls/control.go (3)

45-46: LGTM! Clear and descriptive field comment.

The new Suppress field is well-documented and its purpose is clear.


103-106: LGTM! Simple and effective implementation.

The SuppressWarning method provides a clean way to control warning visibility.


126-126: LGTM! Thread-safe warning suppression.

The warning suppression check is correctly placed before the OnceWarn.Do call, maintaining thread safety.

test/integration_strict_test.go (2)

39-39: LGTM! Consistent error messages.

The error messages are clear, consistent, and provide actionable guidance to users.

Also applies to: 46-46


55-55: LGTM! Simplified command-line arguments.

Removed the --terragrunt- prefix from flags, making the commands more concise.

internal/strict/controls/controls.go (1)

58-59: LGTM! Clear and actionable messages.

The error and warning messages are:

  • Direct about the deprecation status
  • Clear about what users should do instead
  • Consistent with the PR's goal of improving warning messages
internal/strict/control.go (1)

50-52: LGTM! Clean interface extension.

The new SuppressWarning method is:

  • Well-documented
  • Focused on a single responsibility
  • Consistent with the interface's style
test/integration_common_test.go (1)

363-363: LGTM! Environment variable name updated for consistency.

The change from TERRAGRUNT_TFPATH to TG_TF_PATH aligns with the standardization of environment variable naming across the codebase.

test/integration_json_test.go (3)

32-32: LGTM! Variable name follows Go naming convention.

The change from terragruntHcl to terragruntHCL follows Go's standard naming convention for acronyms.


40-40: LGTM! Command flags simplified.

Removing the redundant --terragrunt- prefix from command flags makes the code cleaner while maintaining the same functionality.


117-117: LGTM! Improved error message clarity.

Adding expected and actual values in the assertion message will make debugging test failures much easier.

config/config_partial.go (2)

461-464: LGTM! Improved error handling for missing control.

The code now properly checks for missing control and returns a clear error message.


466-471: LGTM! Enhanced warning suppression and logging.

The changes improve warning control by:

  1. Suppressing warnings when needed
  2. Using clearer log message wording
test/helpers/package.go (1)

638-638: LGTM! Environment variable name updated for consistency.

The change from TERRAGRUNT_TFPATH to TG_TF_PATH aligns with the standardization of environment variable naming across the codebase.

test/integration_serial_test.go (3)

235-235: LGTM! Command flag update looks good.

The change from --terragrunt-non-interactive to --non-interactive and --terragrunt-working-dir to --working-dir aligns with the shorter flag names.


245-245: LGTM! Command flag update is consistent.

The flag updates here match the changes in the previous command, maintaining consistency across the test.


258-263: LGTM! Good addition of deprecation warning check.

The new assertion verifies that users are properly warned about the deprecated functionality of reading inputs from dependencies. This helps ensure a smooth transition as the feature will be removed in a future version.

test/integration_test.go (1)

3043-3043: LGTM! Command flag updates are consistent.

The command flag updates match the changes made above, maintaining consistency throughout the test.

test/fixtures/hooks/error-hooks/tf.sh (1)

3-3:

❓ Verification inconclusive

Environment variable and default value updates look good!

The changes standardize the environment variable name to TG_TF_PATH and update the default value to tofu, aligning with the OpenTofu migration.

Let's verify if this environment variable is consistently used across the codebase:


🏁 Script executed:

#!/bin/bash
# Search for any remaining instances of the old environment variable
rg "TERRAGRUNT_TFPATH"

Length of output: 355


Environment variable and default value updates confirmed!

The changes in test/fixtures/hooks/error-hooks/tf.sh correctly update the variable from TERRAGRUNT_TFPATH to TG_TF_PATH and switch the default from terraform to tofu, aligning with the OpenTofu migration.

However, I did notice that our docs still refer to TERRAGRUNT_TFPATH (e.g., in docs-starlight/src/content/docs/04-reference/01-hcl/03-attributes.mdx and docs/_docs/04_reference/02-cli-options.md) with a deprecation note. If this is intentional for backward compatibility, that’s fine—but if not, please update the documentation accordingly for consistency.

Great job on the update!

test/fixtures/render-json-metadata/attributes/terragrunt.hcl (1)

18-18: Environment variable update is consistent!

The change to use TG_TF_PATH with default value tofu matches the pattern established in other files.

.circleci/config.yml (1)

33-33:

❓ Verification inconclusive

CI configuration updates are thorough and consistent!

All CI jobs have been updated to use the new TG_TF_PATH environment variable. The changes are consistent across different test configurations and environments.

Let's verify that all CI jobs that need the environment variable have been updated:

Also applies to: 47-47, 99-99, 184-184, 208-208, 269-269, 290-291, 313-314, 336-337, 380-382, 535-536


🏁 Script executed:

#!/bin/bash
# Search for any jobs that might still be using the old environment variable
rg "TERRAGRUNT_TFPATH" .circleci/

Length of output: 33


Everything Looks Updated, But Please Give It a Final Manual Look!

The switch from TERRAGRUNT_TFPATH to TG_TF_PATH appears spot on. Our automated search in the .circleci/ directory didn’t find any leftover instances of the old variable, which suggests that all relevant CI jobs have been updated. Key points:

  • Update at .circleci/config.yml (e.g., line 33) and across the other noted locations (47, 99, 184, 208, 269, 290-291, 313-314, 336-337, 380-382, 535-536) look consistent.
  • The rg "TERRAGRUNT_TFPATH" .circleci/ command returned no output, implying no remnants of the old variable exist.

That said, since automated checks can occasionally miss context, please manually verify the CI configurations to ensure nothing’s lurking under the hood.

docs/_docs/04_reference/04-config-blocks-and-attributes.md (1)

1177-1187: Documentation updates for mock outputs configuration are clear and comprehensive!

The new mock_outputs_merge_strategy_with_state attribute is well documented with clear explanations of the available merge strategies (no_merge, shallow, deep_map_only). The deprecation notice for mock_outputs_merge_with_state is properly included.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
config/config.go (2)

873-885: Consider improving error handling and context consistency.

The validation logic for deprecated dependency inputs is solid, but there are two areas for improvement:

  1. The error message when the control is not found could be more descriptive
  2. The context variable naming could be more consistent (using evalCtx in one place and ctx in another)

Consider applying these changes:

-			return nil, errors.New("failed to find control " + controls.SkipDependenciesInputs)
+			return nil, errors.New(fmt.Sprintf("strict control '%s' not found - this is required for validating dependency inputs", controls.SkipDependenciesInputs))

-		evalCtx := log.ContextWithLogger(ctx, ctx.TerragruntOptions.Logger)
-		if err := skipDependenciesInputs.Evaluate(evalCtx); err != nil {
+		ctx = log.ContextWithLogger(ctx, ctx.TerragruntOptions.Logger)
+		if err := skipDependenciesInputs.Evaluate(ctx); err != nil {

959-991: Enhance documentation and constant naming.

The implementation is solid, but could benefit from more detailed documentation and clearer constant naming.

Consider these improvements:

-// detectInputsCtyUsage detects if an identifier matching dependency.foo.inputs.bar is used in the given HCL file.
-//
-// This is deprecated functionality, so we look for this to determine if we should throw an error or warning.
+// detectInputsCtyUsage scans the HCL file for deprecated dependency input references.
+//
+// This function looks for identifiers matching the pattern dependency.foo.inputs.bar, which represents the deprecated
+// way of accessing dependency outputs. When found, this triggers strict mode validation to ensure proper migration
+// to the new dependency output access pattern.
+//
+// Example of deprecated usage:
+//   - dependency.backend.inputs.bucket  // deprecated
+//   - dependency.backend.outputs.bucket // new way
 func detectInputsCtyUsage(file *hclparse.File) bool {

-			const dependencyInputsIdentifierMinParts = 3
+			const minPartsForDependencyInputs = 3  // minimum parts needed for "dependency.name.inputs"
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2d5e986 and ae3bc0b.

📒 Files selected for processing (1)
  • config/config.go (3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (5)
  • GitHub Check: Pull Request has non-contributor approval
  • GitHub Check: Pull Request has non-contributor approval
  • GitHub Check: Pull Request has non-contributor approval
  • GitHub Check: build-and-test
  • GitHub Check: Pull Request has non-contributor approval
🔇 Additional comments (1)
config/config.go (1)

21-22: LGTM! Clean import additions.

The new imports for controls and hclsyntax packages are well-organized and necessary for the new functionality.

@yhakbar yhakbar merged commit 6bf5e1c into main Feb 13, 2025
7 of 9 checks passed
@yhakbar yhakbar deleted the fix/fixing-excessive-warning-for-dependency-input branch February 13, 2025 18:43
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TG warns about skip-dependencies-inputs option even if not used
2 participants