Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix internal formula json v3 frozen hash parsing bug #17201

Merged
merged 1 commit into from
May 2, 2024

Conversation

apainintheneck
Copy link
Contributor

@apainintheneck apainintheneck commented May 2, 2024

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew typecheck with your changes locally?
  • Have you successfully run brew tests with your changes locally?

This caused formulae with uses from macos bounds to not load correctly because they tried to modify a frozen hash. It wasn't obvious from the tests because I didn't replicate the real world JSON parsing conditions closely enough. I also had to modify Cachable#clear_cache so that it can clear frozen hashes.

Follow-up to #17153.

Note: This doesn't affect users at all since nobody uses this other than me right now.

Error:

Error: can't modify frozen Hash: {"since"=>"catalina"}
Warning: Removed Sorbet lines from backtrace!
Rerun with `--verbose` to see the original backtrace
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:123:in `delete'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:123:in `block in _deep_transform_keys_in_object!'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:122:in `each'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:122:in `_deep_transform_keys_in_object!'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:48:in `deep_transform_keys!'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:230:in `block (2 levels) in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:218:in `each'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:218:in `block in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:304:in `instance_exec'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:304:in `block (2 levels) in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formula.rb:3664:in `instance_eval'
/usr/local/Homebrew/Library/Homebrew/formula.rb:3664:in `stable'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:293:in `block in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `initialize'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `new'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:962:in `load_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:955:in `klass'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:569:in `get_formula'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:1009:in `factory'
/usr/local/Homebrew/Library/Homebrew/dependency.rb:41:in `to_formula'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:46:in `block (2 levels) in formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:45:in `each'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:45:in `block in formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:39:in `each'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:39:in `formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:59:in `unused_formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:16:in `removable_formulae'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:693:in `autoremove'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:291:in `clean!'
/usr/local/Homebrew/Library/Homebrew/cmd/cleanup.rb:52:in `run'
/usr/local/Homebrew/Library/Homebrew/brew.rb:92:in `<main>'

This caused formulae with uses from macos bounds to not load correctly
because they tried to modify a frozen hash. It wasn't obvious from
the tests because I didn't replicate the real world JSON parsing
conditions closely enough. I also had to modify `Cachable#clear_cache`
so that it can clear frozen hashes.

Error:
```
Error: can't modify frozen Hash: {"since"=>"catalina"}
Warning: Removed Sorbet lines from backtrace!
Rerun with `--verbose` to see the original backtrace
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:123:in `delete'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:123:in `block in _deep_transform_keys_in_object!'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:122:in `each'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:122:in `_deep_transform_keys_in_object!'
/usr/local/Homebrew/Library/Homebrew/extend/hash/keys.rb:48:in `deep_transform_keys!'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:230:in `block (2 levels) in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:218:in `each'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:218:in `block in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:304:in `instance_exec'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:304:in `block (2 levels) in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formula.rb:3664:in `instance_eval'
/usr/local/Homebrew/Library/Homebrew/formula.rb:3664:in `stable'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:293:in `block in load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `initialize'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `new'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:283:in `load_formula_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:962:in `load_from_api'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:955:in `klass'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:569:in `get_formula'
/usr/local/Homebrew/Library/Homebrew/formulary.rb:1009:in `factory'
/usr/local/Homebrew/Library/Homebrew/dependency.rb:41:in `to_formula'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:46:in `block (2 levels) in formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:45:in `each'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:45:in `block in formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:39:in `each'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:39:in `formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:59:in `unused_formulae_with_no_formula_dependents'
/usr/local/Homebrew/Library/Homebrew/utils/autoremove.rb:16:in `removable_formulae'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:693:in `autoremove'
/usr/local/Homebrew/Library/Homebrew/cleanup.rb:291:in `clean!'
/usr/local/Homebrew/Library/Homebrew/cmd/cleanup.rb:52:in `run'
/usr/local/Homebrew/Library/Homebrew/brew.rb:92:in `<main>'
```
@apainintheneck apainintheneck force-pushed the fix-internal-formula-json-v3-frozen-hash-bug branch from 215fceb to 37cbfc4 Compare May 2, 2024 04:36
@apainintheneck apainintheneck marked this pull request as ready for review May 2, 2024 04:37
Comment on lines +10 to +13
# NOTE: We overwrite here instead of using `Hash#clear` to handle frozen hashes.
sig { void }
def clear_cache
cache.clear
overwrite_cache!({})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The internal JSON v3 format gets parsed into a frozen hash with JSON.parse(string, frozen: true) and then gets moved into the cache with Cachable#overwrite_cache! so the cache is frozen in that case.

Copy link
Contributor Author

@apainintheneck apainintheneck May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there are any real downsides to creating a new hash here instead of using Hash#clear.

Edit: I guess we could also do this if we don't want to create new hashes here.

def clear_cache
  cache.frozen? ? overwrite_cache!({}) : cache.clear
end

@MikeMcQuaid MikeMcQuaid merged commit bd72ec8 into master May 2, 2024
25 checks passed
@MikeMcQuaid MikeMcQuaid deleted the fix-internal-formula-json-v3-frozen-hash-bug branch May 2, 2024 07:39
@MikeMcQuaid
Copy link
Member

Thanks @apainintheneck!

This caused formulae with uses from macos bounds to not load correctly because they tried to modify a frozen hash. It wasn't obvious from the tests because I didn't replicate the real world JSON parsing conditions closely enough.

I think it's worth modifying tests where you don't need to do this replication but it's loading formulae as close to the usual API path as possible.

@apainintheneck
Copy link
Contributor Author

I agree. I think that #16895 will get us most of the way there if/when someone has the time to work on it. Last time I looked into it I realized that it wasn't going to be a quick migration so I gave up on it.

@github-actions github-actions bot added the outdated PR was locked due to age label Jun 3, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 3, 2024
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants