Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

FI-3063: Add MustSupport assertion #622

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

dehall
Copy link
Contributor

@dehall dehall commented Feb 14, 2025

Summary

Adds a new assertion to check whether all Must Support elements on a given profile are present in given resources, and a method that just returns the missing element names

assert_must_support_elements_present(resources, profile_url, 
  validator_name: :default, metadata: nil, 
  requirement_extension: nil, &)

missing_must_support_elements(resources, profile_url, 
  validator_name: :default, metadata: nil,
  requirement_extension: nil, &)

This assertion calls into the new MustSupport logic that was migrated into core recently (see #584 for description although it was actually merged by #601).

To support this logic, we need to load the IG that contains the relevant profile and parse its StructureDefinition. The IG is referenced in the validator so we pick it from there. All the logic to load IGs has been migrated from the CLI evaluate task to the IGs repository, so now the primary way to load an IG from anywhere in the system is with Inferno::Repositories::IGs.new.find_or_load(ig_path). As part of this migration the "download IG" logic had to be updated to no longer use a method from Thor.
The IG entity now stores its original file path as well, to make it easy to find an IG by either identifier or by file path.

The most significant change to the IG logic is that now downloaded IGs will be cached in the user's package cache (~/.fhir/packages) so that they don't have to be downloaded every time. The approach is modeled after how the HL7 FHIR validator works - it downloads the TGZ, extracts it to a temporary folder within the package cache, then renames that temporary folder to the appropriate name, all using a lockfile. This only applies to downloaded IGs, not ones referenced by filename.

I also moved some logic from AllMustSupportsPresent rule to a new module Inferno::DSL::MustSupportTest to match how it was originally structured in the US Core test kit. I would have liked to refactor this to only expose a few methods but I'm concerned about breaking things. This module should not be included by default. (And I'm certainly open to renaming or moving this to a different folder)

Testing Guidance

Some important things this touches:

  • IG loading affects the inferno new and inferno evaluate CLI
  • FHIR package cache

Possibly the quickest way to test the IG loading into the package cache is with the evaluator CLI:

# list what's already in your package cache
ls ~/.fhir/packages
# delete or move something
rm -rf ~/.fhir/packages/hl7.fhir.us.core\#3.1.1
# run the evaluator on that same IG to load it again
bundle exec inferno evaluate "hl7.fhir.us.core#3.1.1"
# check the package was re-loaded
ls ~/.fhir/packages
ls ~/.fhir/packages/hl7.fhir.us.core\#3.1.1

To test the actual must_support_elements_missing? , I put up a demonstration branch on the US Core Test kit: https://github.com/inferno-framework/us-core-test-kit/compare/fi-3063-ms-assertion-demo
To try this out locally:

  1. Add the following to your local core gemfile:
gem 'us_core_test_kit',
    git: 'https://github.com/inferno-framework/us-core-test-kit.git',
    branch: 'fi-3063-ms-assertion-demo'
  1. Run bundle install
  2. Add a require at the top of dev_suites/dev_demo_ig_stu1/demo_suite.rb : require 'us_core_test_kit'

Then start up inferno and select one of the US Core test suites. An easy way to get a passing or failing result is to run US Core 3 suite with the Inferno Reference Server preset -- with Patient IDs "85,355" then all MS tests should pass, but remove 355 and set Patient IDs to only "85" and you will get a skip. See prod for comparison: https://inferno.healthit.gov/suites/us_core_v311/kZdWLoDZphb#2.2

For a similar example with a test kit that uses file-based IGs, try the demo branch on Carin4BB:

https://github.com/inferno-framework/carin-for-blue-button-test-kit/compare/fi-3063-ms-assertion-demo

  1. Add the following to your local core gemfile:
gem 'carin_for_blue_button_test_kit',
    git: 'https://github.com/inferno-framework/carin-for-blue-button-test-kit.git',
    branch: 'fi-3063-ms-assertion-demo'
  1. Run bundle install
  2. Add a require at the top of dev_suites/dev_demo_ig_stu1/demo_suite.rb : require 'carin_for_blue_button_test_kit'
  3. Copy the carin IG tgz files from https://github.com/inferno-framework/carin-for-blue-button-test-kit/tree/main/lib/carin_for_blue_button_test_kit/igs to the core repo ./igs/ folder (you may need to create it; this is where the docker volume for the validator maps to -- see )
  4. Restart the services

Copy link

codecov bot commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 89.68610% with 23 lines in your changes missing coverage. Please review.

Project coverage is 84.73%. Comparing base (95c7a1d) to head (c0795c1).

Files with missing lines Patch % Lines
lib/inferno/dsl/must_support_test.rb 88.14% 16 Missing ⚠️
lib/inferno/repositories/igs.rb 88.00% 6 Missing ⚠️
lib/inferno/dsl/fhir_resource_validation.rb 90.90% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #622      +/-   ##
==========================================
+ Coverage   84.66%   84.73%   +0.06%     
==========================================
  Files         283      284       +1     
  Lines       12439    12526      +87     
  Branches     1508     1528      +20     
==========================================
+ Hits        10532    10614      +82     
- Misses       1899     1904       +5     
  Partials        8        8              
Flag Coverage Δ
backend 92.38% <89.68%> (+0.03%) ⬆️
frontend 79.12% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dehall dehall force-pushed the fi-3063-mustsupport-assertions branch from b18da09 to f6d27a8 Compare February 18, 2025 19:22
# @param requirement_extension [String] Extension URL that implies "required" as an alternative to the MS flag
# @yield [Metadata] Customize the metadata before running the test
# @return [Array<String>] List of missing elements
def missing_must_support_elements(resources, profile_url, validator_name: :default, metadata: nil,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize this method still doesn't really fit here in assertions, but I'm not sure where else to put it. I don't want to include the MustSupportTest automatically in every test, and I don't feel like it's super worth creating a whole new module just for one method. I'm open to suggestions

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant