Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

abstract CWL test: recusively validate #93

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mr-c
Copy link
Contributor

@mr-c mr-c commented Nov 4, 2023

Copy link

codecov bot commented Nov 4, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 40.41%. Comparing base (785cd40) to head (b355f82).

❗ Current head b355f82 differs from pull request most recent head 33c7572. Consider uploading reports for the commit 33c7572 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##           master      #93   +/-   ##
=======================================
  Coverage   40.41%   40.41%           
=======================================
  Files          48       48           
  Lines        1915     1915           
  Branches      433      433           
=======================================
  Hits          774      774           
  Misses       1025     1025           
  Partials      116      116           
Flag Coverage Δ
unittests 40.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mr-c mr-c marked this pull request as draft November 4, 2023 11:17
@mvdbeek
Copy link
Member

mvdbeek commented Nov 6, 2023

I guess it's "working":

https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:123:

=================================== FAILURES ===================================
  _____________________________ test_abstract_export _____________________________
  
      def test_abstract_export():
          for name, example in EXAMPLES.items():
              format2, native = _both_formats(example)
  >           _run_example(format2, _examples_path_for(f"{name}_from_format2.cwl"))
  
  tests/test_export_abstract.py:59: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  tests/test_export_abstract.py:143: in _run_example
      loadingContext, uri = recursive_resolve_and_validate_document(
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/load_tool.py:6[55](https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:57): in recursive_resolve_and_validate_document
      tool = make_tool(uri, loadingContext)
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/load_tool.py:[59](https://github.com/galaxyproject/gxformat2/actions/runs/6754508062/job/18394521652?pr=93#step:5:61)7: in make_tool
      tool = loadingContext.construct_tool_object(processobj, loadingContext)
  .tox/py310-unit/lib/python3.10/site-packages/cwltool/workflow.py:55: in default_make_tool
      return Workflow(toolpath_object, loadingContext)
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  self = <cwltool.workflow.Workflow object at 0x7fcd5223d9f0>
  toolpath_object = {'class': 'Workflow', 'cwlVersion': 'v1.2', 'doc': 'Simple workflow that no-op cats a file and then selects 10 random ...tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl', 'http://commonwl.org/cwltool#original_cwlVersion': 'v1.2'}
  loadingContext = <cwltool.context.LoadingContext object at 0x7fcd51dae2f0>
  
      def __init__(
          self,
          toolpath_object: CommentedMap,
          loadingContext: LoadingContext,
      ) -> None:
          """Initialize this Workflow."""
          super().__init__(toolpath_object, loadingContext)
          self.provenance_object: Optional[ProvenanceProfile] = None
          if loadingContext.research_obj is not None:
              run_uuid: Optional[UUID] = None
              is_main = not loadingContext.prov_obj  # Not yet set
              if is_main:
                  run_uuid = loadingContext.research_obj.ro_uuid
      
              self.provenance_object = ProvenanceProfile(
                  loadingContext.research_obj,
                  full_name=loadingContext.cwl_full_name,
                  host_provenance=loadingContext.host_provenance,
                  user_provenance=loadingContext.user_provenance,
                  orcid=loadingContext.orcid,
                  run_uuid=run_uuid,
                  fsaccess=loadingContext.research_obj.fsaccess,
              )  # inherit RO UUID for main wf run
              # TODO: Is Workflow(..) only called when we are the main workflow?
              self.parent_wf = self.provenance_object
      
          # FIXME: Won't this overwrite prov_obj for nested workflows?
          loadingContext.prov_obj = self.provenance_object
          loadingContext = loadingContext.copy()
          loadingContext.requirements = self.requirements
          loadingContext.hints = self.hints
      
          self.steps: List[WorkflowStep] = []
          validation_errors = []
          for index, step in enumerate(self.tool.get("steps", [])):
              try:
                  self.steps.append(
                      self.make_workflow_step(step, index, loadingContext, loadingContext.prov_obj)
                  )
              except ValidationException as vexc:
                  if _logger.isEnabledFor(logging.DEBUG):
                      _logger.exception("Validation failed at")
                  validation_errors.append(vexc)
      
          if validation_errors:
  >           raise ValidationException("\n".join(str(v) for v in validation_errors))
  E           schema_salad.exceptions.ValidationException: tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl:22:7: Workflow step output 'out_file1'
  E                                                                            does not correspond to
  E           tests/examples/abstractcwl/BASIC_WORKFLOW_from_format2.cwl:27:7:   tool output (expected '')

@mr-c
Copy link
Contributor Author

mr-c commented Nov 6, 2023

I guess it's "working":

Yeah 😄 but probably the abstract export should be fixed :-P

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CWL export validation not as strict as cwltool's
2 participants