Feat/advanced templates

[anngvu]

• Refactor templates to use class inheritance #338. Why:
  • This is an under-the-hood change that allows cleaner and more consistent reusable building blocks. I think I tried to do this within what schematic provided, but it didn't work as well as I'd like and felt like a mistake. This is conceptually nicer.
  • Easier to compare templates and see how they are different/specialized. See more meaningful relationship between templates through the inheritance hiearchy.
  • One possible cons is that because it is easier to make templates now, we'll have over-proliferation of templates (some more specific than we need, so need to keep that in mind.)
• data type required when not applicable #251. Why:
  • Yes, just don't have these in non-applicable templates (like for protocols). Looks more obvious what's happening now.
• Add constraints to match template to assay/datatype #339. Why:
  • While technically not able to be used by schematic yet in the future it might actually be useful for improved UI and validation.
  • Outside of schematic, we can possibly use these constraints to make recommendations on Synapse/in custom validation workflow (nfportalutils). If we only know about assay, the default is to suggest the raw data template for that assay. If we know both assay and datatype (ultimately, datatype is more important), we can do a better job of directing contributors to the most appropriate template 🥺
  • Additional clarity, helps to realize whether or not all expected assays/datatypes are actually accounted for in the available template selection...
• Align/map templates to HTAN #114 / Break genomics assay templates into levels like HTAN #168. Why:
  • Technically, we already started doing this for NF-processed data. Having these mappings/alignments is helpful and can improve quality.
  • Perhaps provide starting example for cross-team data model mapping.
• Re-evaluate readPairOrientation, readStrandOrigin #138 -> "libraryStrand". Why:
  • This was just overdue, but this change will "improve quality".
• Update schematic version for manifest-generation testing #343. Why:
  • Assurance that everything still works with latest stable version of schematic!

Update:

• Added tests, not sure if Christina has gotten a pre-official-review time as a heads-up for how proposed changes might fit into this, but I'll schedule some time for interactive working session as well where we can experiment with a fork of this.

[github-actions]

Test Suite Report

Template Generation

template	result	link
ClinicalAssayTemplate	😄	template link
EpigeneticsAssayTemplate	😄	template link
FlowCytometryTemplate	😄	template link
GenomicsAssayTemplate	😄	template link
GenomicsAssayTemplateExtended	😄	template link
ImagingAssayTemplate	😄	template link
LightScatteringAssayTemplate	😄	template link
MRIAssayTemplate	😄	template link
PharmacokineticsAssayTemplate	😄	template link
PlateBasedReporterAssayTemplate	😄	template link
ProcessedAlignedReadsTemplate	😄	template link
ProcessedExpressionTemplate	😄	template link
ProcessedVariantCallsTemplate	😄	template link
ProteomicsAssayTemplate	😄	template link
ProtocolTemplate	😄	template link
RNASeqTemplate	😄	template link
ScRNASeqTemplate	😄	template link
UpdateMilestoneReport	😄	template link
WESTemplate	😄	template link
WGSTemplate	😄	template link

Manifest Validation

manifest	result	expectation
GenomicsAssayTemplate_0.csv	😄	Lists can be blank if attr not required using ‘list like’ rule
GenomicsAssayTemplate_1.csv	😄	Mixing blanks and regular list values works
GenomicsAssayTemplate_2.csv	😄	Conditional validation for attributes is currently not supported
ScRNASeqTemplate_0.csv	😄	Single list val works by using ‘list like’ rule
ScRNASeqTemplate_1.csv	😄	Fail because of missing data in required field libraryStrand

[allaway]

Looks great- I had a couple of thoughts for discussion.