Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Nf test additions #16

Merged
merged 32 commits into from
Feb 19, 2025
Merged

Nf test additions #16

merged 32 commits into from
Feb 19, 2025

Conversation

DLBPointon
Copy link
Collaborator

@DLBPointon DLBPointon commented Feb 7, 2025 •
edited
Loading

2nd pipeline to add NF-TESTS

TODO: Expand on tests with .hist and .ktab files / db

@DLBPointon DLBPointon added the enhancement New feature or request label Feb 7, 2025
@DLBPointon DLBPointon self-assigned this Feb 7, 2025
@DLBPointon DLBPointon changed the base branch from main to dev February 7, 2025 13:03
@github-actions GitHub Actions
Copy link

github-actions bot commented Feb 7, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 8a416e8

+| ✅ 148 tests passed       |+
#| ❔  35 tests were ignored |#
!| ❗  24 tests had warnings |!

❗ Test warnings:

  • files_exist - File not found: conf/igenomes_ignored.config
  • nextflow_config - Config variable not found: validation.help.beforeText
  • nextflow_config - Config variable not found: validation.help.afterText
  • nextflow_config - Config variable not found: validation.summary.beforeText
  • nextflow_config - Config variable not found: validation.summary.afterText
  • nextflow_config - Config manifest.version should end in dev: 0.7.0
  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in ro-crate-metadata.json: "description": "# sanger-tol/ear\n\nGitHub Actions CI Status\nGitHub Actions Linting StatusCite with Zenodo\nnf-test\n\nNextflow\nrun with conda\nrun with docker\nrun with singularity\nLaunch on Seqera Platform\n\n## Introduction\n\nsanger-tol/ear is a bioinformatics pipeline that ...\n\n TODO nf-core:\n Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n major pipeline sections and the types of output it produces. You're giving an overview to someone new\n to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples. \n Fill in short bullet-pointed list of the default steps in the pipeline 1. Read QC (FastQC)2. Present QC for raw reads (MultiQC)\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\nsamplesheet.csv:\n\ncsv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\nbash\nnextflow run sanger-tol/ear \\\n -profile <docker/singularity/.../institute> \\\n --input samplesheet.csv \\\n --outdir <OUTDIR>\n\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.\n\n## Credits\n\nsanger-tol/ear was originally written by DLBPointon.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the contributing guidelines.\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use sanger-tol/ear for your analysis, please cite it using the following doi: 10.5281/zenodo.XXXXXX \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.\n\nThis pipeline uses code and infrastructure developed and maintained by the nf-core community, reused here under the MIT license.\n\n> The nf-core framework for community-curated bioinformatics pipelines.\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.\n",
  • pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
  • pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • local_component_structure - sanger_tol_btk.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - generate_samplesheet.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - sanger_tol_cpretext.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - yaml_input.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • files_exist - File is ignored: CODE_OF_CONDUCT.md
  • files_exist - File is ignored: assets/nf-core-ear_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-ear_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-ear_logo_dark.png
  • files_exist - File is ignored: lib/nfcore_external_java_deps.jar
  • files_exist - File is ignored: lib/NfcoreSchema.groovy
  • files_exist - File is ignored: lib/NfcoreTemplate.groovy
  • files_exist - File is ignored: lib/Utils.groovy
  • files_exist - File is ignored: lib/WorkflowMain.groovy
  • files_exist - File is ignored: lib/nfcore_external_java_deps.jar
  • files_exist - File is ignored: lib/NfcoreSchema.groovy
  • files_exist - File is ignored: lib/NfcoreTemplate.groovy
  • files_exist - File is ignored: lib/Utils.groovy
  • files_exist - File is ignored: lib/WorkflowMain.groovy
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • files_exist - File is ignored: conf/igenomes.config
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • nextflow_config - Config variable ignored: params.validationSchemaIgnoreParams
  • files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
  • files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
  • files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting_comment.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/email_template.html
  • files_unchanged - File ignored due to lint config: assets/nf-core-ear_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-ear_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-ear_logo_dark.png
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/ear/ear/.github/workflows/awstest.yml

✅ Tests passed:

Run details

  • nf-core/tools version 3.2.0
  • Run at 2025-02-19 12:42:24

@DLBPointon
Copy link
Collaborator Author

@muffato
Could you update the tests here as well please? I thought updating the template would fix this, but I'm just more confused.

@DLBPointon
Copy link
Collaborator Author

DLBPointon commented Feb 10, 2025 •
edited
Loading

At this point, this should make up a v2

Already changed the CICD to a more test-complete version (NF-test which looks at the generated files not just for pipeline completion).
Add specified file input for hic and pacbio files <-- not for release because we also have to take into account curationpretext which currently only accepts folder path.
Generate merquryfk data for the test set
merquryfk might still have to be downloaded prior to run as the ktabs are in a folder w/ hidden file
databases will still need to be downloaded first
Update the test data
remove the unnecessary mapping, which btk does anyway

This was linked to issues Feb 12, 2025
Mapping can be removed #12
Closed
Remove mapping, instead rely on BTK #22
Closed
@DLBPointon DLBPointon merged commit 7c83e83 into dev Feb 19, 2025
4 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers
No reviews
Assignees

@DLBPointon DLBPointon

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Remove mapping, instead rely on BTK Mapping can be removed
1 participant
@DLBPointon