Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

To fix bug #55 #57

Merged
merged 1 commit into from
Feb 7, 2023
Merged

To fix bug #55 #57

merged 1 commit into from
Feb 7, 2023

Conversation

DLBPointon
Copy link
Contributor

During testing today we found a bug caused by the CSV_PULL module, I has recently begun mixing it's input channels with another instance of the module, e.g, the module was being run twice with the correct input values and yet the module would pull a variable from another instance. This resulted in the pipeline failing due to incorrect output files.

This has now been corrected, by removing 1 of the inputs which is not needed for the running of the pipeline.

The pipeline passes the local tests and s3 tests will be fixed in the future.

nf-core lint passes

prettier is failing due to local node-14 requirements

…cond was not needed). Yaml also updated to reflect new data in use
@DLBPointon DLBPointon added the bug Something isn't working label Feb 6, 2023
@DLBPointon DLBPointon added this to the Phase1.5 Completion milestone Feb 6, 2023
@DLBPointon DLBPointon self-assigned this Feb 6, 2023
@muffato
Copy link
Member

muffato commented Feb 6, 2023

What is the purpose of this csv_pull module in the first place ? It seems it's just a pass-through module, the output being the same as the input. If the goal is to make sure that the input file is downloaded to disk just once (in case it's a remote file) and used throughout, I think that's not needed because (as far as I understand !), Nextflow will anyway download the file just once under work/stage and symlink that into all the jobs that need it.

@DLBPointon
Copy link
Contributor Author

It converts the string to a path object. Without it, it will not stage anything from S3 only from local.

With the function is can do both, for local it is simply a pass-through. If there is a cleaner method then I'll happily switch over.

@muffato
Copy link
Member

muffato commented Feb 7, 2023

Ah, I see. It feels like something that could be fixed by adding a path or file somewhere, but my first attempt at it didn't work. Let's merge this pull-request, and if I find a solution I'll let you know.

@muffato muffato linked an issue Feb 7, 2023 that may be closed by this pull request
@muffato muffato merged commit 16dff6b into dev Feb 7, 2023
@muffato muffato deleted the third-round-fix branch February 7, 2023 13:50
DLBPointon pushed a commit that referenced this pull request Sep 7, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CSV_PULL creating unsorted list of input data
2 participants