Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

script to parallelize uploads (GDEV-1502) #3

Merged
merged 21 commits into from
Jan 18, 2023
Merged

script to parallelize uploads (GDEV-1502) #3

merged 21 commits into from
Jan 18, 2023

Conversation

KerstenBreuer
Copy link
Contributor 

Added a script that takes a tsv file that lists files to be uploaded as input
and triggers the upload of multiple files in parallel.

@KerstenBreuer KerstenBreuer requested a review from PMenges January 11, 2023 10:58
PMenges
PMenges approved these changes Jan 11, 2023
View reviewed changes
Copy link
Member

@PMenges PMenges left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Tested and worked on the odcf cluster.

PMenges
PMenges previously approved these changes Jan 11, 2023
View reviewed changes
MoritzHahn1337
MoritzHahn1337 previously approved these changes Jan 11, 2023
View reviewed changes
Copy link

@MoritzHahn1337 MoritzHahn1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

mephenor
mephenor previously approved these changes Jan 12, 2023
View reviewed changes
KerstenBreuer and others added 5 commits January 13, 2023 14:35
@KerstenBreuer @mephenor
Update src/otp_tsv_to_upload.py
1fa8808 
Co-authored-by: Thomas Zajac <thomas-jakob.zajac@uni-tuebingen.de>
@KerstenBreuer
Update src/otp_tsv_to_upload.py
4b92028 
Co-authored-by: Moritz Hahn <Moritz.Hahn@uni-tuebingen.de>
@KerstenBreuer
make file paths absolute
7341805
@KerstenBreuer
Merge branch 'feature/tsv_to_upload_job_GDEV_1502' of https://github.…
fca1a7d 
…com/ghga-de/data-steward-scripts into feature/tsv_to_upload_job_GDEV_1502
@KerstenBreuer
fix
33bf75f
mephenor
mephenor previously approved these changes Jan 13, 2023
View reviewed changes
KerstenBreuer and others added 3 commits January 18, 2023 12:44
@KerstenBreuer
resolve
64df74d
@KerstenBreuer @mephenor
Update src/otp_tsv_to_upload.py
8eb5bdf 
Co-authored-by: Thomas Zajac <thomas-jakob.zajac@uni-tuebingen.de>
@KerstenBreuer
Merge branch 'feature/tsv_to_upload_job_GDEV_1502' of https://github.…
b9e6d9e 
…com/ghga-de/data-steward-scripts into feature/tsv_to_upload_job_GDEV_1502
PMenges
PMenges approved these changes Jan 18, 2023
View reviewed changes
Copy link
Member

@PMenges PMenges left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and worked on the odcf cluster.

@KerstenBreuer KerstenBreuer merged commit 3fc36ee into main Jan 18, 2023
@KerstenBreuer KerstenBreuer deleted the feature/tsv_to_upload_job_GDEV_1502 branch January 18, 2023 14:21
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@PMenges PMenges PMenges approved these changes

@mephenor mephenor mephenor approved these changes

@MoritzHahn1337 MoritzHahn1337 MoritzHahn1337 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@KerstenBreuer @PMenges @mephenor @MoritzHahn1337