Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

dvc import: breaks progress bar when importing folders #10677

Closed
petrchmelar opened this issue Jan 28, 2025 · 0 comments · Fixed by #10678
Closed

dvc import: breaks progress bar when importing folders #10677

petrchmelar opened this issue Jan 28, 2025 · 0 comments · Fixed by #10678

Comments

@petrchmelar
Copy link
Contributor

Bug Report

dvc import: breaks progress bar when importing folders

Description

It seems that progress bar is not working correctly when importing folder.

Image

Reproduce

# prepare repo
mkdir reproducer && cd reproducer
git init && dvc init 

# generate dummy data
mkdir data
dd if=/dev/zero of=./data/file0 bs=4k iflag=fullblock,count_bytes count=1G
dd if=/dev/zero of=./data/file1 bs=4k iflag=fullblock,count_bytes count=1G

# commit data
dvc add data
git add .
git commit -m "Initial commit" 

# try to import
dvc import . -o imported-data data

Expected

Progress bar working.

Environment information

Output of dvc doctor:

╰─❯ dvc doctor
DVC version: 3.59.0 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-5.15.0-117-generic-x86_64-with-glibc2.35
Subprojects:
        dvc_data = 3.16.8
        dvc_objects = 5.1.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.3.9
Supports:
        http (aiohttp = 3.11.11, aiohttp-retry = 2.9.1),
        https (aiohttp = 3.11.11, aiohttp-retry = 2.9.1),
        s3 (s3fs = 2024.9.0, boto3 = 1.36.3)
Config:
        Global: /home/pchmelar/.config/dvc
        System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: nfs4 on 10.11.72.32:/mlops_data/workplace
Caches: local
Remotes: None
Workspace directory: nfs4 on 10.11.72.32:/mlops_data/workplace
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/f179e68f4c2a6cf27b5dc4aa96041e39

Additional Information (if any):

It can be probably fixed by adding child progress bars on line https://github.com/iterative/dvc/blob/main/dvc/fs/dvc.py#L581
I will prepare bugfix pull request

petrchmelar pushed a commit to petrchmelar/dvc that referenced this issue Jan 28, 2025
Add child progress bar to dvc fs to report directory import progress correctly.

Fixes iterative#10677
petrchmelar pushed a commit to petrchmelar/dvc that referenced this issue Jan 28, 2025
Add child progress bar to dvc fs to report directory import progress correctly.

Fixes iterative#10677
petrchmelar pushed a commit to petrchmelar/dvc that referenced this issue Jan 28, 2025
Add child progress bar to dvc fs to report directory import progress correctly.

Fixes iterative#10677
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant