Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[zip] open Parquet files from inside an archive #1916

Closed
ajkerrigan opened this issue Jun 8, 2023 · 1 comment · Fixed by #2133
Closed

[zip] open Parquet files from inside an archive #1916

ajkerrigan opened this issue Jun 8, 2023 · 1 comment · Fixed by #2133
Labels

Comments

@ajkerrigan
Copy link
Collaborator

Small description

If we open a Zip file in VisiData and then try to open a Parquet file inside that Zip, the pyarrow.parquet reader looks for the file on the local filesystem outside the zip.

Expected result

What happens in the Zip stays in the Zip. Or something.

Actual result with screenshot
If you get an unexpected error, please include the full stack trace that you get with Ctrl-E.

The screenshot is an empty sheet, which isn't super helpful.

Steps to reproduce with sample data and a .vd

Adapted from here:

Create a Parquet file, then add it to a zip in the same directory:

├── test.zip
│   ├── benchmark.parquet
benchmark.parquet
  • Try to run vd test.zip and open the Parquet file inside. It should work.
  • Quit VisiData and remove the Parquet file outside the zip.
  • Try opening test.zip again and then the Parquet file inside. It should not work anymore.

Please attach the commandlog (saved with Ctrl-D) to show the steps that led to the issue.
See here for more details.

Additional context
Please include the version of VisiData and Python.

@saulpw
Copy link
Owner

saulpw commented Oct 20, 2023

Thanks for the report, @ajkerrigan ! This would be useful, but I think it would be tricky without extracting the parquet file to disk first. I think it's more of a wishlist item. At best we could fail with a more appropriate error instead of trying to incorrectly read from the filesystem.

Kondo'ed for now.

@saulpw saulpw added wishlist and removed bug labels Oct 20, 2023
@saulpw saulpw changed the title Can't open Parquet files from inside an archive [zip] open Parquet files from inside an archive Oct 20, 2023
@saulpw saulpw closed this as not planned Won't fix, can't repro, duplicate, stale Oct 20, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants