You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi @saulpw@anjakefala@takacsd - it seems that forcing opening the path as file with .open() - introduced with #2133 - breaks the use case where the multiple parquet files are stored in a directory, and this directory is then read by visidata. This is common with Hive partitioning or when working with spark. A simple fix would be to check if the path is a directory with os.path.is_dir() and then retaining old behavior of passing it as a string to read_table(). If it is not an existing directory, we move to the new way of opening as a binary buffer.
I have already added this workaround to my clone of visidata, and it fixes my issue, but maybe you have some better ideas how to handle it instead of if-else statement in the ParquetSheet.
Small description
Hi @saulpw @anjakefala @takacsd - it seems that forcing opening the path as file with
.open()
- introduced with #2133 - breaks the use case where the multiple parquet files are stored in a directory, and this directory is then read by visidata. This is common with Hive partitioning or when working with spark. A simple fix would be to check if the path is a directory withos.path.is_dir()
and then retaining old behavior of passing it as a string toread_table()
. If it is not an existing directory, we move to the new way of opening as a binary buffer.I have already added this workaround to my clone of visidata, and it fixes my issue, but maybe you have some better ideas how to handle it instead of
if-else
statement in theParquetSheet
.Expected result
should load a parquet into visidata
Actual result with screenshot
Additional context
# freshest develop visidata@9fd728b72c115e50e99c24b455caaf020381b48e pyarrow==12.0.0 python 3.10.2
The text was updated successfully, but these errors were encountered: