Skip to content

Commit

Permalink
fix(pandas): drop __index_level_N__ cols before applying schema
Browse files Browse the repository at this point in the history
  • Loading branch information
gforsyth authored and cpcloud committed Nov 15, 2023
1 parent 697d325 commit b53feac
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions ibis/backends/pandas/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,13 @@ def to_pyarrow(
output = pa.Table.from_pandas(
self.execute(table_expr, params=params, limit=limit, **kwargs)
)

# cudf.pandas adds a column with the name `__index_level_0__` (and maybe
# other index level columns) but these aren't part of the known schema
# so we drop them
output = output.drop_columns(
filter(lambda col: col.startswith("__index_level_"), output.column_names)
)
table = PyArrowData.convert_table(output, table_expr.schema())
return expr.__pyarrow_result__(table)

Expand Down

0 comments on commit b53feac

Please # to comment.