Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Implement Glue Catalog #249

Closed
liurenjie1024 opened this issue Mar 11, 2024 · 5 comments · Fixed by #314
Closed

Implement Glue Catalog #249

liurenjie1024 opened this issue Mar 11, 2024 · 5 comments · Fixed by #314
Milestone

Comments

@liurenjie1024
Copy link
Collaborator

Implement glue catalog.

@liurenjie1024 liurenjie1024 modified the milestone: 0.3.0 Release Mar 11, 2024
@liurenjie1024 liurenjie1024 changed the title Glue Catalog Implement Glue Catalog Mar 11, 2024
@liurenjie1024
Copy link
Collaborator Author

Blocked by #250

@marvinlanhenke
Copy link
Contributor

marvinlanhenke commented Mar 22, 2024

@liurenjie1024
... I started prototyping on this, here is what I'm trying to do:

Tasks:

  • Setup basic structure + test infra
  • Implement namespace operations
  • Implement table operations

@liurenjie1024
Copy link
Collaborator Author

Hi, @marvinlanhenke Thanks for this. I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?

@marvinlanhenke
Copy link
Contributor

marvinlanhenke commented Mar 22, 2024

I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?

I looked at the pyiceberg tests which utilize moto.
Moto provides a docker-compose to mock most of AWS Services / I tested this with a quick prototype and it seems to work fine in conjunction with the AWS SDK Rust.

https://github.com/apache/iceberg-python/blob/main/tests/catalog/test_glue.py#L23
https://docs.getmoto.org/en/latest/docs/server_mode.html#run-using-docker

@liurenjie1024
Copy link
Collaborator Author

I'm curious how do you plan to setup test infra for glue? It seems that there is no docker for glue?

I looked at the pyiceberg tests which utilize moto. Moto provides a docker-compose to mock most of AWS Services / I tested this with a quick prototype and it seems to work fine in conjunction with the AWS SDK Rust.

https://github.com/apache/iceberg-python/blob/main/tests/catalog/test_glue.py#L23 https://docs.getmoto.org/en/latest/docs/server_mode.html#run-using-docker

That's great! Thanks!

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants