Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add data source Trino #662

Merged
merged 4 commits into from
Jul 9, 2024
Merged

Add data source Trino #662

merged 4 commits into from
Jul 9, 2024

Conversation

grieve54706
Copy link
Contributor

@grieve54706 grieve54706 commented Jul 5, 2024

Description

Make POST /v2/connector/{data_source}/query support new data source trino

Request body

  • host: string, required
  • port: integer, required
  • catalog: string, required
  • schema: string, required
  • user: string, required
  • password: string
{
  "sql": "select * from \"Orders\" limit 1",
  "manifestStr": "eyJjYXRhbG9nIjoibXlfY2F0YWxvZyIsInNjaGVtYSI6Im15X3...",
  "connectionInfo": {
    "host": "localhost",
    "port": 8080,
    "catalog": "catalog",
    "schema": "schema",
    "user": "username",
    "password": "password"
  }
}

or

  • connectionUrl: string, required
{
  "sql": "select * from \"Orders\" limit 1",
  "manifestStr": "eyJjYXRhbG9nIjoibXlfY2F0YWxvZyIsInNjaGVtYSI6Im15X3...",
  "connectionInfo": {
    "connectionUrl": "trino://username:password@localhost:8080/catalog/schema"
  }
}

Response body

  • columns
  • data
  • dtypes: Type through the pandas
{
    "columns": [
        "o_orderkey",
        "o_custkey",
        "o_orderstatus",
        "o_totalprice",
        "o_orderdate",
        "o_orderpriority",
        "o_clerk",
        "o_shippriority",
        "o_comment"
    ],
    "data": [
        [
            1,
            370,
            "O",
            172799.49,
            820540800000,
            "5-LOW",
            "Clerk#000000951",
            0,
            "nstructions sleep furiously among "
        ]
    ],
    "dtypes": {
        "o_orderkey": "int32",
        "o_custkey": "int32",
        "o_orderstatus": "object",
        "o_totalprice": "object",
        "o_orderdate": "object",
        "o_orderpriority": "object",
        "o_clerk": "object",
        "o_shippriority": "int32",
        "o_comment": "object"
    }
}

Additional information

Add taplo to format toml.
Testcontainers does not support Trino, so I created TrinoContainer for ourselves. I will add a PR to testcontainers-python.

@grieve54706 grieve54706 marked this pull request as ready for review July 8, 2024 06:20
@grieve54706
Copy link
Contributor Author

Copy link
Contributor

@goldmedal goldmedal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @grieve54706 overall looks good but some minor comments.

ibis-server/app/main.py Outdated Show resolved Hide resolved
ibis-server/justfile Show resolved Hide resolved
ibis-server/docker-compose.yml Outdated Show resolved Hide resolved
@goldmedal goldmedal merged commit f6f558d into main Jul 9, 2024
4 checks passed
@goldmedal goldmedal deleted the feature/trino branch July 9, 2024 06:20
grieve54706 added a commit that referenced this pull request Dec 13, 2024
* Add taplo

* Add data source Trino

* Fix import

* Add execute to trigger node register
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants