Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

feat: enable experimental async use of driver #901

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

feat: enable experimental async use of driver #901

wants to merge 6 commits into from

Conversation

Mause
Copy link
Owner

@Mause Mause commented Feb 14, 2024

No description provided.

@sinaiy
Copy link

sinaiy commented Mar 24, 2024

FYI:
I am using the duckdb_engine as Async engine

I had to patch
duckdb_engine.duckdb_engine.Dialect.get_pool_class
in order to be able to set an Async Connection pool

@Mause
Copy link
Owner Author

Mause commented Apr 7, 2024

FYI: I am using the duckdb_engine as Async engine

I had to patch duckdb_engine.duckdb_engine.Dialect.get_pool_class in order to be able to set an Async Connection pool

That shouldn't be required, you should just be able to pass poolclass=... to the create_engine(...) call

@sinaiy
Copy link

sinaiy commented Apr 8, 2024

latest version of SQLAlchemy is more explicit
it adds a verification that the async status of the Dialect matches the async status of the connection pool.

I followed the psycopg3 engine example, and created AsyncDuckDbDialect as a subclass of the original dialect

import duckdb_engine

class AsyncDialect(duckdb_engine.Dialect):
    is_async = True

    @classmethod
    def get_pool_class(cls, url):
        async_fallback = url.query.get("async_fallback", False)

        if util.asbool(async_fallback):
            return pool.FallbackAsyncAdaptedQueuePool
        else:
            return pool.AsyncAdaptedQueuePool


duckdb_engine.Dialect.get_async_dialect_cls = lambda cls: AsyncDialect

@Mause Mause changed the title chore: async test feat: enable experimental async use of driver Apr 16, 2024
@Mause Mause marked this pull request as ready for review April 16, 2024 02:18
@kodiakhq kodiakhq bot removed the automerge label Apr 18, 2024
Copy link
Contributor

kodiakhq bot commented Apr 18, 2024

This PR currently has a merge conflict. Please resolve this and then re-add the automerge label.

@ShravanSunder
Copy link

ShravanSunder commented Jul 9, 2024

the method @sinaiy mentioned works for me in the mean time. here it is with type hints

import logging
import typing as t

import duckdb_engine
from pydantic_core import from_json, to_json
from sqlalchemy import URL, pool, util
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine

# Assuming the logger is already configured elsewhere
logger = logging.getLogger(__name__)


class AsyncDialect(duckdb_engine.Dialect):
  is_async = True

  @classmethod
  def get_pool_class(cls, url: URL):
    async_fallback = url.query.get("async_fallback", False)

    if util.asbool(async_fallback):
      return pool.FallbackAsyncAdaptedQueuePool
    else:
      return pool.AsyncAdaptedQueuePool


duckdb_engine.Dialect.get_async_dialect_cls = lambda cls: AsyncDialect  # type: ignore

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants