-
Notifications
You must be signed in to change notification settings - Fork 15
Home
Thibault Bétrémieux edited this page Oct 3, 2023
·
10 revisions
Welcome to the pangres wiki!
Install pangres
with:
pip install pangres
Here is a basic usage example (see more in the sections below):
import pandas as pd
from pangres import upsert
from sqlalchemy import create_engine
# create or get a pandas DataFrame
data = {'id': [10],
'name': ['Albert']}
df = pd.DataFrame(data)
df = df.set_index('id') # a unique index is required!
# get a SQL engine from sqlalchemy (used to connect to the SQL database)
engine = create_engine('sqlite://')
table_name = 'test'
# update given SQL table using the DataFrame
upsert(
con=engine,
df=df,
table_name=table_name,
if_row_exists='update',
dtype=None, # same logic as the parameter in pandas.to_sql
chunksize=1000,
create_table=True # create a new table if it does not exist
)
Upserting DataFrames in SQL asynchronously
Checking and adjusting the size of chunks to upsert
Fix bad column names for Postgres
See also demo notebooks below.
Demo notebook for pangres.upsert
Demo notebook for pangres.upsert
with a progress bar
Notebook with transaction control and commit-as-you-go workflows example
Gotchas with asynchronous upserts (pangres.aupsert
)
Parts of the documentation were automatically generated using the docstrings of pangres' functions/classes/methods via my library npdoc_to_md.