Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Issue found on page 'Loading JSON' #3977

Open
maxmoehl opened this issue Oct 28, 2024 · 1 comment
Open

Issue found on page 'Loading JSON' #3977

maxmoehl opened this issue Oct 28, 2024 · 1 comment
Assignees
Labels

Comments

@maxmoehl
Copy link

I was importing large JSON objects with many fields and duckdb refused to expand each field into its own column and instead give me a single STRUCT... or MAP(VARCHAR, VARCHAR) column. When messing around with the parameters I stumbled across this list of available arguments:

D SELECT * FROM read_json('./test-flat.json', format='array', auto_detectt=true) LIMIT 1;
Binder Error: Invalid named parameter "auto_detectt" for function read_json
Candidates:
    map_inference_threshold BIGINT
    records VARCHAR
    timestampformat VARCHAR
    field_appearance_threshold DOUBLE
    date_format VARCHAR
    dateformat VARCHAR
    sample_size BIGINT
    columns ANY
    convert_strings_to_integers BOOLEAN
    format VARCHAR
    ignore_errors BOOLEAN
    maximum_object_size UINTEGER
    maximum_depth BIGINT
    auto_detect BOOLEAN
    union_by_name BOOLEAN
    maximum_sample_files BIGINT
    compression VARCHAR
    timestamp_format VARCHAR
    hive_types ANY
    hive_partitioning BOOLEAN
    hive_types_autocast BOOLEAN
    filename ANY

It seems like not all of them are documented, the one I needed in my case was map_inference_threshold which apparently has a default which is too low for my objects, after setting it to -1 the columns got generated. Searching via GitHub also yields no results.

Page URL: https://duckdb.org/docs/data/json/loading_json.html

@szarnyasg
Copy link
Collaborator

Hi @maxmoehl, thanks for reporting this. Indeed there are missing flags – I'm working on documenting them in #3922.

@szarnyasg szarnyasg self-assigned this Oct 30, 2024
@szarnyasg szarnyasg added the json label Oct 30, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants