Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Set SQLITE_LIMIT_FUNCTION_ARG to SQLite's normal default. #2493

Merged
merged 1 commit into from
Aug 9, 2024

Conversation

kentonv
Copy link
Member

@kentonv kentonv commented Aug 7, 2024

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412

People keep legitimately running up against this limit. It's not really clear what value it is providing, so let's just match the default.

Fixes #2412
@kentonv kentonv requested review from elithrar and geelen August 7, 2024 18:23
@kentonv kentonv requested review from a team as code owners August 7, 2024 18:23
@justin-mp
Copy link
Contributor

Just to add color here, according to https://www.sqlite.org/limits.html (search for "Maximum Number Of Arguments On A Function"), the default is 100 and absolute maximum is 127 because this value is sometimes stored in an unsigned char.

Sure enough, sqliteLimit.h has:

#ifndef SQLITE_MAX_FUNCTION_ARG
# define SQLITE_MAX_FUNCTION_ARG 127
#endif

I can't seem to figure out where the default of 100 is set.

@@ -809,7 +809,9 @@ void SqliteDatabase::setupSecurity() {
// https://www.sqlite.org/limits.html#max_compound_select
sqlite3_limit(db, SQLITE_LIMIT_COMPOUND_SELECT, 5);
sqlite3_limit(db, SQLITE_LIMIT_VDBE_OP, 25000);
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/default/maximum/ ?

@kentonv
Copy link
Member Author

kentonv commented Aug 7, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

My intent was to use the default.

I guess I can try to find out what sqlite actually reports its own default as being...

@justin-mp
Copy link
Contributor

justin-mp commented Aug 7, 2024

Digging more: the default is 127 and https://www.sqlite.org/limits.html is buggy. Here's what my sqlite3 reports:

sqlite> .limit
              length 1000000000
          sql_length 1000000000
              column 2000
          expr_depth 1000
     compound_select 500
             vdbe_op 250000000
        function_arg 127
            attached 10
 like_pattern_length 50000
     variable_number 250000
       trigger_depth 1000
      worker_threads 0

The limits from the compile-time constants are initialized into a static const int aHardLimit[] and then that array is copied into the per-database limit in openDatabase(...):

  assert( sizeof(db->aLimit)==sizeof(aHardLimit) );
  memcpy(db->aLimit, aHardLimit, sizeof(db->aLimit));

So it's the default and maximum.

sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 32);
// For SQLITE_LIMIT_FUNCTION_ARG we use the default instead of the "security" recommendation
// because there are too many valid use cases for large argument lists, especially json_object.
sqlite3_limit(db, SQLITE_LIMIT_FUNCTION_ARG, 127);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this might change in the future, maybe we should add a link to the existing documentation?

@anonrig
Copy link
Member

anonrig commented Aug 8, 2024

Weird, https://www.sqlite.org/security.html says that the default is 127, not 100.

PS: https://www.sqlite.org/c3ref/create_function.html Also mentions max 127 value.

@kentonv
Copy link
Member Author

kentonv commented Aug 9, 2024

OK seems like this is correct as-is, just gonna merge.

@kentonv kentonv merged commit e0c1e78 into main Aug 9, 2024
9 checks passed
@kentonv kentonv deleted the kenton/sqlite-arg-limit branch August 9, 2024 21:27
@HendrixString
Copy link

@kentonv Hi,

when is this gonna be released up stream into production ?

Unfortunately, i am hitting the upper limit of 32 (I have 36 params in json_object)

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error: too many arguments on function JSON_OBJECT - 🐛 Bug Report — Runtime APIs
4 participants