Question: how to register a FTS3/FTS4 custom tokenizer?

[Luminqi]

From the document of SQLite: https://www.sqlite.org/fts3.html, the way to register a custom tokenizer is executing a sql:

SELECT fts3_tokenizer(<tokenizer-name>, <sqlite3_tokenizer_module ptr>)

I am stuck on how to provide the second parameter, which is a buffer storing the pointer to the tokenizer. How can this be done in Node.js?

[Luminqi]

Find a solution:

1. write tokenizer as a sqlite extension and call SELECT fts3_tokenizer(<tokenizer-name>, <sqlite3_tokenizer_module ptr>) in extension init function
2. load the extension using loadExtension api

I am not sure if this is the best solution.