v0.8.4

This release of Kwil DB contains a fix to snapshot creation that could occur for records larger than 64kB, and a fix for a SQL parser bug that would prevent in-line expressions from being comparable (e.g. $1 is null without an explicit typecast).

The parse/v0.2.4 tag for the parse module accompanies this Kwil DB release.

What's Changed

• Fix a snapshot creation error with bufio.Scanner exceeding its line scan limit. This affected both kwil-admin, and the background statesync functionality in kwild. (b3460a7)
• Fix: in-line type assertions not being comparable. Actions using in-line statements with type casts for ambiguous types would not generate SQL properly. This would only occur when performing an expression (such as a comparison). Also fix a stack overflow when using the parse debugger tool via kwil-cli. (faf879e)
• Prepare v0.8.4 (version package and go.mod update for parse module) (2c71b80)

Full Changelog: v0.8.3...v0.8.4

Build

The provided application archives on this release are built with Go 1.22.5.

v0.8.3

This release of Kwil DB contains several minor fixes to the parse module, a fix to kwild's procedural language generation with unary expressions, and a fix to a possible startup crash if generated snapshot files on disk are corrupt or unreadable.

The parse/v0.2.3 tag for the parse module accompanies this Kwil DB release.

What's Changed

• engine: fix generated unary where there was a missing space (6d24370)
• kwild: fix handling invalid snapshot files (507e528)
• parse: fix unknown columns in indexes / foreign keys (7eb5a26)
• parse: removed an unneeded uuid case (16efeef)
• parse: fix Execute handling of parameters (9794a49)
• kwild,kwil-cli: use parse/v0.2.3 in the main module (5d29af4)
• version: 0.8.3 (b2d1346)

Full Changelog: v0.8.2...v0.8.3

Build

The provided application archives on this release are built with Go 1.22.4.

v0.8.2

This release of contains several bug fixes and backported feature additions.

What's Changed

• Introduce genesis consensus params to configure max limits for votebody transactions. (3d6772e)
• Add genesis and end block hooks. (881ebfb)
• Added look back for token errors. (624bff9)
• Fix/kwil admin config. (e4c2f82)
• Fixed panic on parser returns. (062c9eb)
• version: bump self report version to 0.8.2 (1130d69)
• Use parse/v0.2.2 (e525214)
• Change log level for failed resolutions (6a09f17)
• CI: Backport release workflow update (082e44d)

Full Changelog: v0.8.1...v0.8.2

v0.8.1

Kwil DB v0.8.1

This is a major new release of Kwil DB.

The highlights are:

• A new Kuneiform element called "procedures" that uses a procedural language for building logic into schemas.
• Added a JSON-RPC server.
• Network migration support.
• Enable state sync mode and snapshot creation.
• Absorb the kuneiform and dependent repositories into the parse module in the kwil-db repository.

Note that v0.8.0 was not published due to issues with the upgrade procedures.

Important Upgrading Notes

Upgrading to this release requires a network migration. See the network migration documents for instructions on how to perform a network migration. Only upgrading from v0.7 is supported.

In certain cases, deployed schemas may not be compatible with v0.8. See UPGRADING.md for details.

Notable Changes

Procedures

In previous releases, SQL queries were specified entirely within Kuneiform "actions". This release introduces the concept of "procedures", which use a basic procedural language for building logic into schemas. Procedures can be used for building access control logic, performing arithmetic, and managing control flow.

Procedures are declared using the procedure keyword. A procedure may return either a single record (a tuple) or a table.

A procedure body may contain variable declarations, basic arithmetic operations, calls to various built-in functions, control flow (e.g. if/then/for), and SQL queries.

See the procedure docs for more information.

Variable Types

Procedures are strongly typed. The parameters, returns, and variables declared within a procedure must be assigned a type. The recognized types are:

• int is a 64-bit signed integer
• text is a variable length character array (a string)
• bool is a boolean (true or false)
• blob is a byte array
• uint256 is a 256-bit unsigned integer
• decimal is a fixed precision type based
• uuid is 128-bit universally unique identifier (UUID)

Arrays of any of the above types are also permitted.

Legacy actions only support int and text in table declarations, and use no types in their signatures or bodies.

Foreign Procedure Calls

It is now possible to interact with procedures in a different database using foreign procedure declarations. For instance, to use a procedure called target_procedure in the database 'x_target_dbid', declare a foreign procedure in your database and specify the target database and procedure name when calling:

foreign procedure get_user($id uuid) returns (text, int)

procedure call_get_user($id uuid) public  {
    $username, $age = get_user['x_target_dbid', 'target_procedure']($id);
    // $username is a text and $age is an int
}

JSON-RPC Server and gRPC Gateway Deprecation

This release of kwild adds a JSON-RPC server. It listens on TCP port 8484 of all network interfaces by default. JSON-RPC requests are handled by the /rpc/v1 path.

The CLI utilities, the Go SDK client, and the JS SDK now use the JSON-RPC server by default. RPC providers should expose TCP port 8484, and client configuration should be updated to use port 8484.

The HTTP API, which is a REST gateway for the gRPC server, is now deprecated, although it listens on all network interfaces on port 8080 by default. The gRPC server, which was previously deprecated, now listens only on a loopback address to support the internal HTTP gateway (Swagger). The gRPC and the HTTP gateway will be removed in the next release.

The OpenRPC specification document for the "user" RPC service may be found in the source code repository. The server itself will also provide the server's active service specification at the /spec/v1 HTTP endpoint and in the response to the rpc.discover JSON-RPC methods.

The "params" value in all requests are named rather than positional. This means that the JSON-RPC 1.0 calling convention with an array [] for the parameters is not supported, only the JSON-RPC convention using an object {}. However, this is a detail that the Kwil clients hide from users. Only if doing manual HTTP POST requests, such as with curl or Postman, is this important.

The "admin" service and the kwil-admin tool are also updated to use JSON-RPC.

SQL Syntax

• Support for more built-in functions. See Supported Functions.
• Conflicting column names in the returned values from a query are disallowed.
  For example, this means that it is now an error if a SELECT * is used with a
  JOIN between tables with an identical column name.
• There is now a requirement that at least one side of a JOIN must be a
  table's column. It is an error to have literal or computed values on both sides. (Q: does every joined column need to be just a column, or all joined columns. What about joining on a function of a column?)

Snapshots and State Sync

State sync is a feature that allows a new node to bootstrap directly from a snapshot of the blockchain state instead of replaying all the blocks from the genesis. This reduces the time needed to sync a new node to a network. However, the node will only be aware of transactions after the snapshot height.

kwild may now be configured to periodically generate state snapshots, which are provided to nodes that are joining the network. The app.snapshots config section adjust this functionality.

To enable the use of state sync on a new node, trusted snapshot providers must be configured. The chain.statesync config section adjust this functionality.

State sync is only used for a new node. Catch up after a previously-running node has restarted uses the standard "block sync" technique, which fully validates and executes all blockchain transactions to rebuild state.

See the State Sync docs for details.

Network Migrations

To launch a new network with an initial genesis state, special state snapshots may be generated using the kwil-admin snapshot create command.

When launching a network from this genesis state, the app_hash field of the new genesis.json file will be non-null. The initial validator nodes must use the genesis_state config.toml setting to specify the snapshot that corresponds to the app_hash value. Once a trusted snapshot provider has generated an initial snapshot for the new running network, other nodes may rely on the state sync mechanism to join the network without having to manually distribute the genesis_state file.

See Network Migration documents for details.

Coordinated Upgrades

This release includes an experimental system for specifying planned changes to code that affects consensus. This is an presently experimental system that is being evaluated to help avoid the need for network migrations when important fixes and updates might otherwise break consensus. The the documentation for details.

Command Line Application Changes

User CLI (kwil-cli) Changes

Moved kwil-cli Configuration Folder

The default config folder for kwil-cli is now called kwil-cli in the user's home directory. It was previously called kwil_cli.

New RPC Provider Setting

Within config.json, the RPC provider is specified with a "provider" field. This was previously called "grpc_url".

The --kwil-provider command line flag is DEPRECATED and will be removed in the subsequent release. Use the --provider flag instead.

When updating for the new provider setting, ensure it corresponds to the JSON-RPC server at port 8484 instead of the REST API at port 8080. Otherwise, ensure that the chosen RPC provider is updated to direct to the JSON-RPC server.

Command Changes

The result from the utils query-tx command no longer includes the transaction details for an unconfirmed transaction. Confirmed transactions will still include the decoded transaction. A new --raw flag may be provided with this command to retrieve the serialized transaction data given a transaction hash.

Added the utils parse command to parse a Kuneiform file and output the schema as JSON.

Added the utils decode-tx command to decode a serialized transaction provided as a base64 string.

Many of the fields in the JSON objects returned by query-tx and other commands with the --output json option are renamed with consistent snake case formatting. See the See UPGRADING.md for details on changes to the core module types and their JSON tags.

The account balance command may now be given a --pending flag to return account information that includes changes that consider any unconfirmed transactions. This is useful for determining the next nonce for an account when broadcasting multiple transaction per block.

Node Application (kwild) Changes

The default setting for the admin service (admin_listen_addr) is now "/tmp/kwild.socket".

The jsonrpc_listen_addr setting replaces the grpc_listen_addr setting. The default value for jsonrpc_listen_addr is 0.0.0.0:8484 (listen on port 8484 on all network interfaces).

In addition to standard output, kwild also writes the log to "kwild.log" in the application's root directory by default. The output_paths setting is used to change this.

Two new timeouts are added:

• rpc_timeout sets a timeout on requests on the user R...

v0.7.6

What's Changed

• kwild,rpc: db and rpc timeout settings (235a1f1). This introduces two timeout settings: rpc_timeout and db_read_timeout. The DB timeout applies to DB read operations in the Call and Query service methods. The RPC timeout applies to all RPCs, and includes the entire request-response roundtrip.
• pg: always release conn on commit/rollback error (5824d29). This resolves a potential leak of DB reader connections, and a shutdown hang.

Full Changelog: v0.7.5...v0.7.6

There are no changes to the core or parse modules.

Build

This release is built with Go 1.22.3

v0.7.5

What's Changed

• Fix loading of environment variables for kwild config. (a325df6). Environment variables for all kwild config settings are now being respected. They were incorrectly bound previously.
• core: kgw cookie on multi providers (5bdd5c3). Add support for kgw cookies that contain tokens for multiple domains.
• Update go.mod for core module version bump (c4e11f5).
• kwild: fix cometbft client not setting "syncing" (2730871). This fixes the "syncing" field of the node status command response always being false.

Full Changelog: v0.7.4...v0.7.5

Build

This release is built with Go 1.22.3

v0.7.4

What's Changed

• kwild: fixes bug where oracle txs were not properly voted on in single-node networks (57cb1d6)
• kwild: fixes bug where schema extensions are not properly returned by the RPC service (1e4abb3)
• kwild: returns a declared error type when an invalid signature is verified in the functions RPC (4863d69)
• core: gwclient auth precheck (577f5e4)
• core readme and example app (62f3099)

(since v0.7.2, as v0.7.3 was unreleased)

Full Changelog: v0.7.2...v0.7.4

v0.7.2

What's Changed

• core: fix parse kgw error, so it output detailed message (1417711). Correctly parse common kgw errors, so it outputs detailed a message.
• version: bump to 0.7.2 in vanilla go builds (b22a5f2).
• ci: test kgw in release workflow (1744789).
• http: strip trailing slashes from "BasePath" (d6820ac) This change fixes our http client constructors to trim trailing slashes.
• ci: run kwil-js test (0df049b)
• gatewayclient: fix domain compare when match cookie (e567538). Prevent re-prompting for authentication because of incorrect cookie matching based on a domain string that included port
• use core/v0.1.2 for release (4b1819b).

Full Changelog: v0.7.1...v0.7.2

v0.6.7

What's Changed

• added read timeouts for registry SQL queries (e5216ad).
• core/rpc/client/user/http: read and interpret error messages (47ccd8b). Add uniform error parsing to each of the http client methods. The server doesn't emit "unknown" status codes resulting in http responses with 500 status codes.
• core: fix parse kgw error, so it output detailed message (83d4844). Correctly parse common kgw errors, so it outputs detailed a message.
• bump version string to v0.6.7 (ad4a4eb).
• http: strip trailing slash from "BasePath" (7226024). This change fixes our http client constructors to trim trailing slashes
  from the base URL, preventing confusing 404 not found errors from kwil-cli.
• gatewayclient: fix domain compare when match cookie (0bbb562). Prevent re-prompting for authentication because of incorrect cookie matching based on a domain string that included port.

Full Changelog: v0.6.6...v0.6.7

v0.7.1

What's Changed

• abci,kwild: write in-memory cometbft confs to disk for inspect (69adf12). The cometbft command line application's inspect and rollback commands are supported on the "abci" subfolder of kwild's root directory, while the node is stopped. A rollback will require recreating the entire PostgreSQL database to replace all blocks.
• engine: create a max procedure call stack depth (b74380e). Limit action recursion depth.
• sql,pg,engine: rework sql interfaces (6ef9117). This internal change does not affect users or node operation.
• abci,txapp,kwild: new chain meta store (5dc2d6b, ff06172). Migrate the applications metadata store, which contains current best block, into the PostgreSQL tables with the other application data. This prevents a number of deployment gotchas.
• server,pg: fix shutdown sequence (8326adf). This makes shutdown more robust, preventing a possible apphash computation error for a block being finalized during shutdown.

Full Changelog: v0.7.0...v0.7.1