D1 bindings not present in Miniflare 3

[limzykenneth]

I'm trying to use Miniflare 3 to run automated tests for my workers and I noticed workers using D1 does not have the database properly attached to the namespace.

In the env bindings I have a D1 database bound to DB and I can see the data is stored in .wrangler/state/v3/d1/[ID] and accessible through wrangler dev but when running it through Miniflare the env binding DB is an empty object. Is D1 fully supported in Miniflare 3 already? Is there some other configuration I will need to provide? I can't find existing examples or documentation about how it should be used.

Here's my configuration to run Miniflare

mf = new Miniflare({
  scriptPath: "./dist/worker.js",
  modules: true,
  d1Databases: {
    "DB": "[DATABASE_ID]"
  }
});

[mrbbot]

Hey! 👋 Could you try __D1_BETA__DB: "[DATABASE_ID]" instead? The current D1 Wrangler shims add this extra prefix, that should be removed once D1 is out of alpha.

[limzykenneth]

Just tried that and it is still not working. I have filed the prefix issue with wrangler directly before and it has since been fixed in wrangler dev, including for wrangler 3 which I think is using workerd as well now, which is why I think this may be a Miniflare specific issue. (Actually that was for D1 in DO)

I'm still not sure why the Miniflare setup doesn't work since wrangler dev with wrangler 3 locally works without issue.

[skepticfx]

I see this behavior too. Here is what I'm using with the Miniflare 3 API

index.js

import { Miniflare } from "@miniflare/tre";

// Create a new Miniflare instance, starting a workerd server
const mf = new Miniflare({
    d1Persist: true,
    verbose: true,
    workers: [
        {
            name: "test-d1-binding",
            script: `
            export default {
                async fetch(request, env) {
                    console.log("Testing D1 bindings");
                    const db = await env.__D1_BETA__DB;
                    
                    console.debug(db);
                    console.debug(db.prepare);
                    
                    db.prepare("SELECT * FROM tablename WHERE id = 1");
                    return new Response("ok");
                },
            };`,
            routes: ["https://test-d1-binding.local/*"],
            d1Databases: { __D1_BETA__DB: "db" },

            modules: true,
        },
    ],
});

// Send a request to the workerd server, the host is ignored
const response = await mf.dispatchFetch("https://test-d1-binding.local/");
console.log(await response.text());

// Cleanup Miniflare, shutting down the workerd server
await mf.dispose();

Running with node v18.14.1

node index.js

workerd/io/worker.c++:1617: info: console.log(); message() = ["Testing D1 bindings"]
workerd/io/worker.c++:1617: info: console.log(); message() = ["[object Object]"]
workerd/io/worker.c++:1617: info: console.log(); message() = [null]
workerd/io/worker.c++:1778: info: uncaught exception; source = Uncaught (in promise); exception = TypeError: db.prepare is not a function
workerd/util/symbolizer.c++:99: warning: Not symbolizing stack traces because $LLVM_SYMBOLIZER is not set. To symbolize stack traces, set $LLVM_SYMBOLIZER to the location of the llvm-symbolizer binary. When running tests under bazel, use `--test_env=LLVM_SYMBOLIZER=<path>`.
workerd/io/io-context.c++:380: info: uncaught exception; exception = workerd/jsg/_virtual_includes/jsg/workerd/jsg/value.h:1343: failed: jsg.TypeError: db.prepare is not a function
stack: /tmp/mf3/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1a573f0 /tmp/mf3/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1a58109 /tmp/mf3/node_modules/@cloudflare/workerd-linux-64/bin/workerd@190f110 /tmp/mf3/node_modules/@cloudflare/workerd-linux-64/bin/workerd@190f740
TypeError: db.prepare is not a function

Perhaps we need to initialize the SQLite DB like in Wrangler dev?

[mrbbot]

Hey! 👋 D1's implementation is split between an internal service and a JavaScript "shim" that exposes the D1 API and makes appropriate HTTP calls to the service. The internal service is bound to your worker as a service binding, and just provides a fetch() method. The shim is added by Wrangler.

Previously, Miniflare 2 provided both the internal service and the shim for you when you specified d1Databases. In Miniflare 3, this was changed to just providing the internal service to match the behaviour of the remote runtime, and avoid duplicating the shim code in Wrangler/Miniflare.

With cloudflare/workerd#764, the shim has now been moved inside the runtime. This means we can remove the shim from Wrangler/Miniflare, and expose the internal service with the D1 API instead, without injecting anything into your user code.

It's likely the reason you're seeing an empty object when logging the binding, is that workerd essentially calls JSON.stringify() on everything before logging, and Fetchers (service binding type) have no enumerable properties, so appear as {}.

This should all be fixed by #628, which should be landing soon. 👍

[limzykenneth]

Tested the latest release and it works as intended. Thanks very much!