Do not refcount Trace instances in TraceItem API

[jasnell]

The underlying io Trace objects are kj heap allocated. They cannot be referenced unprotected by the TraceItem jsg object and we do not want to restrict them to the IoContext lifetime. Instead we'll extract the detail we need on TraceItem creation.

[ohodson]

Looks okay here, but definitely covers ground I'm not familiar with.

[kentonv]

Added @jclee as a reviewer and removed myself since he wrote a lot of this originally.

[jclee]

So, the motivation for the original design was to minimize allocation overhead, if a single trace record needed to pass through several trace handlers, each of which might only examine a portion of the data. I guess this hasn't turned out to be a major concern today, with only a few trace handlers installed. It does mean, though, that an entire copy of the trace data needs to be made on each trace handler invocation, which seems like a pity, for something that is generally request-scoped, or at least pipeline-scoped, so the work done during the trace handler will be proportional to the amount of trace data recorded regardless of what the trace handler is doing... e.g. if it's only being used to collect a few request metrics.

But maybe I'm overthinking it... The cost wouldn't be more than was already paid during the trace recording, and we do have safeguards in place to limit the size of the recorded data.

If I'm understanding the problem correctly:

• We see refcount errors due to the refcount being modified in multiple threads.
• This is most likely due to accesses in a GC thread conflicting with an access in a request thread.
• One way we address similar problems is by using kj::AtomicRefcounted, but we don't want to do that here due to overhead.
• We address similar situations for kj I/O objects via IoOwn, but that assumes the value is request-scoped, which might not be the case for these objects.
• We don't have an existing analogous solution for things that are pipeline-scoped.

Hm... I suppose if we have to make a copy, long-term, it might be more efficient to just pass around something that's a bunch of views onto a single arena allocation, such the allocation and copy can just be done once. Or, perhaps, just a rpc::Trace::Reader?

[jasnell]

It does mean, though, that an entire copy of the trace data needs to be made on each trace handler invocation...

We could potentially get around that by following a pattern similar to the new Detail struct this PR introduces. Specifically, use kj::Refcounted instances that are guaranteed to be own entirely by the isolate rather than split between the kj heap and the isolate heap. I do have to wonder, however, how much of an actual issue that's going to be tho.

Hm... I suppose if we have to make a copy, long-term, it might be more efficient to just pass around something that's a bunch of views onto a single arena allocation, such the allocation and copy can just be done once. Or, perhaps, just a rpc::Trace::Reader?

Maybe? I've not dug in that far on this. There's likely lots of better ways of improving this.