rowexec: upgrade hyperloglog for table stats processors

[yuzefovich]

sql: stop storing statement in the context

We currently store the statement in the context as a value in the hopes of enriching the sentry reports. Anecdotally, most reports don't get the stmt information (because the current mechanism is missing some code paths), and even when it's present in most cases the report is still unactionable. A more lengthy discussion can be found in this slack thread.

This commit removes this altogether in order to free up a spot in the "fast values" context storage that will be utilized by the following commit. We also try to include the plan gist into sentry reports, and I find that a bit more helpful, so I chose to keep plan gists over the stmts.

Fixes: #38516.

execversion: extract DistSQL version into a new package

The following commits will introduce dependency on this version into other low-dependency packages, so this extraction is needed to avoid import cycles.

execversion: refactor DistSQL version

This commit refactors the DistSQL version mechanism to better serve our needs. When it was first introduced, it was too coarse since a change in the logic in one processor would make all physical plans not be distributable in the mixed-version state, so we have avoided performing the DistSQL version bumps since 23.1 cockroach version. However, we still need a mechanism to perform execution logic "upgrades".

The main idea is that we will need to keep the code that can execute every version in [MinAccepted, Latest] range, and we’d have execution logic conditional on the version that is put into SetupFlowRequest. The gateway for the query will be consulting cluster version to decide which DistSQL exec version is safe to use. MinAccepted can only be bumped (and the corresponding execution code retired / conditionals removed) once compatibility with the cluster version that uses it is no longer needed.

For example, right now DistSQL version is 71, and it’s what all 24.3 and prior binaries use; say in 25.1 we introduce a change in the execution engine, then we bump the version to 72 but keep MinAccepted as 71. In mixed-version 24.3-25.1 cluster we’ll be using version 71 so that all nodes can participate in the distributed plan. Once compatibility with 24.3 is no longer needed, we can bump MinAccepted to 72 and remove the old execution code.

In order to reduce the amount of plumbing necessary for these migrations, we'll store the DistSQL exec version in the context. This commit already bumps the 25.1 exec version to 72 (in anticipation of its usage in the following commit).

Fixes: #98550.

rowexec: upgrade hyperloglog for table stats processors

This commit utilizes the updated DistSQL version mechanism to upgrade the hyperloglog library used for the distinct count estimation in the sampler and sample aggregator. In sketchInfo struct we now have fields corresponding to the old and the new versions and we populate only one, based on the DistSQL version provided on the processor creation via the context. Only one of the two fields is updated throughout the execution, so there is no confusion when unmarshalling the sketch datum.

It turns out that we already had a sketch version baked into the sketch proto, so we could have used that to perform this upgrade. However, we should rely on the universal DistSQL version mechanism instead, so this commit also begins the process of removing the sketch version from the proto by deleting the version checks.

Fixes: #138546.

Release note: None

[cockroach-teamcity]

This change is 

[mgartner]

Nice!

Is there an easy way to add this regression test? #137386 (comment)

Reviewed 2 of 2 files at r1, 18 of 18 files at r2, 10 of 10 files at r3, 8 of 8 files at r4, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @michae2 and @yuzefovich)

---

-- commits line 4 at r1:
nit: currenntly => currently

---

pkg/sql/execversion/version.go line 18 at r3 (raw file):

// execution logic to be used for a particular DistSQL flow and is picked by the
// gateway node after consulting the cluster version.
type DistSQLVersion uint32

🚲 shed: should this type be called simply V?

[yuzefovich]

I spent some time trying to figure out how to add a regression test for this. I could think of two options:

• a full-blown roachtest, which seems like an overkill, and we do get that coverage implicitly via other mixed-version roachtests
• using cockroach-go-testserver-configs logic test config which in theory is exactly what we want. However, I was unable to get the test to fail with the revert of stats: revert upgrade of hyperloglog library #138358 and not sure why. I also often run into errors on EXPERIMENTAL_RELOCATE commands after having upgraded a node in the cluster, even with retry directive, so it seems like the test could be flaky. Perhaps we need to harden this config a bit. I'll reach out to Foundations to ask if they have any advice.

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @michae2)

---

pkg/sql/execversion/version.go line 18 at r3 (raw file):

Previously, mgartner (Marcus Gartner) wrote…

🚲 shed: should this type be called simply V?

Nice, I like it, done.

[yuzefovich]

Added a regression test via the logic test framework. There were a couple of difficulties with the framework itself (see here) that I stumbled upon and will improve in a separate patch. Let's see whether CI accepts this test (it works locally and properly failed when reverting #138358).

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @michae2)

[yuzefovich]

Looks like CI is green (modulo unrelated flake), so let's keep the test. I'm a bit concerned that it'll be flaky, and if so, we can remove it later.

TFTR!

bors r+

[craig]

Build succeeded:

• linux_arm64_build
• macos_amd64_build
• acceptance
• docker_image_amd64
• local_roachtest_fips
• check_generated_code
• macos_arm64_build
• linux_amd64_build
• unit_tests
• local_roachtest
• linux_amd64_fips_build
• examples_orms
• lint
• windows_build

[mgartner]

Nice job figuring out the test. 🤞 it's not flaky.