Potential Proposal: Cloudberry DB Extension Compatibility Framework

[edespino]

Background

Recently, a question was raised in our Slack channel about ensuring interoperability between Cloudberry DB tagged releases and the latest versions of extensions (e.g., pgvector). This got me thinking about how we might improve our approach to extension compatibility.

Potential Proposal

I'm considering putting together a proposal for a more structured approach to managing extension compatibility in Cloudberry DB. Before diving into a full proposal, I'd like to gauge community interest and gather initial thoughts.

The framework could potentially include:

1. Extension Testing Framework: A standardized way to test extensions across different Cloudberry DB versions and in MPP environments.

2. Community-Driven Testing: Encouraging community members to volunteer for testing specific extensions (e.g., hll, pgvector).

3. Collaborative Process Development: Working together to define how we validate and maintain extension compatibility.

4. CI/CD Integration: Integrating extension compatibility checks into our development pipeline.

5. Documentation: Creating guides and best practices for extension compatibility.

6. Regular Check-ins: Establishing periodic community discussions on extension compatibility.

7. Compatibility Roadmap: Aligning Cloudberry DB's development with key extensions to preempt issues.

Questions for the Community

1. Do you see value in developing such a framework?
2. Which aspects do you think are most crucial or interesting?
3. Would you be interested in contributing to this effort if we move forward?
4. Are there any additional considerations you think should be included?

Next Steps

Based on the feedback received, we'll determine whether to proceed with a full proposal and how to shape it to best serve the community's needs.

Your thoughts and input are greatly appreciated. Let's discuss!

[edespino]

For reference, here are the current hll and pgvector test output:

PG HLL

06:54:32 gpadmin@build-rocky9-instance-0 postgresql-hll ±| ✗|→       make installcheck REGRESS_OPTS='--init-file=/home/gpadmin/cloudberrydb/src/test/regress/init_file'
/usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/../../src/test/regress/scan_flaky_fault_injectors.sh && /usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/local/cloudberry-db-1.6.0/bin'    --load-extension=gp_inject_fault --init-file=/home/gpadmin/cloudberrydb/src/test/regress/init_file setup add_agg agg_oob auto_sparse card_op cast_shape copy_binary cumulative_add_cardinality_correction cumulative_add_comprehensive_promotion cumulative_add_sparse_edge cumulative_add_sparse_random cumulative_add_sparse_step cumulative_union_comprehensive cumulative_union_explicit_explicit cumulative_union_explicit_promotion cumulative_union_probabilistic_probabilistic cumulative_union_sparse_full_representation cumulative_union_sparse_promotion cumulative_union_sparse_sparse disable_hashagg equal explicit_thresh hash hash_any meta_func murmur_bigint murmur_bytea nosparse notequal scalar_oob storedproc transaction typmod typmod_insert union_op
scanning for flaky fault-injection tests...
done
(using postmaster on Unix socket, default port)
============== dropping database "regression"         ==============
DROP DATABASE
============== faultinjector enabled                  ==============
============== creating database "regression"         ==============
CREATE DATABASE
ALTER DATABASE
============== installing gp_inject_fault             ==============
CREATE EXTENSION
============== installing gp_toolkit                  ==============
CREATE EXTENSION
============== checking external_fts status           ==============
External fts disabled. Using default answer files
============== checking optimizer status              ==============
Optimizer enabled. Using optimizer answer files whenever possible
============== checking gp_resource_manager status    ==============
Resource group disabled. Using default answer files
============== checking gp_internal_is_singlenode status ==============
Normal cluster detected. Replace '@hostname@' by hostname of contentid = 0
============== checking force_parallel_mode status    ==============
Force parallel mode disabled. Using default answer files
============== running regression test queries        ==============
test setup                        ... ok           99 ms
test add_agg                      ... ok           93 ms
test agg_oob                      ... ok          219 ms
test auto_sparse                  ... ok           20 ms
test card_op                      ... ok           25 ms
test cast_shape                   ... ok           20 ms
test copy_binary                  ... ok           66 ms
test cumulative_add_cardinality_correction ... FAILED      304 ms
test cumulative_add_comprehensive_promotion ... FAILED      335 ms
test cumulative_add_sparse_edge   ... FAILED       82 ms
test cumulative_add_sparse_random ... ok           75 ms
test cumulative_add_sparse_step   ... ok           74 ms
test cumulative_union_comprehensive ... ok         1228 ms
test cumulative_union_explicit_explicit ... ok          142 ms
test cumulative_union_explicit_promotion ... FAILED      400 ms
test cumulative_union_probabilistic_probabilistic ... ok         3072 ms
test cumulative_union_sparse_full_representation ... FAILED       96 ms
test cumulative_union_sparse_promotion ... FAILED     3244 ms
test cumulative_union_sparse_sparse ... ok          458 ms
test disable_hashagg              ... FAILED      179 ms
test equal                        ... ok           37 ms
test explicit_thresh              ... ok           41 ms
test hash                         ... ok           58 ms
test hash_any                     ... ok           39 ms
test meta_func                    ... ok           53 ms
test murmur_bigint                ... ok           50 ms
test murmur_bytea                 ... ok           63 ms
test nosparse                     ... ok           17 ms
test notequal                     ... ok           35 ms
test scalar_oob                   ... ok           44 ms
test storedproc                   ... ok           34 ms
test transaction                  ... ok           24 ms
test typmod                       ... ok          392 ms
test typmod_insert                ... ok           70 ms
test union_op                     ... ok           19 ms

=======================
 7 of 35 tests failed. 
=======================

The differences that caused some tests to fail can be viewed in the
file "/home/gpadmin/postgresql-hll/regression.diffs".  A copy of the test summary that you see
above is saved in the file "/home/gpadmin/postgresql-hll/regression.out".

make: *** [/usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/pgxs.mk:446: installcheck] Error 1

PG Vector

06:55:09 gpadmin@build-rocky9-instance-0 pgvector ±| ✗|→ make installcheck REGRESS_OPTS='--init-file=/home/gpadmin/cloudberrydb/src/test/regress/init_file --inputdir=test --load-extension=vector --dbname=contrib_regression'
/usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/../../src/test/regress/scan_flaky_fault_injectors.sh && /usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/../../src/test/regress/pg_regress --inputdir=./ --bindir='/usr/local/cloudberry-db-1.6.0/bin'    --load-extension=gp_inject_fault --init-file=/home/gpadmin/cloudberrydb/src/test/regress/init_file --inputdir=test --load-extension=vector --dbname=contrib_regression bit btree cast copy halfvec hnsw_bit hnsw_halfvec hnsw_sparsevec hnsw_vector ivfflat_bit ivfflat_halfvec ivfflat_vector sparsevec vector_type
scanning for flaky fault-injection tests...
done
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression" ==============
DROP DATABASE
============== faultinjector enabled                  ==============
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== installing gp_inject_fault             ==============
CREATE EXTENSION
============== installing vector                      ==============
CREATE EXTENSION
============== installing gp_toolkit                  ==============
CREATE EXTENSION
============== checking external_fts status           ==============
External fts disabled. Using default answer files
============== checking optimizer status              ==============
Optimizer enabled. Using optimizer answer files whenever possible
============== checking gp_resource_manager status    ==============
Resource group disabled. Using default answer files
============== checking gp_internal_is_singlenode status ==============
Normal cluster detected. Replace '@hostname@' by hostname of contentid = 0
============== checking force_parallel_mode status    ==============
Force parallel mode disabled. Using default answer files
============== running regression test queries        ==============
test bit                          ... ok           32 ms
test btree                        ... ok          178 ms
test cast                         ... ok           49 ms
test copy                         ... ok          165 ms
test halfvec                      ... ok          112 ms
test hnsw_bit                     ... FAILED      128 ms
test hnsw_halfvec                 ... FAILED      231 ms
test hnsw_sparsevec               ... FAILED      341 ms
test hnsw_vector                  ... FAILED      306 ms
test ivfflat_bit                  ... FAILED      109 ms
test ivfflat_halfvec              ... FAILED      195 ms
test ivfflat_vector               ... FAILED      312 ms
test sparsevec                    ... ok           94 ms
test vector_type                  ... ok          113 ms

=======================
 7 of 14 tests failed. 
=======================

The differences that caused some tests to fail can be viewed in the
file "/home/gpadmin/pgvector/regression.diffs".  A copy of the test summary that you see
above is saved in the file "/home/gpadmin/pgvector/regression.out".

make: *** [/usr/local/cloudberry-db-1.6.0/lib/postgresql/pgxs/src/makefiles/pgxs.mk:446: installcheck] Error 1

I modified how the tests were executed to include an init_file. This helps filter Cloudberry/Greenplum specific known diffs.

[jianlirong]

Thanks to @edespino for bring up this important issue. It's certainly a challenging one.

Given that many extensions were initially developed for PostgreSQL, I've always wondered how the PostgreSQL community ensures compatibility between these extensions and different PostgreSQL versions. Based on my experience, it seems that the maintainers of each extension are primarily responsible for guaranteeing compatibility.

[tuhaihe]

Cool! This is valuable for the Cloudberry ecosystem's growth and future development.

Here are my inputs for the information:

• Should we create a draft extension list that includes only the extensions frequently used by Cloudberry/Greenplum users? I believe not every extension from upstream PostgreSQL is necessary for Cloudberry users. So, if we have a curated list, it could be more targeted towards helping us evaluate the test framework workload. For example, I spoke with some Greenplum users in the spatial industry, and they are using Q3C, so we can include it in the list. More cases like pgvector, PostgreML...

• Some extensions need to be customized for Cloudberry. How are we planning to maintain these changes? Are we considering contributing back to the upstream? If the changes are not accepted by the upstream, should we maintain them in separate repositories?

[tuhaihe]

Here is one blog for introducing the DuckDB community extensions: https://duckdb.org/2024/07/05/community-extensions.html.