Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Bug] Dynamic Partitioning Generates Incorrect Partition Values with Extra Day Component #47962

Open
3 tasks done
Baymine opened this issue Feb 17, 2025 · 1 comment
Open
3 tasks done

Comments

@Baymine
Copy link
Contributor

Baymine commented Feb 17, 2025

Search before asking

  • I had searched in the issues and found no similar issues.

Version

doris-2.1.6

What's Wrong?

When creating a table with dynamic partitioning in Doris, the generated partitions include an extra day component (e.g., '20250201' instead of '202502'). This causes insertions with the expected year_month format to fail due to missing partitions.

What You Expected?

The partitions should be created with the year_month values as integers representing the year and month (e.g., '202502'), without the extra day component.

How to Reproduce?

  1. Create a table with dynamic partitioning set to partition by month.
CREATE TABLE `test_tbl` (
  `year_month` int(11) NOT NULL,
  `unique_product_id` varchar NOT NULL 
) ENGINE=OLAP
UNIQUE KEY(`year_month`,`unique_product_id`)
PARTITION BY RANGE(`year_month`)
(
 PARTITION p_initial VALUES LESS THAN (202201)
)
DISTRIBUTED BY HASH(`unique_product_id`) BUCKETS 16
PROPERTIES (
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "month",
"dynamic_partition.start" = "-25",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "16",
"replication_num" = "3",
"in_memory" = "false",
"storage_format" = "V2"
);

2.Execute SHOW CREATE TABLE to view the partition definitions.

CREATE TABLE `test_tbl` (
  `year_month` int NOT NULL,
  `unique_product_id` varchar(65533) NOT NULL
) ENGINE=OLAP
UNIQUE KEY(`year_month`, `unique_product_id`)
PARTITION BY RANGE(`year_month`)
(PARTITION p202502 VALUES [("20250201"), ("20250301")),
PARTITION p202503 VALUES [("20250301"), ("20250401")),
PARTITION p202504 VALUES [("20250401"), ("20250501")))
DISTRIBUTED BY HASH(`unique_product_id`) BUCKETS 16
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "month",
"dynamic_partition.time_zone" = "Asia/Shanghai",
"dynamic_partition.start" = "-25",
"dynamic_partition.end" = "2",
"dynamic_partition.prefix" = "p",
"dynamic_partition.replication_allocation" = "tag.location.default: 3",
"dynamic_partition.buckets" = "16",
"dynamic_partition.create_history_partition" = "false",
"dynamic_partition.history_partition_num" = "-1",
"dynamic_partition.hot_partition_num" = "0",
"dynamic_partition.reserved_history_periods" = "NULL",
"dynamic_partition.storage_policy" = "",
"dynamic_partition.start_day_of_month" = "1",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"enable_unique_key_merge_on_write" = "true",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728",
"enable_mow_light_delete" = "false"
);
  1. Attempt to insert data with a year_month value expected to fit into a created partition.
insert into test_tbl (year_month, unique_product_id) values ('202503', '1232');
  1. Observe the error indicating no partition exists for the tuple.
Reason: no partition for this tuple.

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@w517424787
Copy link

应该是没控制好range字段类型,文档上说只允许用date/datetime类型创建,你这int类型都创建成功了,这里肯定是有bug

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants