Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Date32 wrong proto representation #319

Closed
jkaflik opened this issue Aug 17, 2023 · 0 comments · Fixed by #320
Closed

Date32 wrong proto representation #319

jkaflik opened this issue Aug 17, 2023 · 0 comments · Fixed by #320
Assignees
Labels
bug Something isn't working

Comments

@jkaflik
Copy link
Contributor

jkaflik commented Aug 17, 2023

Describe the bug

Steps to reproduce

  1. Append any value to proto.ColDate32
  2. Send data to ClickHouse
  3. The date32Epoch constant shifts values. In most cases, it causes an overflow with Date32 represent as uint32

const date32Epoch = -1420070400

Expected behaviour

Date32 value should be represented as int32 and not be shifted. It represents the number of days since the start of Unix time: https://clickhouse.com/docs/en/sql-reference/data-types/date32

Code example

package main

import (
	"context"
	"io"
	"time"

	"github.com/ClickHouse/ch-go"
	"github.com/ClickHouse/ch-go/proto"
)

func main() {
	ctx := context.Background()

	conn, err := ch.Dial(ctx, ch.Options{})
	if err != nil {
		panic(err)
	}

	if err := conn.Do(ctx, ch.Query{
		Body: `CREATE TABLE IF NOT EXISTS test_table_insert
(
    d32			      Date32,
) ENGINE = Memory`,
	}); err != nil {
		panic(err)
	}

	// Define all columns of table.
	var (
		d32       proto.ColDate32
	)

	// Append 10 rows to initial data block.
	for i := 0; i < 10; i++ {
		d32.Append(time.Now())
	}

	// Insert single data block.
	input := proto.Input{
		{Name: "d32", Data: &d32},
	}
	if err := conn.Do(ctx, ch.Query{
		Body: "INSERT INTO test_table_insert VALUES",
		// Or "INSERT INTO test_table_insert (ts, severity_text, severity_number, body, name, arr) VALUES"
		Input: input,
	}); err != nil {
		panic(err)
	}

	// Stream data to ClickHouse server in multiple data blocks.
	var blocks int
	if err := conn.Do(ctx, ch.Query{
		Body:  input.Into("test_table_insert"), // helper that generates INSERT INTO query with all columns
		Input: input,

		// OnInput is called to prepare Input data before encoding and sending
		// to ClickHouse server.
		OnInput: func(ctx context.Context) error {
			// On OnInput call, you should fill the input data.
			//
			// NB: You should reset the input columns, they are
			// not reset automatically.
			//
			// That is, we are re-using the same input columns and
			// if we will return nil without doing anything, data will be
			// just duplicated.

			input.Reset() // calls "Reset" on each column

			if blocks >= 10 {
				// Stop streaming.
				//
				// This will also write tailing input data if any,
				// but we just reset the input, so it is currently blank.
				return io.EOF
			}

			// Append new values:
			for i := 0; i < 10; i++ {
				d32.Append(time.Now())
			}

			// Data will be encoded and sent to ClickHouse server after returning nil.
			// The Do method will return error if any.
			blocks++
			return nil
		},
	}); err != nil {
		panic(err)
	}
}

Configuration

Environment

  • Client version: HEAD

ClickHouse server

  • ClickHouse Server version: latest
@jkaflik jkaflik added the bug Something isn't working label Aug 17, 2023
@jkaflik jkaflik self-assigned this Aug 17, 2023
@jkaflik jkaflik changed the title Date32 overflow Date32 wrong proto representation Aug 17, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant