Schema registry support for Protobuf

[kleewho]

I'd like to add schema registry support for Protobuf. I'd love to put some work myself but I need some guidance on how the configuration should look like. Schema registry will require similar configuration as is already present for Avro, but I don't think it should be taken from there. Should it be mirrored on Protobuf side or maybe it should be shared config for both Avro and Protobuf?

[kleewho]

In context of just deserialization I would propose:

• keep for now the old config for Avro's schema registry. If someone uses it everything works as previously
• introduce "new" config schemaRegistry that would be almost a copy of what's for Avro:

type SchemaRegistryConfig struct {
	Url            string
	RequestTimeout time.Duration
	TLS            TLSConfig
	Username       string
	Password       string
}

If someone uses new config it would create a new deserializer.

[d-rk]

Hey @kleewho ,

in general I think we should have a shared config for schema registries as it will be easier to understand the config file if the "schemaRegistry Block" exists only in one place.

I would expected the config to be split into a general schemaRegistry part and more specific config part for avro and protobuf (if needed). Something like this:

    # optional: schema registry
    schemaRegistry:
      url: localhost:8081
      # optional: timeout for requests (defaults to 5s)
      requestTimeout: 10s

      # optional: basic auth credentials
      username: admin
      password: admin

      # optional: tls config for avro
      tls:
        enabled: true
        ca: my-ca
        cert: my-cert
        certKey: my-key
        # set insecure to true to ignore all tls verification (defaults to false)
        insecure: false

    # optional: avro settings
    avro:
      # optional: configure codec for (de)serialization as standard,avro (defaults to standard)
      # see: https://github.com/deviceinsight/kafkactl/issues/123
      jsonCodec: avro

Additionally, I would ignore backwards compatibility in favor of a simpler code structure and less maintenance.
Mentioning the breaking change in the changelog as well as a bump in the semver version should be sufficient.

[kleewho]

@d-rk Sure! I have some code already in place. I'll adjust it to what you wrote and hopefully be back with a PR in next few weeks