-
Notifications
You must be signed in to change notification settings - Fork 41.2k
Add Kafka AutoConfiguration #5516
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Conversation
@dsyer could you unlink spring-boot from waffle.io, the |
|
||
@EnableKafka | ||
@ConditionalOnMissingBean(name = KafkaListenerConfigUtils.KAFKA_LISTENER_ANNOTATION_PROCESSOR_BEAN_NAME) | ||
protected static class EnableRabbitConfiguration { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rabbit
:-)
The diff for |
Thanks - that was the feedback I was looking for - I didn't want to write a huge test case then throw it away. However, I am not sure how we will be able to determine what the subset should be. I guess we can just take a stab and refine it with user feedback. |
Perhaps we can offer an escape hatch in the form of a customiser interface that allows users to further configure the auto-configured stuff for cases where the properties don't cover their needs. |
Actually, they designate their props with an importance HIGH, MEDIUM and LOW - I figure I'll add just the HIGHs, one or two others and add a customizer for the rest - will rework. |
c25aa62
to
70057d4
Compare
Reworked as discussed; squashed. Still needs some more tests and docs but I want to see if this is closer to the mark. |
Looks much better now, do you have any idea how easy it will be to write tests for this? |
@philwebb If you mean integration tests with an embedded broker, theoretically it's possible - spring-kafka-test provides an embedded broker. However, when I added it to the CP, I got hundreds of test failures in other autoconfigs - perhaps because of some transitive deps pulled in by kafka itself, but it wasn't obvious to me what the problem was. If you mean mocks to verify the autoconfig is set up correctly, that should be pretty easy. If M2 was still tomorrow, I probably couldn't do it, but if the release calendar is correct I should be able to have tests in a day or so. |
I'm interested in what those failures were. It might suggest that there's an incompatibility that we need to deal with.
M2's scheduled for next week (6th April) now as we're waiting for Spring Framework 4.3.0.RC1 |
@wilkinsona I figured it out - it was our old friend slf4j; excluding that from the test dependency solved it. Produced some weird failures, though; which is what threw me and made me think it was something worse...
|
@garyrussell Thanks. We have some tests that capture System.out and System.err to make sure that the right things got logged. I would guess that the extra SLF4J dependencies ( |
70057d4
to
8775499
Compare
Pushed tests for remaining properties and an integration test. Removed 'REVIEW ONLY' from PR title. |
@garyrussell Sorry, I have some bad news… Every field in |
@wilkinsona It won't be a big deal to copy it from Kafka if/when I get a green light. |
@garyrussell Don't forget that we don't support Javadoc tags ( |
efa423a
to
ec65e32
Compare
Rebased, pushed property docs. |
Change Common Props to Hyphenated (asciidoc) Only Support HIGH and a Few Other Props
Also change filename properties to Resources.
ec65e32
to
2bc6919
Compare
@garyrussell What's the release plan for spring-kafka? When is GA due? |
@philwebb Early May - the RC will certainly be before boot's RC1. |
@philwebb Let me know if there is some remaining reticence about merging this - we are about to put out the spring-kafka release candidate - if the auto config doesn't go into boot, we can move it to the project itself. TBH, it's not really clear to me how such decisions are made. Cheers. |
I am going to close this - we just hit a blocker in spring-kafka and might not make an end of month GA. |
It needs more tests and docs.
But, before I do that, I need to get the approach validated/approved (there are a huge number of properties; I generated most of that code from the internal Kafka properties code).
I also need some guidance with regards to a starter - should that be part of this PR or a separate PR?
The auto-config itself is pretty straightforward; just a few beans to support a listener container factory and template.
Tested with a stand-alone boot app:
Needs a running Kafka with a topic
foo
andspring.kafka.consumer.group-id=foo
in app.props.