-
Notifications
You must be signed in to change notification settings - Fork 71
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
fix: Preserve __alias__
when mapping streams with repeated schema messages
#1524
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1524 +/- ##
=======================================
Coverage 85.50% 85.51%
=======================================
Files 57 57
Lines 4692 4693 +1
Branches 802 802
=======================================
+ Hits 4012 4013 +1
Misses 489 489
Partials 191 191
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @DanilJr!
Thanks for the PR. Just one question. Otherwise this looks good 🙂
__alias__
is when mapping streams with repeated schema messages
I would like to have automated tests but at least manual testing confirms that the patch works as intended: 1. Singer input{"type": "SCHEMA", "stream": "example", "schema": {"properties": {"id": {"type": "integer"}}}}
{"type": "RECORD", "stream": "example", "record": {"id": 1}}
{"type": "SCHEMA", "stream": "example", "schema": {"properties": {"id": {"type": "integer"}}}}
{"type": "RECORD", "stream": "example", "record": {"id": 2}} 2. Stream config{
"stream_maps": {
"example": {
"__alias__": "example_v2"
}
}
} 3. Mapper output{"type": "SCHEMA", "stream": "example_v2", "schema": {"properties": {"id": {"type": "integer"}}}, "key_properties": [], "bookmark_properties": []}
{"type": "RECORD", "stream": "example_v2", "record": {"id": 1}, "time_extracted": "2023-03-24T19:41:37.953424+00:00"}
{"type": "SCHEMA", "stream": "example_v2", "schema": {"properties": {"id": {"type": "integer"}}}, "key_properties": [], "bookmark_properties": []}
{"type": "RECORD", "stream": "example_v2", "record": {"id": 2}, "time_extracted": "2023-03-24T19:41:37.953966+00:00"} I can also confirm that using a explicit shallow copy ( |
__alias__
is when mapping streams with repeated schema messages__alias__
when mapping streams with repeated schema messages
It relates to this issue
Closes #1521
📚 Documentation preview 📚: https://meltano-sdk--1524.org.readthedocs.build/en/1524/