Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

bug: Stream map __alias__ is lost after first schema for a stream is processed #1521

Closed
1 task
edgarrmondragon opened this issue Mar 22, 2023 · 0 comments · Fixed by #1524
Closed
1 task
Assignees
Labels
kind/Bug Something isn't working valuestream/SDK

Comments

@edgarrmondragon
Copy link
Collaborator

Singer SDK Version

0.22.0

Is this a regression?

  • Yes

Python Version

NA

Bug scope

Mapping (stream maps, flattening, etc.)

Operating System

NA

Description

Reported by a user in Slack:

Within the current Singer SDK version, the PluginMapper class in mapper.py has this function register_raw_stream_schema where the code is 'popping' values from stream definition. I think the intention was the for loop here would copy the stream definition (stream_def), but that is not the case.

sdk/singer_sdk/mapper.py

Lines 668 to 670 in aee57cc

for stream_map_key, stream_def in self.stream_maps_dict.items():
stream_alias: str = stream_map_key
source_stream: str = stream_map_key

Within the for loop, the following is later executed, and it is altering the stream definition:

sdk/singer_sdk/mapper.py

Lines 699 to 707 in aee57cc

if MAPPER_SOURCE_OPTION in stream_def:
source_stream = stream_def.pop(MAPPER_SOURCE_OPTION)
if source_stream != stream_name:
# Not a match
continue
if MAPPER_ALIAS_OPTION in stream_def:
stream_alias = stream_def.pop(MAPPER_ALIAS_OPTION)

When I altered the for loop locally to make a copy of the stream definition, the alias configuration was maintained and we got the expected outcome.

This can be fixed by copying the stream_def dict.

Code

No response

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
kind/Bug Something isn't working valuestream/SDK
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant