Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Add support for partitioning expressions #127

Merged
merged 7 commits into from
Oct 21, 2024
Merged

Add support for partitioning expressions #127

merged 7 commits into from
Oct 21, 2024

Conversation

george-zubrienko
Copy link
Contributor

No description provided.

@george-zubrienko george-zubrienko requested a review from a team as a code owner October 18, 2024 13:48
@github-actions GitHub Actions
Copy link

Coverage after merging partitioning-sql into main will be

80.42%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Contracts
   Logging.cs0%0%0%0%19–21, 21, 21–23, 26–27, 27, 27–30, 32–33
src/Providers/Hosting
   HostBuilderExtensions.cs84.75%58.33%100%90.48%190–192, 227–229
src/Services/Models
   PartitionsMetadataDefinition.cs0%100%0%0%14, 20, 26
   StreamMetadataDefinition.cs0%100%0%0%15
src/Sinks
   SchemaBoundSink.cs100%100%100%100%
   SchemaFreeSink.cs100%100%100%100%
src/Sinks/Extensions
   SchemaFreeSourceExtensions.cs100%100%100%100%
   StreamMetadataExtensions.cs100%100%100%100%
   StreamPartitionExtensions.cs0%100%0%0%17–24
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs80.51%52%100%83.12%132, 132, 132, 132, 134–136, 146–149, 152–155, 231–233, 233, 233, 233, 233, 233, 236–238, 240–241, 243–244, 246, 248, 259, 259–263
src/Sinks/Models
   StreamMetadata.cs100%100%100%100%
src/Sinks/Parquet
   ParquetOperations.cs74.68%66.67%87.50%77.45%100–102, 102, 102, 102, 102, 105–107, 159–161, 61–62, 62, 62–65, 65, 65–67, 69, 71–72, 89, 89, 89, 92–96, 96, 96, 98–99
   ParquetSink.cs83.88%64.10%100%86.49%126, 126, 126, 126, 128–130, 146–149, 174, 230–233, 270–272, 272, 272, 272, 272, 272, 275–277, 279–280, 282–283, 285, 287, 298, 298, 304–306, 49
src/Sinks/Parquet/Models
   DataCell.cs100%100%100%100%
src/Sinks/Services/StreamMetadata/V1
   PartitionsWriter.cs100%100%100%100%
src/Sources
   SchemaBoundSource.cs45.45%100%33.33%53.85%27, 31, 35, 40, 43, 52
   SchemaFreeSource.cs86.67%100%83.33%88.89%29
src/Sources/Base
   PollingSource.cs100%100%100%100%
src/Sources/BlobStorage
   BlobStorageSource.cs76%33.33%90.91%77.59%105–108, 112–113, 113, 113, 116–117, 119–120, 122, 84, 84, 86–87
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs87.96%77.78%95.45%89.52%167, 169, 198, 202–205, 210–211, 236–239, 266, 290–293, 317–320, 350, 364, 380, 380, 382, 384, 86–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs98.91%97.22%100%100%54
   JsonDocumentOperations.cs100%100%100%100%
   SimpleCdmAttributeExtensions.cs94.12%91.67%100%94.59%27–28, 41
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs55.24%16.67%84.62%63.24%108, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–150, 153–156, 91, 96
   SimpleCdmEntity.cs80%64.29%88.89%82.69%101–103, 106, 106, 112–114, 86–88, 96–98
src/Sources/Extensions
   GraphStageLogicExtensions.cs100%100%100%100%
   SchemaFreeSourceExtensions.cs100%100%100%100%
   SinkExtensions.cs100%100%100%100%
   SourceExtensions.cs100%100%100%100%
   SqlServerUtils.cs100%100%100%100%
src/Sources/RestApi
   RestApiSource.cs73.77%55.77%85%77.91%132, 136–142, 350, 355, 355, 355, 355, 355, 355, 357–360, 375–378, 402, 402–404, 404, 404, 407–408, 410–411, 414, 473–476, 484, 494–495, 497, 499, 503, 503, 503, 503, 503, 503, 503, 503, 503, 509, 513–514, 83–88
src/Sources/RestApi/Extensions
   RestApiExtensions.cs100%100%100%100%
   StringExtensions.cs0%0%0%0%21–26, 26, 26–29, 32–33
src/Sources/RestApi/Models
   PageResolverConfiguration.cs50%100%50%50%27, 33
   RestApiTemplatedField.cs100%100%100%100%
src/Sources/RestApi/Services
   RestApiTemplate.cs75.81%66.67%100%75%100–102, 62–64, 67–69, 73–77, 99
src/Sources/RestApi/Services/AuthenticatedMessageProviders
   DynamicBearerAuthenticatedMessageProvider.cs52.07%38.89%75%56.79%101–103, 108–110, 110, 110, 110, 110, 110, 110–115, 117, 137, 137, 137, 137, 137, 143, 143, 143–144, 147, 147–150, 77–92, 92, 92–95, 95, 95–96
   FixedHeaderAuthenticatedMessageProvider.cs100%100%100%100%
src/Sources/RestApi/Services/PageResolvers
   PageCountingResolver.cs80.49%75%100%81.48%37, 40–41, 43, 52, 59–61
   PageNextTokenResolver.cs65.52%50%100%66.67%30, 37–38, 41, 41, 41–45
   PageOffsetResolver.cs73.68%60%100%76.92%33, 47, 51, 51, 51–54, 57–58
src/Sources/RestApi/Services/PageResolvers/Base
   PageResolverBase.cs100%100%100%100%
src/Sources/RestApi/Services/UriProviders
   PagedUriProvider.cs67.16%50%83.33%72.83%102–104, 106, 113–116, 126, 126, 126, 132–133, 135, 149, 157, 157, 163, 163, 163–165, 168–171, 173–174, 50, 52, 67, 67, 67, 77, 77, 77, 84–86, 88, 95, 95, 95
   SimpleUriProvider.cs60%50%66.67%64.15%100, 102, 104, 34–35, 44, 57–59,

@github-actions GitHub Actions
Copy link

Coverage after merging partitioning-sql into main will be

80.08%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Contracts
   Logging.cs0%0%0%0%19–21, 21, 21–23, 26–27, 27, 27–30, 32–33
src/Providers/Hosting
   HostBuilderExtensions.cs84.75%58.33%100%90.48%190–192, 227–229
src/Services/Models
   DatePartitionMetadataDefinition.cs0%100%0%0%11, 14, 22
   PartitionMetadataDefinition.cs0%100%0%0%14, 20, 26
   StreamMetadataDefinition.cs0%100%0%0%16, 23
src/Sinks
   SchemaBoundSink.cs100%100%100%100%
   SchemaFreeSink.cs100%100%100%100%
src/Sinks/Extensions
   SchemaFreeSourceExtensions.cs100%100%100%100%
   StreamMetadataExtensions.cs100%100%100%100%
   StreamPartitionExtensions.cs0%0%0%0%17–18, 18, 18–26
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs80.51%52%100%83.12%132, 132, 132, 132, 134–136, 146–149, 152–155, 231–233, 233, 233, 233, 233, 233, 236–238, 240–241, 243–244, 246, 248, 259, 259–263
src/Sinks/Models
   StreamMetadata.cs100%100%100%100%
src/Sinks/Parquet
   ParquetOperations.cs74.68%66.67%87.50%77.45%100–102, 102, 102, 102, 102, 105–107, 159–161, 61–62, 62, 62–65, 65, 65–67, 69, 71–72, 89, 89, 89, 92–96, 96, 96, 98–99
   ParquetSink.cs83.88%64.10%100%86.49%126, 126, 126, 126, 128–130, 146–149, 174, 230–233, 270–272, 272, 272, 272, 272, 272, 275–277, 279–280, 282–283, 285, 287, 298, 298, 304–306, 49
src/Sinks/Parquet/Models
   DataCell.cs100%100%100%100%
src/Sinks/Services/StreamMetadata/V1
   PartitionsWriter.cs100%100%100%100%
src/Sources
   SchemaBoundSource.cs45.45%100%33.33%53.85%27, 31, 35, 40, 43, 52
   SchemaFreeSource.cs86.67%100%83.33%88.89%29
src/Sources/Base
   PollingSource.cs100%100%100%100%
src/Sources/BlobStorage
   BlobStorageSource.cs76%33.33%90.91%77.59%105–108, 112–113, 113, 113, 116–117, 119–120, 122, 84, 84, 86–87
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs87.96%77.78%95.45%89.52%167, 169, 198, 202–205, 210–211, 236–239, 266, 290–293, 317–320, 350, 364, 380, 380, 382, 384, 86–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs98.91%97.22%100%100%54
   JsonDocumentOperations.cs100%100%100%100%
   SimpleCdmAttributeExtensions.cs94.12%91.67%100%94.59%27–28, 41
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs55.24%16.67%84.62%63.24%108, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–150, 153–156, 91, 96
   SimpleCdmEntity.cs80%64.29%88.89%82.69%101–103, 106, 106, 112–114, 86–88, 96–98
src/Sources/Extensions
   GraphStageLogicExtensions.cs100%100%100%100%
   SchemaFreeSourceExtensions.cs100%100%100%100%
   SinkExtensions.cs100%100%100%100%
   SourceExtensions.cs100%100%100%100%
   SqlServerUtils.cs100%100%100%100%
src/Sources/RestApi
   RestApiSource.cs73.77%55.77%85%77.91%132, 136–142, 350, 355, 355, 355, 355, 355, 355, 357–360, 375–378, 402, 402–404, 404, 404, 407–408, 410–411, 414, 473–476, 484, 494–495, 497, 499, 503, 503, 503, 503, 503, 503, 503, 503, 503, 509, 513–514, 83–88
src/Sources/RestApi/Extensions
   RestApiExtensions.cs100%100%100%100%
   StringExtensions.cs0%0%0%0%21–26, 26, 26–29, 32–33
src/Sources/RestApi/Models
   PageResolverConfiguration.cs50%100%50%50%27, 33
   RestApiTemplatedField.cs100%100%100%100%
src/Sources/RestApi/Services
   RestApiTemplate.cs75.81%66.67%100%75%100–102, 62–64, 67–69, 73–77, 99
src/Sources/RestApi/Services/AuthenticatedMessageProviders
   DynamicBearerAuthenticatedMessageProvider.cs52.07%38.89%75%56.79%101–103, 108–110, 110, 110, 110, 110, 110, 110–115, 117, 137, 137, 137, 137, 137, 143, 143, 143–144, 147, 147–150, 77–92, 92, 92–95, 95, 95–96
   FixedHeaderAuthenticatedMessageProvider.cs100%100%100%100%
src/Sources/RestApi/Services/PageResolvers
   PageCountingResolver.cs80.49%75%100%81.48%37, 40–41, 43, 52, 59–61
   PageNextTokenResolver.cs65.52%50%100%66.67%30, 37–38, 41, 41, 41–45
   PageOffsetResolver.cs73.68%60%100%76.92%33, 47, 51, 51, 51–54, 57–58
src/Sources/RestApi/Services/PageResolvers/Base
   PageResolverBase.cs100%100%100%100%
src/Sources/RestApi/Services/UriProviders
   PagedUriProvider.cs67.16%50%83.33%72.83%102–104, 106, 113–116, 126, 126, 126, 132–133, 135, 149, 157, 157, 163, 163, 163–165, 168–171, 173–174, 50, 52, 67, 67, 67, 77, 77, 77, 84–86, 88, 95, 95, 95
   

@github-actions GitHub Actions
Copy link

Coverage after merging partitioning-sql into main will be

80.15%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Contracts
   Logging.cs0%0%0%0%19–21, 21, 21–23, 26–27, 27, 27–30, 32–33
src/Providers/Hosting
   HostBuilderExtensions.cs84.75%58.33%100%90.48%190–192, 227–229
src/Services/Models
   DatePartitionMetadataDefinition.cs0%100%0%0%16
   PartitionMetadataDefinition.cs0%100%0%0%14, 20, 26
   StreamMetadataDefinition.cs0%100%0%0%16, 23
src/Sinks
   SchemaBoundSink.cs100%100%100%100%
   SchemaFreeSink.cs100%100%100%100%
src/Sinks/Extensions
   SchemaFreeSourceExtensions.cs100%100%100%100%
   StreamMetadataExtensions.cs100%100%100%100%
   StreamPartitionExtensions.cs0%0%0%0%17–18, 18, 18–26
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs80.51%52%100%83.12%132, 132, 132, 132, 134–136, 146–149, 152–155, 231–233, 233, 233, 233, 233, 233, 236–238, 240–241, 243–244, 246, 248, 259, 259–263
src/Sinks/Models
   StreamMetadata.cs100%100%100%100%
src/Sinks/Parquet
   ParquetOperations.cs74.68%66.67%87.50%77.45%100–102, 102, 102, 102, 102, 105–107, 159–161, 61–62, 62, 62–65, 65, 65–67, 69, 71–72, 89, 89, 89, 92–96, 96, 96, 98–99
   ParquetSink.cs83.88%64.10%100%86.49%126, 126, 126, 126, 128–130, 146–149, 174, 230–233, 270–272, 272, 272, 272, 272, 272, 275–277, 279–280, 282–283, 285, 287, 298, 298, 304–306, 49
src/Sinks/Parquet/Models
   DataCell.cs100%100%100%100%
src/Sinks/Services/StreamMetadata/V1
   PartitionsWriter.cs100%100%100%100%
src/Sources
   SchemaBoundSource.cs45.45%100%33.33%53.85%27, 31, 35, 40, 43, 52
   SchemaFreeSource.cs86.67%100%83.33%88.89%29
src/Sources/Base
   PollingSource.cs100%100%100%100%
src/Sources/BlobStorage
   BlobStorageSource.cs76%33.33%90.91%77.59%105–108, 112–113, 113, 113, 116–117, 119–120, 122, 84, 84, 86–87
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs87.96%77.78%95.45%89.52%167, 169, 198, 202–205, 210–211, 236–239, 266, 290–293, 317–320, 350, 364, 380, 380, 382, 384, 86–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs98.91%97.22%100%100%54
   JsonDocumentOperations.cs100%100%100%100%
   SimpleCdmAttributeExtensions.cs94.12%91.67%100%94.59%27–28, 41
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs55.24%16.67%84.62%63.24%108, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–150, 153–156, 91, 96
   SimpleCdmEntity.cs80%64.29%88.89%82.69%101–103, 106, 106, 112–114, 86–88, 96–98
src/Sources/Extensions
   GraphStageLogicExtensions.cs100%100%100%100%
   SchemaFreeSourceExtensions.cs100%100%100%100%
   SinkExtensions.cs100%100%100%100%
   SourceExtensions.cs100%100%100%100%
   SqlServerUtils.cs100%100%100%100%
src/Sources/RestApi
   RestApiSource.cs73.77%55.77%85%77.91%132, 136–142, 350, 355, 355, 355, 355, 355, 355, 357–360, 375–378, 402, 402–404, 404, 404, 407–408, 410–411, 414, 473–476, 484, 494–495, 497, 499, 503, 503, 503, 503, 503, 503, 503, 503, 503, 509, 513–514, 83–88
src/Sources/RestApi/Extensions
   RestApiExtensions.cs100%100%100%100%
   StringExtensions.cs0%0%0%0%21–26, 26, 26–29, 32–33
src/Sources/RestApi/Models
   PageResolverConfiguration.cs50%100%50%50%27, 33
   RestApiTemplatedField.cs100%100%100%100%
src/Sources/RestApi/Services
   RestApiTemplate.cs75.81%66.67%100%75%100–102, 62–64, 67–69, 73–77, 99
src/Sources/RestApi/Services/AuthenticatedMessageProviders
   DynamicBearerAuthenticatedMessageProvider.cs52.07%38.89%75%56.79%101–103, 108–110, 110, 110, 110, 110, 110, 110–115, 117, 137, 137, 137, 137, 137, 143, 143, 143–144, 147, 147–150, 77–92, 92, 92–95, 95, 95–96
   FixedHeaderAuthenticatedMessageProvider.cs100%100%100%100%
src/Sources/RestApi/Services/PageResolvers
   PageCountingResolver.cs80.49%75%100%81.48%37, 40–41, 43, 52, 59–61
   PageNextTokenResolver.cs65.52%50%100%66.67%30, 37–38, 41, 41, 41–45
   PageOffsetResolver.cs73.68%60%100%76.92%33, 47, 51, 51, 51–54, 57–58
src/Sources/RestApi/Services/PageResolvers/Base
   PageResolverBase.cs100%100%100%100%
src/Sources/RestApi/Services/UriProviders
   PagedUriProvider.cs67.16%50%83.33%72.83%102–104, 106, 113–116, 126, 126, 126, 132–133, 135, 149, 157, 157, 163, 163, 163–165, 168–171, 173–174, 50, 52, 67, 67, 67, 77, 77, 77, 84–86, 88, 95, 95, 95
   SimpleUriProvider.cs60%50%66.67%64.15%100,

@george-zubrienko george-zubrienko merged commit 4a20ebc into main Oct 21, 2024
1 check passed
@george-zubrienko george-zubrienko deleted the partitioning-sql branch October 21, 2024 09:18
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@s-vitaliy s-vitaliy s-vitaliy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@george-zubrienko @s-vitaliy