Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add interruption token to ParquetSink #139

Merged
merged 1 commit into from
Oct 25, 2024
Merged

Conversation

s-vitaliy
Copy link
Contributor

@s-vitaliy s-vitaliy commented Oct 25, 2024

Part of #128

Implemented:

  • The ParquetSink now has InterruptionToken as a parameter. It will not save completion token if the stream was interrupted.

Checklist

  • GitHub issue exists for this change.
  • Unit tests added and they pass.
  • Line Coverage is at least 80%.
  • Review requested on latest commit.

Part of #128

Implemented:
- The ParquetSink now has InterruptionToken as a parameter. It will not save completion token if the stream was interrupted.
@s-vitaliy s-vitaliy requested a review from a team as a code owner October 25, 2024 10:14
Copy link

Coverage after merging add-interruption-token into main will be

80.90%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
src/Contracts
   Logging.cs0%0%0%0%19–21, 21, 21–23, 26–27, 27, 27–30, 32–33
src/Providers/Hosting
   HostBuilderExtensions.cs86.57%68.75%100%91.30%195–197, 232–234
src/Services/Models
   DatePartitionMetadataDefinition.cs0%100%0%0%16
   PartitionMetadataDefinition.cs0%100%0%0%14, 20, 26
   StreamMetadataDefinition.cs0%100%0%0%16, 23
src/Sinks
   SchemaBoundSink.cs100%100%100%100%
   SchemaFreeSink.cs100%100%100%100%
src/Sinks/Extensions
   SchemaFreeSourceExtensions.cs100%100%100%100%
   StreamMetadataExtensions.cs100%100%100%100%
   StreamPartitionExtensions.cs0%0%0%0%17–18, 18, 18–26
src/Sinks/Json
   JsonSink.cs74.66%42.86%100%77.68%102–105, 149–151, 151, 151, 151, 151, 151, 154–156, 158–159, 161–162, 164, 166, 177, 177–180, 82, 82, 82, 82, 84–86, 96–99
   MultilineJsonSink.cs81.60%58.62%100%83.73%134, 134, 136–137, 148–151, 154–157, 236–238, 238, 238, 238, 238, 238, 241–243, 245–246, 248–249, 251, 253, 264, 264–268, 291, 291, 300, 302
src/Sinks/Models
   StreamMetadata.cs100%100%100%100%
src/Sinks/Parquet
   ParquetOperations.cs74.68%66.67%87.50%77.45%100–102, 102, 102, 102, 102, 105–107, 159–161, 61–62, 62, 62–65, 65, 65–67, 69, 71–72, 89, 89, 89, 92–96, 96, 96, 98–99
   ParquetSink.cs89.86%71.11%100%92.92%151, 151, 153–154, 201, 265–268, 307, 307, 307, 307, 314–315, 317–318, 320, 333, 333, 339–341, 360, 360, 369, 371, 63
src/Sinks/Parquet/Models
   DataCell.cs100%100%100%100%
src/Sinks/Services/StreamMetadata/V1
   PartitionsWriter.cs100%100%100%100%
src/Sources
   SchemaBoundSource.cs45.45%100%33.33%53.85%27, 31, 35, 40, 43, 52
   SchemaFreeSource.cs86.67%100%83.33%88.89%29
src/Sources/Base
   PollingSource.cs100%100%100%100%
src/Sources/BlobStorage
   BlobStorageSource.cs76%33.33%90.91%77.59%105–108, 112–113, 113, 113, 116–117, 119–120, 122, 84, 84, 86–87
src/Sources/CdmChangeFeedSource
   CdmChangeFeedSource.cs87.96%77.78%95.45%89.52%167, 169, 198, 202–205, 210–211, 236–239, 266, 290–293, 317–320, 350, 364, 380, 380, 382, 384, 86–92
src/Sources/CdmChangeFeedSource/Extensions
   CsvOperations.cs98.91%97.22%100%100%54
   JsonDocumentOperations.cs100%100%100%100%
   SimpleCdmAttributeExtensions.cs94.12%91.67%100%94.59%27–28, 41
src/Sources/CdmChangeFeedSource/Models
   SimpleCdmAttribute.cs55.24%16.67%84.62%63.24%108, 123–124, 124, 124–126, 129, 129, 129–131, 134, 134, 134–136, 139, 139, 139–141, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144, 144–150, 153–156, 91, 96
   SimpleCdmEntity.cs80%64.29%88.89%82.69%101–103, 106, 106, 112–114, 86–88, 96–98
src/Sources/Extensions
   GraphStageLogicExtensions.cs100%100%100%100%
   SchemaFreeSourceExtensions.cs100%100%100%100%
   SinkExtensions.cs100%100%100%100%
   SourceExtensions.cs100%100%100%100%
   SqlServerUtils.cs100%100%100%100%
src/Sources/RestApi
   RestApiSource.cs73.77%55.77%85%77.91%132, 136–142, 350, 355, 355, 355, 355, 355, 355, 357–360, 375–378, 402, 402–404, 404, 404, 407–408, 410–411, 414, 473–476, 484, 494–495, 497, 499, 503, 503, 503, 503, 503, 503, 503, 503, 503, 509, 513–514, 83–88
src/Sources/RestApi/Extensions
   RestApiExtensions.cs100%100%100%100%
   StringExtensions.cs0%0%0%0%21–26, 26, 26–29, 32–33
src/Sources/RestApi/Models
   PageResolverConfiguration.cs50%100%50%50%27, 33
   RestApiTemplatedField.cs100%100%100%100%
src/Sources/RestApi/Services
   RestApiTemplate.cs75.81%66.67%100%75%100–102, 62–64, 67–69, 73–77, 99
src/Sources/RestApi/Services/AuthenticatedMessageProviders
   DynamicBearerAuthenticatedMessageProvider.cs52.07%38.89%75%56.79%101–103, 108–110, 110, 110, 110, 110, 110, 110–115, 117, 137, 137, 137, 137, 137, 143, 143, 143–144, 147, 147–150, 77–92, 92, 92–95, 95, 95–96
   FixedHeaderAuthenticatedMessageProvider.cs100%100%100%100%
src/Sources/RestApi/Services/PageResolvers
   PageCountingResolver.cs80.49%75%100%81.48%37, 40–41, 43, 52, 59–61
   PageNextTokenResolver.cs65.52%50%100%66.67%30, 37–38, 41, 41, 41–45
   PageOffsetResolver.cs73.68%60%100%76.92%33, 47, 51, 51, 51–54, 57–58
src/Sources/RestApi/Services/PageResolvers/Base
   PageResolverBase.cs100%100%100%100%
src/Sources/RestApi/Services/UriProviders
   PagedUriProvider.cs67.16%50%83.33%72.83%102–104, 106, 113–116, 126, 126, 126, 132–133, 135, 149, 157, 157, 163, 163, 163–165, 168–171, 173–174, 50, 52, 67, 67, 67, 77, 77, 77, 84–86, 88, 95, 95, 95
   SimpleUriProvider.cs60%50%66.67%64.15%100, 102,

@s-vitaliy s-vitaliy merged commit 8ffdc74 into main Oct 25, 2024
1 check passed
@s-vitaliy s-vitaliy deleted the add-interruption-token branch October 25, 2024 10:25
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants