Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix(ingest): datahub-api - move instantiation to the right config class #5878

Merged
merged 3 commits into from
Sep 9, 2022

Conversation

shirshanka
Copy link
Contributor

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@shirshanka shirshanka added the ingestion PR or Issue related to the ingestion of metadata label Sep 8, 2022
@github-actions github-actions bot removed the ingestion PR or Issue related to the ingestion of metadata label Sep 8, 2022
@github-actions
Copy link

github-actions bot commented Sep 8, 2022

Unit Test Results (metadata ingestion)

       8 files  ±0         8 suites  ±0   57m 1s ⏱️ +43s
   686 tests +3     682 ✔️ +2  3 💤 ±0  1 +1 
1 372 runs  +6  1 365 ✔️ +5  6 💤 ±0  1 +1 

For more details on these failures, see this check.

Results for commit 7ad0e02. ± Comparison against base commit 2115d5b.

♻️ This comment has been updated with latest results.

@shirshanka shirshanka requested a review from hsheth2 September 8, 2022 23:07
@shirshanka shirshanka added the ingestion PR or Issue related to the ingestion of metadata label Sep 8, 2022
@github-actions
Copy link

github-actions bot commented Sep 8, 2022

Unit Test Results (build & test)

511 tests   - 10   509 ✔️  - 12   14m 19s ⏱️ -40s
120 suites  -   2       0 💤 ±  0 
120 files    -   2       2 +  2 

For more details on these failures, see this check.

Results for commit 7ad0e02. ± Comparison against base commit 2115d5b.

This pull request removes 10 tests.
datahub.spark.TestCoalesceJobLineage ‑ testHiveInHiveOutCoalesce
datahub.spark.TestSparkJobsLineage ‑ test1HdfsInOut
datahub.spark.TestSparkJobsLineage ‑ test2HdfsInHiveOut
datahub.spark.TestSparkJobsLineage ‑ test3HdfsJdbcInJdbcOut
datahub.spark.TestSparkJobsLineage ‑ test4HiveInHiveOut
datahub.spark.TestSparkJobsLineage ‑ test5HdfsInJdbcOut
datahub.spark.TestSparkJobsLineage ‑ test6HdfsJdbcInJdbcOutTwoLevel
datahub.spark.TestSparkJobsLineage ‑ test7HdfsInPersistHdfsOut
datahub.spark.TestSparkJobsLineage ‑ test8PersistHdfsJdbcInJdbcOut
datahub.spark.TestSparkJobsLineage ‑ test9PersistJdbcInHdfsOut

♻️ This comment has been updated with latest results.

@hsheth2
Copy link
Collaborator

hsheth2 commented Sep 9, 2022

Looks like the smoke test error is real

WARNING  datahub.ingestion.run.pipeline:pipeline.py:277 Failed to configure reporter: datahub
Traceback (most recent call last):
  File "/home/runner/work/datahub/datahub/metadata-ingestion/src/datahub/ingestion/run/pipeline.py", line 266, in _configure_reporting
    ctx=self.ctx,
  File "/home/runner/work/datahub/datahub/metadata-ingestion/src/datahub/ingestion/reporting/datahub_ingestion_run_summary_provider.py", line 80, in create
    config_dict or {}
  File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 342, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for DatahubIngestionRunSummaryProviderConfig
datahub_api
  extra fields not permitted (type=value_error.extra)

@shirshanka shirshanka merged commit 056add1 into datahub-project:master Sep 9, 2022
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
ingestion PR or Issue related to the ingestion of metadata
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants