Skip to content
This repository has been archived by the owner on Jun 19, 2023. It is now read-only.

Commit

Permalink
Merge pull request #64 from PubChimps/main
Browse files Browse the repository at this point in the history
Adding Connection Tests
  • Loading branch information
PubChimps authored Dec 9, 2022
2 parents 1e2b99a + 2d7981f commit 20ffd5a
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 8 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ pip install airflow-provider-fivetran

In the Airflow user interface, configure a Connection for Fivetran. Most of the Connection config fields will be left blank. Configure the following fields:

* `Conn Id`: `fivetran`
* `Conn Id`: `fivetran_default`
* `Conn Type`: `Fivetran`
* `Login`: Fivetran API Key
* `Password`: Fivetran API Secret
* `Fivetran API Key`: Your Fivetran API Key
* `Fivetran API Secret`: Your Fivetran API Secret

Find the Fivetran API Key and Secret in the [Fivetran Account Settings](https://fivetran.com/account/settings), under the **API Config** section. See our documentation for more information on [Fivetran API Authentication](https://fivetran.com/docs/rest-api/getting-started#authentication).

The sensor and operator assume the `Conn Id` is set to `fivetran`, however if you are managing multipe Fivetran accounts, you can set this to anything you like. See the DAG in examples to see how to specify a custom `Conn Id`.
The sensor and operator assume the `Conn Id` is set to `fivetran_default`, however if you are managing multipe Fivetran accounts, you can set this to anything you like. See the DAG in examples to see how to specify a custom `Conn Id`.

## Modules

Expand Down
19 changes: 16 additions & 3 deletions fivetran_provider/hooks/fivetran.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ class FivetranHook(BaseHook):
default_conn_name = "fivetran_default"
conn_type = "fivetran"
hook_name = "Fivetran"
api_user_agent = "airflow_provider_fivetran/1.1.2"
api_user_agent = "airflow_provider_fivetran/1.1.3"
api_protocol = "https"
api_host = "api.fivetran.com"
api_path_connectors = "v1/connectors/"
Expand Down Expand Up @@ -234,8 +234,8 @@ def start_fivetran_sync(self, connector_id):
succeeded_at = connector_details["succeeded_at"]
failed_at = connector_details["failed_at"]
endpoint = self.api_path_connectors + connector_id
if self._do_api_call(("GET", endpoint))['data']['paused'] == True:
self._do_api_call(("PATCH", endpoint),json.dumps({"paused": False}))
if self._do_api_call(("GET", endpoint))["data"]["paused"] == True:
self._do_api_call(("PATCH", endpoint), json.dumps({"paused": False}))
if succeeded_at == None and failed_at == None:
succeeded_at = str(pendulum.now())
self._do_api_call(("POST", endpoint + "/force"))
Expand Down Expand Up @@ -324,6 +324,19 @@ def _parse_timestamp(self, api_time):
else pendulum.from_timestamp(-1)
)

def test_connection(self):
"""
Ensures Airflow can reach Fivetran API
"""
try:
resp = self._do_api_call(("GET", "v1/users"))
if resp["code"] == "Success":
return True, "Fivetran connection test passed"
else:
return False, resp
except Exception as e:
return False, str(e)


def _retryable_error(exception) -> bool:
return (
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[metadata]
name = airflow-provider-fivetran
version = 1.1.2
version = 1.1.3
description = A Fivetran provider for Apache Airflow
long_description = file: README.md
long_description_content_type = text/markdown
Expand Down

0 comments on commit 20ffd5a

Please # to comment.