Skip to content
This repository has been archived by the owner on Jun 19, 2023. It is now read-only.

Adding Connection Tests #64

Merged
merged 2 commits into from
Dec 9, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,14 @@ pip install airflow-provider-fivetran

In the Airflow user interface, configure a Connection for Fivetran. Most of the Connection config fields will be left blank. Configure the following fields:

* `Conn Id`: `fivetran`
* `Conn Id`: `fivetran_default`
* `Conn Type`: `Fivetran`
* `Login`: Fivetran API Key
* `Password`: Fivetran API Secret
* `Fivetran API Key`: Your Fivetran API Key
* `Fivetran API Secret`: Your Fivetran API Secret

Find the Fivetran API Key and Secret in the [Fivetran Account Settings](https://fivetran.com/account/settings), under the **API Config** section. See our documentation for more information on [Fivetran API Authentication](https://fivetran.com/docs/rest-api/getting-started#authentication).

The sensor and operator assume the `Conn Id` is set to `fivetran`, however if you are managing multipe Fivetran accounts, you can set this to anything you like. See the DAG in examples to see how to specify a custom `Conn Id`.
The sensor and operator assume the `Conn Id` is set to `fivetran_default`, however if you are managing multipe Fivetran accounts, you can set this to anything you like. See the DAG in examples to see how to specify a custom `Conn Id`.

## Modules

Expand Down
19 changes: 16 additions & 3 deletions fivetran_provider/hooks/fivetran.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ class FivetranHook(BaseHook):
default_conn_name = "fivetran_default"
conn_type = "fivetran"
hook_name = "Fivetran"
api_user_agent = "airflow_provider_fivetran/1.1.2"
api_user_agent = "airflow_provider_fivetran/1.1.3"
api_protocol = "https"
api_host = "api.fivetran.com"
api_path_connectors = "v1/connectors/"
Expand Down Expand Up @@ -234,8 +234,8 @@ def start_fivetran_sync(self, connector_id):
succeeded_at = connector_details["succeeded_at"]
failed_at = connector_details["failed_at"]
endpoint = self.api_path_connectors + connector_id
if self._do_api_call(("GET", endpoint))['data']['paused'] == True:
self._do_api_call(("PATCH", endpoint),json.dumps({"paused": False}))
if self._do_api_call(("GET", endpoint))["data"]["paused"] == True:
self._do_api_call(("PATCH", endpoint), json.dumps({"paused": False}))
if succeeded_at == None and failed_at == None:
succeeded_at = str(pendulum.now())
self._do_api_call(("POST", endpoint + "/force"))
Expand Down Expand Up @@ -324,6 +324,19 @@ def _parse_timestamp(self, api_time):
else pendulum.from_timestamp(-1)
)

def test_connection(self):
"""
Ensures Airflow can reach Fivetran API
"""
try:
resp = self._do_api_call(("GET", "v1/users"))
if resp["code"] == "Success":
return True, "Fivetran connection test passed"
else:
return False, resp
except Exception as e:
return False, str(e)


def _retryable_error(exception) -> bool:
return (
Expand Down
2 changes: 1 addition & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[metadata]
name = airflow-provider-fivetran
version = 1.1.2
version = 1.1.3
description = A Fivetran provider for Apache Airflow
long_description = file: README.md
long_description_content_type = text/markdown
Expand Down