Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Detect and copy files in a shared windows folder from under the airflow docker container. #659

Open
coolvs4 opened this issue Jan 14, 2024 · 0 comments

Comments

@coolvs4
Copy link

coolvs4 commented Jan 14, 2024

I need my DAG to check for a file in a shared folder in the Windows operating system.
A suitable solution was to use the FileSensor tool.
I use this tool as follows:

wait_file = FileSensor(
        task_id="wait_file",
        filepath="//192.168.0.100/shared/log.csv",
        fs_conn_id='my_smb_connect',
        poke_interval = 3,
        timeout = 15
    )

There are no problems when using folders in the local file system.
I was setting up a connection via the web interface Admin -> Connections.
I set Connection Type to File (path) and everything worked.

However, to access remote resources in Windows, you must use the SMB protocol. I have installed additional packages in the airflow image apache-airflow-providers-samba because it is not available by default.

My connection settings are following:

Connection Id: my_smb_connect
Connection Type: Samba
Host: 192.168.0.100
Schema: 
Login: user
Password: ****
Port: 445

But this does not give any result. The file is still not visible.
Please help me, is it the wrong connection setup?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant