Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to pull a file file from a NFS (network file system,on-premise machine with disk mounted) as source into Data fusion #9

Open
stevenzhang-support opened this issue Jun 24, 2021 · 0 comments

Comments

@stevenzhang-support
Copy link

Issue type: Feature Request

Problem you have encountered:
I ran into an issue when I wanted to have a feature to be supported to fetch a file from a NFS (network file system,on-premise machine with disk mounted) as a source.

What you expected to happen:

  1. A data fusion plugin or this ftp-plugin to perform file fetching with NFS source can be set and
  2. A file polling can be possible when new files appear from the source and will be picked up automatically or a scheduler can be set together with ftp fetching from NFS source.

Other information (workarounds you have tried, documentation consulted, etc):
I tried to share a solution by considering [1] and combining airflow ftp operators under schedule to fetch a file from a NFS(network file system) source using FTP protocol. It seems there is no existing file or FTP plugin that can do the file fetching using a scheduler.

[1] https://stackoverflow.com/questions/48248369/how-to-implement-polling-in-airflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant