SFTP Data Source
SFTP channel allows user to read data from network file system.
On the HDFS channel, you will be able to read data with formats including JSON, CSV, TEXT, XML, Fixed Length, and Binary.
Configuring SFTP Data Source
| Field | Description |
|---|---|
| Connection Name | Select the SFTP connection name. |
| Override Credentials | Check the override credentials for user specific actions. Provide username and password. |
| File Path | Mention the file path of SFTP file system. Note: In case of AWS, while providing directory the user is required to provide the wildcard character ( * ) along with the directory. For eg: /home/centos/foldername/* |
| Incremental Read | Check mark to read latest file in case of folder. |
| Parallelism | Number of parallel threads to launch in order to run to download file from SFTP. |
| Is Compressed | Check the option to select for file format *.zip, *,tar, *.tar.gz |
| File Filter | Provide a file pattern. File filter is used to only include files with file names matching the pattern. For e.g *.pdf or *emp *.csv |
| Recursive File Lookup | Check the option to retrieve the files from current/sub-folder(s). |
File Filter and recusursive file lookup will be available when the binary format is selected.
Configure Pre-Action in Source →
If you have any feedback on Gathr documentation, please email us!