Skip to content
Connect

Getting data from an SFTP server

Connecting to an SFTP server

Section titled “Connecting to an SFTP server”

To get data from a remote SFTP server, execute connect get sftp with following mandatory parameters:

  • -h, --host followed by the remote IP
  • -l, --login followed by the remote SFTP user login

Depending on the type of authentication you need to either use

  • -p, --password then specify the password (for password-based authentication)
  • --password-file then specify the file containing the password

or

  • -i, --identity-file specify the private key (for key based authentication)
  • -k, --identity-passphrase specify optional password to the private key
  • --identity-passphrase-file specify optional file containing the password to private key

Optionally, specify the target port if the remote server listens on a port other than 22/TCP:

  • --port followed by the target TCP port

You can also set a timeout for network operations, which defaults to 30s. If the server is on a local network, consider lowering the timeout. If the server is slow or distant, consider increasing the timeout:

  • --timeout followed by the timeout in seconds

Getting single file from an SFTP server

Section titled “Getting single file from an SFTP server”

To get a single file from an SFTP server you should use

  • --file followed by the file name to be downloaded

Following command will get file1.zip file from [email protected] SFTP server:

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -p password --file file1.zip
2025/06/04 14:39:42 INFO Getting files from sftp://[email protected]:
[file1.zip] 10.00 MiB / 10.00 MiB done

Getting multiple files from an SFTP server

Section titled “Getting multiple files from an SFTP server”

If you’d like to get multiple files from an SFTP server you need to specify either file mask or file regular expression for the files.

  • --file-mask a file mask
  • --file-regex regular expression

This will download all files that match a certain mask or pattern. You can also provide an optional source folder that contains the files to be downloaded:

  • -s, --src-folder folder containing the files to download

If you don’t provide the source folder files will be downloaded from the home folder of the SFTP user.

Following command will get all .zip files from /data folder of remote SFTP [email protected] It will read the password from password-file file and encrypt the file upon first read.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 --password-file password-file --file-mask "*.zip" -s /data
2025/06/04 14:42:00 INFO Getting files from sftp://[email protected]:/data
[file1.zip] 10.00 MiB / 10.00 MiB done
[file2.zip] 10.00 MiB / 10.00 MiB done
[file3.zip] 10.00 MiB / 10.00 MiB done

You can also provide an optional destination folder where the files will be downloaded to:

  • -d, --dst-folder target folder where to download files

If you don’t provide the destination folder files will be downloaded to the current folder.

In below example all .zip files from /data folder on [email protected] SFTP will be downloaded to local folder.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-regex ".+\.zip" -s /data -d local
2025/06/04 14:43:24 INFO Getting files from sftp://[email protected]:/data
[file1.zip] 10.00 MiB / 10.00 MiB done
[file2.zip] 10.00 MiB / 10.00 MiB done
[file3.zip] 10.00 MiB / 10.00 MiB done

Getting files from an SFTP server in batch mode

Section titled “Getting files from an SFTP server in batch mode”

When using the application in batch or non-interactive mode, such as in a script or scheduler, you should disable the progress bar by using

  • --batch to get files in batch mode

Following command will get all .zip files from the /data folder of SFTP server [email protected] and place them in the local folder, using batch mode: It will read the password to the private key from password-file file and encrypt the file upon first read.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-mask "*.zip" -s /data -d local --batch --identity-passphrase-file passphrase-file
2025/06/04 14:46:32 INFO Getting files from sftp://[email protected]:/data
2025/06/04 14:46:32 INFO Files to get: count=3
2025/06/04 14:46:34 INFO Transferring file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO File transferred successfully bytesTransferred=10485760 duration=245ms transferRate=40.66MB/s file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO Transferring file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO File transferred successfully bytesTransferred=10485760 duration=189ms transferRate=52.70MB/s file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO Transferring file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO File transferred successfully bytesTransferred=10485760 duration=228ms transferRate=43.78MB/s file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:46:34 INFO Disconnected sftpSessionID=1 endpoint=sftp://[email protected]:

Getting files from an SFTP server using scheduler

Section titled “Getting files from an SFTP server using scheduler”

When running the application from a scheduler, it’s recommended to ensure only one instance operates on a specific folder. It’s possible to configure the application to create a flag file at startup and delete it upon completion. If another instance tries to start while one is already running, it will detect the flag file and exit. This approach ensures that only one instance of the application runs at any given time. To do that use

  • --flag path to the flag file

Following command will get all .zip files from the /data folder of SFTP server [email protected] and place them in the local folder using /tmp/flag1 as a flag file. Mind that we are using batch mode as it’s required for the application to run correctly from script or scheduler.

Terminal window
$ connect get sftp --flag /tmp/flag1 -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-mask "*.zip" -s /data -d local --batch
2025/06/04 14:52:20 INFO Using flag file: /tmp/flag1
2025/06/04 14:52:20 INFO Getting files from sftp://[email protected]:/data
2025/06/04 14:52:20 INFO Files to get: count=3
2025/06/04 14:52:22 INFO Transferring file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO File transferred successfully bytesTransferred=10485760 duration=241ms transferRate=41.41MB/s file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO Transferring file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO File transferred successfully bytesTransferred=10485760 duration=180ms transferRate=55.33MB/s file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO Transferring file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO File transferred successfully bytesTransferred=10485760 duration=207ms transferRate=48.12MB/s file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:52:22 INFO Disconnected sftpSessionID=1 endpoint=sftp://[email protected]:
2025/06/04 14:52:22 INFO Removing flag file: /tmp/flag1

Getting files from an SFTP server in parallel

Section titled “Getting files from an SFTP server in parallel”

To speed up file download you can fetch files simultaneously. Application will then fetch files using multiple SFTP sessions. To specify number of concurrent SFTP sessions use:

  • --parallel number of SFTP sessions

Following command will get all .zip files from /data folder of SFTP server [email protected] server using 3 parallel sessions.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-mask "*.zip" -s /data --batch --parallel 3
2025/06/04 14:54:10 INFO Getting files from sftp://[email protected]:/data
2025/06/04 14:54:10 INFO Files to get: count=3
2025/06/04 14:54:11 INFO Transferring file=/data/file1.zip sftpSessionID=1 targetFolder=
2025/06/04 14:54:13 INFO Transferring file=/data/file3.zip sftpSessionID=3 targetFolder=
2025/06/04 14:54:13 INFO Transferring file=/data/file2.zip sftpSessionID=2 targetFolder=
2025/06/04 14:54:13 INFO File transferred successfully bytesTransferred=52428800 duration=1s463ms transferRate=34.16MB/s file=/data/file1.zip sftpSessionID=1 targetFolder=
2025/06/04 14:54:14 INFO File transferred successfully bytesTransferred=52428800 duration=1s483ms transferRate=33.70MB/s file=/data/file3.zip sftpSessionID=3 targetFolder=
2025/06/04 14:54:15 INFO File transferred successfully bytesTransferred=52428800 duration=1s921ms transferRate=26.02MB/s file=/data/file2.zip sftpSessionID=2 targetFolder=
2025/06/04 14:54:15 INFO Disconnected sftpSessionID=1 endpoint=sftp://[email protected]:
2025/06/04 14:54:15 INFO Disconnected sftpSessionID=3 endpoint=sftp://[email protected]:
2025/06/04 14:54:15 INFO Disconnected sftpSessionID=2 endpoint=sftp://[email protected]:

Getting files from an SFTP server in sequence

Section titled “Getting files from an SFTP server in sequence”

If you don’t specify the --parallel option then files will be downloaded using single SFTP session in the same order as they showed up on the filesystem (oldest files first).

Action after file is downloaded from an SFTP server

Section titled “Action after file is downloaded from an SFTP server”

After file is successfully downloaded to local system it’s possible to remove it from source or move it to a different folder at the source system.

  • --delete delete a file after it’s downloaded
  • --move-folder target folder to move the file after it’s downloaded

Following command will get all .zip files from /data folder of [email protected] server and store them in local folder. Then files will be moved to /archive folder at the source system.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-mask "*.zip" -s /data -d local --move-folder /archive --batch
2025/06/04 14:55:50 INFO Getting files from sftp://[email protected]:/data
2025/06/05 14:55:50 INFO Files to get: count=3
2025/06/04 14:55:51 INFO Transferring file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO File moved to folder folder=/archive file=/data/file1.zip sftpSessionID=1
2025/06/04 14:55:51 INFO File transferred successfully bytesTransferred=10485760 duration=292ms transferRate=34.22MB/s file=/data/file1.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO Transferring file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO File moved to folder folder=/archive file=/data/file2.zip sftpSessionID=1
2025/06/04 14:55:51 INFO File transferred successfully bytesTransferred=10485760 duration=179ms transferRate=55.60MB/s file=/data/file2.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO Transferring file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO File moved to folder folder=/archive file=/data/file3.zip sftpSessionID=1
2025/06/04 14:55:51 INFO File transferred successfully bytesTransferred=10485760 duration=220ms transferRate=45.43MB/s file=/data/file3.zip sftpSessionID=1 targetFolder=local
2025/06/04 14:55:51 INFO Disconnected sftpSessionID=1 endpoint=sftp://[email protected]:

Following command gets all .zip files from /data folder of [email protected] and then removes the files from source system.

Terminal window
$ connect get sftp -h 10.0.0.24 -l monitor01 -i .ssh/id_ed25519 --file-mask "*.zip" -s /data --batch --delete
2025/06/04 14:56:51 INFO Getting files from sftp://[email protected]:/data
2025/06/04 14:56:51 INFO Files to get: count=3
2025/06/04 14:56:53 INFO Transferring file=/data/file3.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:53 INFO File deleted file=/data/file3.zip sftpSessionID=1
2025/06/04 14:56:53 INFO File transferred successfully bytesTransferred=10485760 duration=283ms transferRate=35.28MB/s file=/data/file3.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:53 INFO Transferring file=/data/file2.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:53 INFO File deleted file=/data/file2.zip sftpSessionID=1
2025/06/04 14:56:54 INFO File transferred successfully bytesTransferred=10485760 duration=182ms transferRate=54.75MB/s file=/data/file2.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:54 INFO Transferring file=/data/file1.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:54 INFO File deleted file=/data/file1.zip sftpSessionID=1
2025/06/04 14:56:54 INFO File transferred successfully bytesTransferred=10485760 duration=207ms transferRate=48.20MB/s file=/data/file1.zip sftpSessionID=1 targetFolder=
2025/06/04 14:56:54 INFO Disconnected sftpSessionID=1 endpoint=sftp://[email protected]:

All SFTP Get Options

Section titled “All SFTP Get Options”
Terminal window
Usage:
  connect get sftp [command] [flags]


Flags:
  -h, --host string                       Source sftp host
  -i, --identity-file string              Private key file to use for authentication
  -k, --identity-passphrase string        Private key passphrase
      --identity-passphrase-file string   Private key passphrase file
      --idle uint                         Timeout for no network activity in seconds (default 60)
  -l, --login string                      Source sftp host login
  -p, --password string                   Source sftp host password
      --password-file string              Destination sftp host password file
      --port uint                         Source sftp host port (default 22)
  -t, --timeout uint                      Timeout for network operations in seconds (default 30)


Global Flags:
      --batch                     No progress bars
      --delete                    Try to delete files after successful get
  -d, --dst-folder string         Destination folder for retrieved files
      --file string               File name to be get
      --file-mask string          File mask to filter files
      --file-regex string         File regex to filter files
      --flag string               Flag file name
      --from-mail string          From mail used to send notifications (only in batch mode)
      --godebug                   Turns on debug mode
      --help                      Prints help for the command
      --log-format string         Log output format: text|json (default "text")
      --mail-format string        Mail format [text|html] (only in batch mode) (default "plain")
      --move-folder string        Folder to move files after successful get
      --no-color                  Do not use colors in logs
      --parallel uint             Number of sessions used to get files (default 1)
      --quiet                     Makes no output
  -s, --src-folder string         Folder to look for files
      --to-mail-failure strings   Email list to send failure notification (only in batch mode)
      --to-mail-success strings   Email list to send success notification (only in batch mode)

Errors

Section titled “Errors”

Please check here for information on the possible errors.