Linux Crontab

Sending files to a remote SFTP on a schedule

Let’s say we’d like to send all .zip files from /data folder
to SFTP server 10.0.0.24 using monitor02 user and pass password.
Moreover we’d like those files to be send automatically every 10 minutes.

1. First we need to test the connection

$ connect test sftp -h 10.0.0.24 -l monitor02 -p -
2024/11/27 13:32:37 INFO License valid to: 2026-04-13 18:06:28
2024/11/27 13:32:37 INFO Licensed for: Ultimate
Enter Password:
2024/11/27 13:32:41 INFO Testing connection to sftp://[email protected]
The key of 10.0.0.24:22 is unknown. Do you want to add this key to known_hosts (y/n): y
2024/11/27 13:32:44 INFO Connection successful

Note

Upon first connection we’ll be asked to add the remote server host key to known_hosts file.

2. Then we need to store the password in a file and rerun the connection test

$ echo "pass" > /apps/job1_pass
$ connect test sftp -h 10.0.0.24 -l monitor02 --password-file /apps/job1_pass
2024/11/26 22:19:02 INFO License valid to: 2026-04-13 18:06:28
2024/11/26 22:19:02 INFO Licensed for: Ultimate
2024/11/26 22:19:02 INFO Testing connection to sftp://[email protected]
2024/11/26 22:19:04 INFO Connection successful
$ cat /apps/job1_pass
ENC:LyHTac9EUBH8W4WO1RvyIB3m2pg32c5EnkUcTRefVSk=

Note

Upon first use password got encrypted.

3. Then we need to prepare the connect sftp send command

$ connect send sftp \
                         -h 10.0.0.24 \
                         -l monitor02 \
                         --password-file /apps/job1_pass \
                         -s /data \
                         --file-mask "*.zip" \
                         --delete \
                         --parallel 3 \
                         --flag /tmp/job1 \
                         --batch \
                         --no-color
2024/11/26 22:31:02 INFO License valid to: 2026-04-13 18:06:28
2024/11/26 22:31:02 INFO Licensed for: Ultimate
2024/11/26 22:31:02 INFO Using flag file: /tmp/job1
2024/11/26 22:31:02 INFO Sending files to sftp://[email protected]:
2024/11/26 22:31:05 INFO Transferring file=/data/file3.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO File deleted file=/data/file3.zip sftpSessionID=2
2024/11/26 22:31:05 INFO File transferred successfully bytesTransferred=1048576 duration=98ms transferRate=10.12MB/s file=/data/file3.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO Transferring file=/data/file1.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO File deleted file=/data/file1.zip sftpSessionID=2
2024/11/26 22:31:05 INFO File transferred successfully bytesTransferred=1048576 duration=91ms transferRate=10.98MB/s file=/data/file1.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO Transferring file=/data/file2.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO File deleted file=/data/file2.zip sftpSessionID=2
2024/11/26 22:31:05 INFO File transferred successfully bytesTransferred=1048576 duration=28ms transferRate=34.87MB/s file=/data/file2.zip sftpSessionID=2 targetFolder=""
2024/11/26 22:31:05 INFO Disconnected sftpSessionID=3 target=sftp://[email protected]:
2024/11/26 22:31:05 INFO Disconnected sftpSessionID=1 target=sftp://[email protected]:
2024/11/26 22:31:05 INFO Disconnected sftpSessionID=2 target=sftp://[email protected]:
2024/11/26 22:31:05 INFO Removing flag file: /tmp/job1

Parameters

-h 10.0.0.24 IP address of the target SFTP server
-l monitor02 SFTP user name on the target server
--password-file /apps/job1_pass file containing the password
-s /data source folder containing the files to send
--file-mask "*.zip" file mask for the files to send
--delete we want to delete files after successful transmission
--parallel 3 we want to send files using 3 parallel sessions
--flag /tmp/job1 we want only single instance of our job running
--batch we want to progress bar
--no-color disables colored output

4. Lastly we need to add the send command to crontab.

Let’s open the crontab editor with crontab -e and add desired command

*/10 * * * * /usr/bin/connect send sftp -h 10.0.0.24 -l monitor02 --password-file /apps/job1_pass -s /data --file-mask "*.zip" --delete --parallel 3 --flag /tmp/job1 --batch --no-color

And then save and exit the editor.

Explanation

*/10: Every 10 minutes.
* (the second field): Every hour.
* (the third field): Every day of the month.
* (the fourth field): Every month.
* (the fifth field): Every day of the week.

To log the execution of the job to a file add >>/apps/job1.log at the end of the command and also add --no-color flag.

*/10 * * * * /usr/bin/connect send sftp -h 10.0.0.24 -l monitor02 --password-file /apps/job1_pass -s /data --file-mask "*.zip" --delete --parallel 3 --flag /tmp/job1 --batch --no-color >>/apps/job1.log