Linux SystemD Timers

Sending files to a remote SFTP on a schedule

In modern Linux systems you should be using SystemD timers in favor of legacy crontab. Timers allow more flexible scheduling, support for built-in logging, resource limits, sandboxing and others features.

Let’s say we’d like to send all .zip files from /data folder
to target folder of SFTP server 10.0.0.24 using monitor02 user and id_ed25519 key that is password protected. Moreover we’d like those files to be send automatically every 5 minutes past every hour.

First we need to test the connection

$ connect test sftp -h 10.0.0.24 -l monitor02 -i .ssh/id_ed25519 -k -
2024/11/27 13:39:14 INFO License valid to: 2026-04-13 18:06:28
2024/11/27 13:39:14 INFO Licensed for: Ultimate
Enter Private Key Passphrase:
2024/11/27 13:39:17 INFO Testing connection to sftp://[email protected]
The key of 10.0.0.24:22 is unknown. Do you want to add this key to known_hosts (y/n): y
2024/11/27 13:39:18 INFO Connection successful

Then we need to store the key password in a file and rerun the connection test

$ echo "pass" > /apps/job1_pass
$ connect test sftp -h 10.0.0.24 -l monitor02 -i .ssh/id_ed25519 --identity-passphrase-file /apps/job1_pass
2024/11/27 13:43:35 INFO License valid to: 2026-04-13 18:06:28
2024/11/27 13:43:35 INFO Licensed for: Ultimate
2024/11/27 13:43:35 INFO Testing connection to sftp://[email protected]
2024/11/27 13:43:38 INFO Connection successful
$ cat /apps/job1_pass
ENC:ccE8Nqy2p0wNZHeBrwZ2ATr9fm4LKdXNSf9PZ6f5qJk=

Then we need to prepare the connect sftp send command

$ connect send sftp \
                     -h 10.0.0.24 \
                     -l monitor02 \
                     -i /apps/keys/id_ed25519 \
                     --identity-passphrase-file /apps/job1_pass \
                     -s /data \
                     --file-mask "*.zip" \
                     -d target \
                     --delete \
                     --parallel 3 \
                     --flag /tmp/job1 \
                     --batch \
                     --no-color
2024/11/27 15:45:51 INFO License valid to: 2026-04-13 18:06:28
2024/11/27 15:45:51 INFO Licensed for: Ultimate
2024/11/27 15:45:51 INFO Using flag file: /tmp/job1
2024/11/27 15:45:51 INFO Sending files to sftp://[email protected]:target
2024/11/27 15:45:54 INFO Transferring file=/data/file3.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:54 INFO File deleted file=/data/file3.zip sftpSessionID=2
2024/11/27 15:45:54 INFO File transferred successfully bytesTransferred=1048576 duration=39ms transferRate=25.53MB/s file=/data/file3.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:54 INFO Transferring file=/data/file1.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:54 INFO File deleted file=/data/file1.zip sftpSessionID=2
2024/11/27 15:45:54 INFO File transferred successfully bytesTransferred=1048576 duration=30ms transferRate=32.70MB/s file=/data/file1.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:54 INFO Transferring file=/data/file2.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:55 INFO File deleted file=/data/file2.zip sftpSessionID=2
2024/11/27 15:45:55 INFO File transferred successfully bytesTransferred=1048576 duration=25ms transferRate=38.87MB/s file=/data/file2.zip sftpSessionID=2 targetFolder=target
2024/11/27 15:45:55 INFO Disconnected sftpSessionID=3 target=sftp://[email protected]:
2024/11/27 15:45:55 INFO Disconnected sftpSessionID=1 target=sftp://[email protected]:
2024/11/27 15:45:55 INFO Disconnected sftpSessionID=2 target=sftp://[email protected]:
2024/11/27 15:45:55 INFO Removing flag file: /tmp/job1

Prepare sending service

$ systemctl --user edit --force --full job1.service
[Unit]
Description=SFTP job1 Service
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/apps/connect send sftp -h 10.0.0.24 -l monitor02 --password-file /apps/job1_pass -s /data --file-mask "*.zip" -d target --delete --parallel 3 --flag /tmp/job1 --batch --no-color

Prepare SystemD timer

$ systemctl --user edit --force --full job1.timer
[Unit]
Description=Timer for job1 Service

[Timer]
OnCalendar=*:05:00
Persistent=false

[Install]
WantedBy=timers.target

You can use systemd-analyze command to check if your schedule rule is correct.

$ systemd-analyze calendar "*:05:00" --iterations 5
  Original form: *:05:00
Normalized form: *-*-* *:05:00
    Next elapse: Thu 2024-11-28 12:05:00 UTC
       From now: 28min left
   Iteration #2: Thu 2024-11-28 13:05:00 UTC
       From now: 1h 28min left
   Iteration #3: Thu 2024-11-28 14:05:00 UTC
       From now: 2h 28min left
   Iteration #4: Thu 2024-11-28 15:05:00 UTC
       From now: 3h 28min left
   Iteration #5: Thu 2024-11-28 16:05:00 UTC
       From now: 4h 28min left

Enable the timer

$ systemctl --user daemon-reload
$ systemctl --user enable job1.timer
Created symlink /apps/.config/systemd/user/timers.target.wants/job1.timer → /apps/.config/systemd/user/job1.timer.
$ systemctl --user start job1.timer
$ systemctl --user status job1.timer
● job1.timer - Timer for job1
     Loaded: loaded (/apps/.config/systemd/user/job1.timer; enabled; preset: enabled)
     Active: active (waiting) since Thu 2024-11-28 11:49:59 UTC; 27s ago
    Trigger: Thu 2024-11-28 12:05:00 UTC; 14min left
   Triggers: ● job1.service

Nov 28 11:49:59 arm systemd[33107]: Started job1.timer - Timer for job1.

Allow apps user’s systemd services to run even after the user has logged out.

$ sudo loginctl enable-linger apps

Check the job logs

$ journalctl --user -eu job1.service
Dec 03 07:48:44 arm systemd[37960]: Started job1.service - SFTP job1 Service.
Dec 03 07:48:44 arm connect[54943]: 2024/12/03 07:48:44 INFO License valid to: 2026-04-13 18:06:28
Dec 03 07:48:44 arm connect[54943]: 2024/12/03 07:48:44 INFO Licensed for: Ultimate
Dec 03 07:48:44 arm connect[54943]: 2024/12/03 07:48:44 INFO Using flag file: /tmp/job1
Dec 03 07:48:44 arm connect[54943]: 2024/12/03 07:48:44 INFO Sending files to sftp://[email protected]:target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Transferring file=/data/file3.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File deleted file=/data/file3.zip sftpSessionID=2
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File transferred successfully bytesTransferred=1048576 duration=52ms transferRate=19.16MB/s file=/data/file3.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Transferring file=/data/file1.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File deleted file=/data/file1.zip sftpSessionID=2
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File transferred successfully bytesTransferred=1048576 duration=103ms transferRate=9.67MB/s file=/data/file1.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Transferring file=/data/file2.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File deleted file=/data/file2.zip sftpSessionID=2
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO File transferred successfully bytesTransferred=1048576 duration=33ms transferRate=29.99MB/s file=/data/file2.zip sftpSessionID=2 targetFolder=target
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Disconnected sftpSessionID=3 target=sftp://[email protected]:
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Disconnected sftpSessionID=1 target=sftp://[email protected]:
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Disconnected sftpSessionID=2 target=sftp://[email protected]:
Dec 03 07:48:47 arm connect[54943]: 2024/12/03 07:48:47 INFO Removing flag file: /tmp/job1