Skip to content
Connect

Sending data to an Azure blob container

Connecting to an Azure blob container

Section titled “Connecting to an Azure blob container”

To send data to an Azure blob container, you need to execute connect send az.

Depending on the type of authentication you need to either use account name, account key and blob container name:

  • --az-container then the blob container name
  • --az-account-name then the AZ account name
  • --az-account-key then the AZ account key

Or you can use connection string:

  • --az-connection-string then the connection string

Or use Shared Access Signature (SAS) URL:

  • --az-sas-url then the SAS URL

Sending single file to an Azure blob container

Section titled “Sending single file to an Azure blob container”

To send a single file to an Azure blob container you should use

  • --file followed by the file name to be sent

Following command will send file1.zip file to an Azure blob container my-az-blob using account name and account key for authentication.

Terminal window
$ connect send az --file file1.zip --az-container my-az-blob --az-account-name myaccount --az-account-key 1O8MbYTgY8Z/KD7Ip181dar7zvWfpTyzGIVdX1WsCmBSRj+XL1oXTkcMWri2ejhfmcA7/ZAFO6YL+ASt/GRBow==
2025/06/04 08:33:18 INFO Sending files to az://my-az-blob/
2025/06/04 08:33:18 INFO Files to send: count=1
[file1.zip] 10.00 MiB / 10.00 MiB done

Sending multiple files to an Azure blob container

Section titled “Sending multiple files to an Azure blob container”

If you’d like to send multiple files to an Azure blob container you need to specify either file mask or file regular expression for the files.

  • --file-mask a file mask
  • --file-regex regular expression

This will cause all files that match a certain mask or pattern to be sent. You can also provide an optional source folder that contain the files to be sent:

  • -s, --src-folder

If you don’t provide the source folder files will be sent from the current folder.

Following command will send all .zip files from /data folder to an Azure blob container my-az-blob using Shared Access Signature (SAS) URL to authenticate.

Terminal window
$ connect send az --file-mask "*.zip" -s /data --az-sas-url "https://myaccount.blob.core.windows.net/my-az-blob?sp=racwdli&st=2024-09-11T14:53:45Z&se=2024-09-25T22:53:45Z&spr=https&sv=2022-11-02&sr=c&sig=WZl3RWGl6pPHKUVyGlWB0ZxYjcmIK%2FbCkdjglw2wku0%3D"
2025/06/04 10:45:45 INFO Sending files to az://my-az-blob/
2025/06/04 10:45:45 INFO Files to send: count=3
[file2.zip] 10.00 MiB / 10.00 MiB done
[file3.zip] 10.00 MiB / 10.00 MiB done
[file1.zip] 10.00 MiB / 10.00 MiB done

You can also provide an optional destination folder on Azure blob container:

  • -d, --dst-folder

If you don’t provide the destination folder files will be sent to the “home folder” of the Azure blob container (top-level of the Azure blob container).

In below example all .zip files from /data folder will be sent to remote folder of Azure blob container my-az-blob using Shared Access Signature (SAS) URL to authenticate.

Terminal window
$ connect send az --file-mask "*.zip" -s /data -d remote --az-sas-url "https://myaccount.blob.core.windows.net/my-az-blob?sp=racwdli&st=2024-09-11T14:53:45Z&se=2024-09-25T22:53:45Z&spr=https&sv=2022-11-02&sr=c&sig=WZl3RWGl6pPHKUVyGlWB0ZxYjcmIK%2FbCkdjglw2wku0%3D"
2025/06/04 10:56:13 INFO Sending files to az://my-az-blob/remote
2025/06/04 10:56:13 INFO Files to send: count=3
[file1.zip] 10.00 MiB / 10.00 MiB done
[file2.zip] 10.00 MiB / 10.00 MiB done
[file3.zip] 10.00 MiB / 10.00 MiB done

Sending files in batch mode to an Azure blob container

Section titled “Sending files in batch mode to an Azure blob container”

When using the application in batch or non-interactive mode, such as in a script or scheduler, you should disable the progress bar by using

  • --batch to send files in batch mode

Following command will send all .zip files from the /data folder to the remote folder of Azure blob container my-az-blob using Shared Access Signature (SAS) URL to authenticate.

Terminal window
$ connect send az --file-mask "*.zip" -s /data -d remote --batch --az-sas-url "https://myaccount.blob.core.windows.net/my-az-blob?sp=racwdli&st=2024-09-11T14:53:45Z&se=2024-09-25T22:53:45Z&spr=https&sv=2022-11-02&sr=c&sig=WZl3RWGl6pPHKUVyGlWB0ZxYjcmIK%2FbCkdjglw2wku0%3D"
2025/06/04 12:54:58 INFO Sending files to az://my-az-blob/remote
2025/06/04 12:54:58 INFO Files to send: count=3
2025/06/04 12:54:58 INFO Transferring file=/data/file1.zip targetFolder=remote
2025/06/04 12:54:58 INFO File transferred successfully bytesTransferred=10485760 duration=200ms transferRate=49.86MB/s file=/data/file1.zip targetFolder=remote
2025/06/04 12:54:58 INFO Transferring file=/data/file2.zip targetFolder=remote
2025/06/04 12:54:58 INFO File transferred successfully bytesTransferred=10485760 duration=192ms transferRate=51.82MB/s file=/data/file2.zip targetFolder=remote
2025/06/04 12:54:58 INFO Transferring file=/data/file3.zip targetFolder=remote
2025/06/04 12:54:58 INFO File transferred successfully bytesTransferred=10485760 duration=133ms transferRate=75.05MB/s file=/data/file3.zip targetFolder=remote

Sending files from scheduler to an Azure blob container

Section titled “Sending files from scheduler to an Azure blob container”

When running the application from a scheduler, it’s recommended to ensure only one instance operates on a specific folder. To do that you can configure the application to create a flag file at startup and delete it upon completion. If another instance tries to start while one is already running, it will detect the flag file and exit. This approach ensures that only one instance of the application runs at any given time. To do that use

  • --flag path to the flag file

Following command will send all .zip files from the /data folder to the remote folder of Azure blob container my-az-blob using account name and account key for authentication. Application will create a flag wile /tmp/flag1 and remove it on exit. Mind that we are using batch mode as it’s required for the application to run correctly from script or scheduler.

Terminal window
$ connect send az --flag /tmp/flag1 --batch --file-mask "*.zip" -s /data -d remote --az-container my-az-blob --az-account-name myaccount --az-account-key 1O8MbYTgY8Z/KD7Ip181dar7zvWfpTyzGIVdX1WsCmBSRj+XL1oXTkcMWri2ejhfmcA7/ZAFO6YL+ASt/GRBow==
2025/06/04 12:55:41 INFO Using flag file: /tmp/flag1
2025/06/04 12:55:41 INFO Sending files to az://my-az-blob/remote
2025/06/04 12:55:41 INFO Files to send: count=3
2025/06/04 12:55:41 INFO Transferring file=/data/file1.zip targetFolder=remote
2025/06/04 12:55:42 INFO File transferred successfully bytesTransferred=10485760 duration=171ms transferRate=58.34MB/s file=/data/file1.zip targetFolder=remote
2025/06/04 12:55:42 INFO Transferring file=/data/file2.zip targetFolder=remote
2025/06/04 12:55:42 INFO File transferred successfully bytesTransferred=10485760 duration=169ms transferRate=58.89MB/s file=/data/file2.zip targetFolder=remote
2025/06/04 12:55:42 INFO Transferring file=/data/file3.zip targetFolder=remote
2025/06/04 12:55:42 INFO File transferred successfully bytesTransferred=10485760 duration=122ms transferRate=81.58MB/s file=/data/file3.zip targetFolder=remote
2025/06/04 12:55:42 INFO Removing flag file: /tmp/flag1

Sending files in parallel to an Azure blob container

Section titled “Sending files in parallel to an Azure blob container”

To speed up file delivery you can send files simultaneously. Application will then send files using multiple sessions. To specify number of concurrent Azure sessions use:

  • --parallel number of Azure sessions

Following command will send all .zip files from the /data folder to the remote folder of Azure blob container my-az-blob using account name and account key for authentication and 3 parallel sessions.

Terminal window
$ connect send az --file-mask "*.zip" -s /data -d remote --batch --parallel 3 --az-container my-az-blob --az-account-name myaccount --az-account-key 1O8MbYTgY8Z/KD7Ip181dar7zvWfpTyzGIVdX1WsCmBSRj+XL1oXTkcMWri2ejhfmcA7/ZAFO6YL+ASt/GRBow==
2025/06/04 12:56:25 INFO Sending files to az://my-az-blob/remote
2025/06/04 12:56:25 INFO Files to send: count=3
2025/06/04 12:56:25 INFO Transferring file=/data/file3.zip targetFolder=remote
2025/06/04 12:56:25 INFO Transferring file=/data/file1.zip targetFolder=remote
2025/06/04 12:56:25 INFO Transferring file=/data/file2.zip targetFolder=remote
2025/06/04 12:56:25 INFO File transferred successfully bytesTransferred=10485760 duration=161ms transferRate=61.92MB/s file=/data/file2.zip targetFolder=remote
2025/06/04 12:56:25 INFO File transferred successfully bytesTransferred=10485760 duration=180ms transferRate=55.37MB/s file=/data/file1.zip targetFolder=remote
2025/06/04 12:56:25 INFO File transferred successfully bytesTransferred=10485760 duration=238ms transferRate=41.97MB/s file=/data/file3.zip targetFolder=remote

Sending files in sequence to an Azure blob container

Section titled “Sending files in sequence to an Azure blob container”

If you don’t specify the --parallel option then files will be sent using single session in the same order as they showed up on the filesystem (oldest files first).

Action after file is delivered to an Azure blob container

Section titled “Action after file is delivered to an Azure blob container”

After file is successfully delivered to remote location it’s possible to remove it from source or move it to a different folder at the source system.

  • --delete delete a file after it’s delivered
  • --move-folder target folder to move the file after it’s delivered

Following command will send all .zip files from the /data folder to the remote folder of Azure blob container my-s3-bucket using account name and account key for authentication, 3 parallel sessions and then move the files to /archive at the source system.

Terminal window
$ connect send az --file-mask "*.zip" -s /data -d remote --batch --parallel 3 --move-folder /archive --az-container my-az-blob --az-account-name myaccount --az-account-key 1O8MbYTgY8Z/KD7Ip181dar7zvWfpTyzGIVdX1WsCmBSRj+XL1oXTkcMWri2ejhfmcA7/ZAFO6YL+ASt/GRBow==
2025/06/04 12:58:39 INFO Sending files to az://my-az-blob/remote
2025/06/04 12:58:39 INFO Files to send: count=3
2025/06/04 12:58:39 INFO Transferring file=/data/file3.zip targetFolder=remote
2025/06/04 12:58:39 INFO Transferring file=/data/file2.zip targetFolder=remote
2025/06/04 12:58:39 INFO Transferring file=/data/file1.zip targetFolder=remote
2025/06/04 12:58:39 INFO File moved to folder folder=/archive file=/data/file3.zip
2025/06/04 12:58:39 INFO File transferred successfully bytesTransferred=10485760 duration=183ms transferRate=54.36MB/s file=/data/file3.zip targetFolder=remote
2025/06/04 12:58:39 INFO File moved to folder folder=/archive file=/data/file2.zip
2025/06/04 12:58:39 INFO File transferred successfully bytesTransferred=10485760 duration=230ms transferRate=43.35MB/s file=/data/file2.zip targetFolder=remote
2025/06/04 12:58:39 INFO File moved to folder folder=/archive file=/data/file1.zip
2025/06/04 12:58:39 INFO File transferred successfully bytesTransferred=10485760 duration=230ms transferRate=43.34MB/s file=/data/file1.zip targetFolder=remote

Following command will send all .zip files from the /data folder to the remote folder of Azure blob container my-az-blob using account name and account key for authentication, 3 parallel sessions and later removing the files from source.

Terminal window
$ connect send az --file-mask "*.zip" -s /data -d remote --batch --parallel 3 --delete --az-container my-az-blob --az-account-name myaccount --az-account-key 1O8MbYTgY8Z/KD7Ip181dar7zvWfpTyzGIVdX1WsCmBSRj+XL1oXTkcMWri2ejhfmcA7/ZAFO6YL+ASt/GRBow==
2025/06/04 12:59:34 INFO Sending files to az://my-az-blob/remote
2025/06/04 12:59:34 INFO Files to send: count=3
2025/06/04 12:59:34 INFO Transferring file=/data/file3.zip targetFolder=remote
2025/06/04 12:59:34 INFO Transferring file=/data/file2.zip targetFolder=remote
2025/06/04 12:59:34 INFO Transferring file=/data/file1.zip targetFolder=remote
2025/06/04 12:59:34 INFO File deleted file=/data/file2.zip
2025/06/04 12:59:34 INFO File transferred successfully bytesTransferred=10485760 duration=168ms transferRate=59.26MB/s file=/data/file2.zip targetFolder=remote
2025/06/04 12:59:34 INFO File deleted file=/data/file3.zip
2025/06/04 12:59:34 INFO File transferred successfully bytesTransferred=10485760 duration=182ms transferRate=54.70MB/s file=/data/file3.zip targetFolder=remote
2025/06/04 12:59:34 INFO File deleted file=/data/file1.zip
2025/06/04 12:59:34 INFO File transferred successfully bytesTransferred=10485760 duration=256ms transferRate=38.96MB/s file=/data/file1.zip targetFolder=remote

All Azure Send Options

Section titled “All Azure Send Options”
Terminal window
Usage:
  connect send az [command] [flags]


Flags:
      --az-account-key string         AZ account key
      --az-account-name string        AZ account name
      --az-connection-string string   AZ connection string
      --az-container string           AZ container name
      --az-sas-url string             AZ SAS URL


Global Flags:
      --batch                     No progress bars
      --delete                    Delete files after successful send
  -d, --dst-folder string         Folder where files should be sent
      --file string               File name to be sent
      --file-mask string          File mask to filter files for sending
      --file-regex string         File regex to filter files for sending
      --flag string               Flag file name
      --from-mail string          From mail used to send notifications (only in batch mode)
      --godebug                   Turns on debug mode
      --help                      Prints help for the command
      --log-format string         Log output format: text|json (default "text")
      --mail-format string        Mail format [text|html] (only in batch mode) (default "plain")
      --move-folder string        Folder to move files after successful send
      --no-color                  Do not use colors in logs
      --parallel uint             Number of sessions used to send files (default 1)
      --quiet                     Makes no output
  -s, --src-folder string         Folder containing files to be sent (default "./")
      --to-mail-failure strings   Email list to send failure notification (only in batch mode)
      --to-mail-success strings   Email list to send success notification (only in batch mode)

Errors

Section titled “Errors”

Please check here for information on the possible errors.