Rsync Command: The Ideal and Reliable Way to Copy Large Files

Introduction

When transferring files to my Android phone or many of my external hard drives I never use the file manager nor do I use "cp" command, I noticed that with file managers the computer actually slows down and sometimes the file manager becomes non resposnive with large files. I always use rsync command.

And it has been a life saver and reliable, when copying files to a remote server over the network even with network interruptions and low netowrk connection

What is Rsync?

Rsync is a command-line utility that synchronizes files and directories between two locations while minimizing data transfer by only copying the differences between them. This mechanism makes Rsync incredibly efficient, particularly for large files where transferring the entire file each time is impractical. Its cross-platform support ensures seamless operation across Linux, macOS, and Windows systems.

Benefits of Using Rsync

Rsync offers several advantages for large file copying:

•Efficiency: Rsync's delta-transfer algorithm ensures that only the portions of files that have changed are transferred, reducing bandwidth usage.

•Reliability: Rsync's ability to resume interrupted transfers and handle network disruptions makes it a dependable choice for mission-critical operations.

•Versatility: Whether you're copying files over a network or to/from external devices, Rsync's adaptability shines through, providing consistent performance across various scenarios.

Real-World Examples Using Rsync

Example 1: Copying Large Files Over a Network:

Whenever I want to copy files to my website server I use the following command:

sshpass -p "mypassword" \
    rsync -avPh --bwlimit=1000 --timeout=120 \
    --exclude '__pycache__' \
    -e "ssh -oHostKeyAlgorithms=+ssh-dss  -p<portnumber> " "local/directory/or/file" [email protected]:/path/to/my/remote/directory

And it has been a life saver and reliable, even with network interruptions and low netowrk connection

Example 2: Copying Large Files to/from External Devices

Transferring media files to my Android phone via MTP or simply to an external drive can be unreliable with conventional methods. Rsync offers a robust solution, ensuring uninterrupted transfers even in the event of interruptions or connectivity issues.

•Basic Usage: Use the following syntax to copy files:

        rsync [options] source destination
    

When transferring files to my Android phone or many of my external hard drives I never use the file manager nor do I use "cp" command, I noticed that with file managers the computer actually slows down and sometimes the file manager becomes non resposnive with large files. I always use the following command:

        
rsync -ahrS --progress /path/to/source/	 /path/to/destination
# or simply navigate to destination and
rsync -ahrS --progress /path/to/source/	 . 	# notice the dot ie current directory
        
    

•Advanced Options: You can use additional options like '-avz' for archiving and compressing data, '-P' for progress updates, '--partial' for resuming interrupted transfers, --exclude '*.djvu' to exclude files with the .djvu extension, and '--max-size=53m' to limit the transfer of files smaller than 53 megabytes. the last two are especially usefull when synchronizing(copying=) a directory

Best Practices

•Optimize Performance: Experiment with compression and other options to maximize transfer speeds, especially over slower connections.

•Secure Transfers: Use SSH for secure data transfers, safeguarding your files during transit.

•Selective Copying: Leverage Rsync's options to exclude unnecessary files or directories, streamlining the copying process.

Conclusion

In the realm of large file copying, Rsync stands out as a beacon of reliability and efficiency. Whether you're synchronizing files over a network or transferring data to/from external devices, Rsync's prowess remains unmatched. By integrating Rsync into your workflow, you can streamline your file copying processes, ensuring swift and dependable transfers every time.