Syncrify

Document information

Document ID:1361
Subject:Syncrify client appears hung when transferring large files
Creation date:6/3/10 1:57 PM
Last modified on:5/22/13 12:47 PM


Transferring large files


When transferring large file often users get an impression that Syncrify client is hung and is not doing anything. This is not correct. To ensure the Syncrify process is working, try opening the Task Manager on Windows or use the top program on Linux to monitor its activities. You should check its:
  1. CPU usage
  2. Disk I/O - both read and write
  3. Network usage

What takes long

Syncrify uses rsync algorithm to determine the portion of file that has changed and only transfers those bytes to the remove machine. This is done to save network traffic but at the cost of local disk I/O and CPU usage. At any given time Syncrify client can be doing one of the following:
  • Computing and/or comparing blocks - I/O and CPU is used
  • Transferring data - Network is used
  • Waiting for the server to respond - Idle
When you backup a file that is 20GB in size, it takes a long time even to perform a sequential read on the file.

What is block matching

In order to determine the part of file that has changed (aka delta), Syncrify compare data blocks between client and server. This process takes a long time if the file is large (in Gigabytes). This is a CPU and I/O intensive process with hardly any activity on the network.

Prior to the block matching process, the server splits the files into blocks and sends a list of MD5 signatures to the client. The client then searches for matching blocks on its end. The more blocks are matched the faster the matching process goes. The best case scenario occurs when every block on the client matches with the server and new data is appended at the end of the file. On the other hand, the worst case scenario occurs when none of the blocks match.
A few tips
  • Avoid remote disk - Often users try to backup files residing on a remote disk. Tremendous network traffic is generated when Syncrify client tries to backup a file residing on a remote disk. Click here for more information.
  • Avoid zipping a file - Zipping file rearranges every block within a file causing very few or none of the blocks to match. This not only increases the time it takes to match blocks, but the resulting delta is usually as big as the original file itself.
  • Avoid encryption if possible - When encryption is used, every file is encrypted before block matching starts adding more time to the entire process.
  • Avoid versioning - Blocks are matched twice when versioning is used. First on the client machine to determine delta between the file on the client and server and then, on the server to determine the delta between the previous version.




Add a comment to this document

Do you have a helpful tip related to this document that you'd like to share with other users? Please add it below. Your name and tip will appear at the end of the document text.
Your name:
Your email:
Hide my email address
Verification code:
Enter the verification code you see above more submitting your tip
Tip:Please limit tips to 1000 characters

Navigation

Social Media

Powered by 10MinutesWeb.com