Document information

Document ID:	1501
Subject:	Tips on running backup containing many files
Creation date:	9/13/10 10:22 AM
Last modified on:	9/13/12 10:41 AM

Tips on running backup with many files

IMPORTANT:
This page is obsolete as of version 3.0, which automatically handles this situation.

Before backup is performed, Syncrify analyzes the number of files it needs to backup in one run. If you are trying to backup millions of files, it will take a long time to gather this information. In such cases we suggest you break up the task into multiple chunks.

Consider the following scenario where you want to backup your F:\Data folder. This folder contains sub-folders like the following image.

Assume each folder contains around 100,000 files adding to a total of 1,200,000 files.

One way to add these files to a backup profile is to add F:\Data folder, like the image below.

Syncrify builds one list per top-level folder. Since every sub-folder in this example is bundled together, building a file list of that many files can take a long time.

A better approach is to have multiple top-level folders as depicted by the image below.

Try dragging files from Windows Explorer on to Syncrify client to speed up the process. There is no limit on the number of top-level folders for one profile.

There are two benefits of using multiple top-level folders.

Each folder takes less amount of time to backup
Less memory will be used to complete the task.

User comments

Posted by Onno on 4/21/11 4:36 AM

I totally agree wth you Stuart. I had lots of problems when trying to backup a very large disk. Support (Excellent) directed me to this page. I have tried it with succes! But if a new folder gets added I have to manually add it to the job. And thats indeed not very nice behavor. I think Syncrify is a very cool product, but a little to little for the big jobs. But I think with a little performance boost (native C or .NET) and a little smarter backup maintenance (also need a continious backup job or time machine kind of backup mode) it can be every administrator's friend.

Posted by Stuart on 1/25/11 3:38 PM

I really think the developers need to automatically break top level folders into multiple backup jobs. Requiring the user to specify and keep track of additions to and deletions from the top-level folders is pretty lame. It would not be difficult to parse the top level folder and then follow each branch to create 'pseudo folders' that then determine what gets done. Simplistically, you could enumerate down to each leaf folder and go from there.

Add a comment to this document

Do you have a helpful tip related to this document that you'd like to share with other users? Please add it below. Your name and tip will appear at the end of the document text.

Your name:
Your email:	Hide my email address
Verification code:	Enter the verification code you see above more submitting your tip
Tip:	Please limit tips to 1000 characters

Quick Links

Video Tutorials

Industry solutions

Document information

Tips on running backup with many files

User comments

Add a comment to this document