Synametrics Technologies
Revolving around the core of technology
Document information
| Document ID: | 1501 |
|---|---|
| Subject: | Tips on running backup containing many files |
| Creation date: | 9/13/10 10:22 AM |
| Last modified on: | 9/13/12 10:41 AM |
Tips on running backup with many files
Before backup is performed, Syncrify analyzes the number of files it needs to backup in one run. If you are trying to backup millions of files, it will take a long time to gather this information. In such cases we suggest you break up the task into multiple chunks.
Consider the following scenario where you want to backup your F:\Data folder. This folder contains sub-folders like the following image.
Assume each folder contains around 100,000 files adding to a total of 1,200,000 files.
One way to add these files to a backup profile is to add F:\Data folder, like the image below.
Syncrify builds one list per top-level folder. Since every sub-folder in this example is bundled together, building a file list of that many files can take a long time.
A better approach is to have multiple top-level folders as depicted by the image below.
Try dragging files from Windows Explorer on to Syncrify client to speed up the process. There is no limit on the number of top-level folders for one profile.
There are two benefits of using multiple top-level folders.
- Each folder takes less amount of time to backup
- Less memory will be used to complete the task.
User comments
Posted by Onno on 4/21/11 4:36 AM
I totally agree wth you Stuart. I had lots of problems when trying to backup a very large disk. Support (Excellent) directed me to this page. I have tried it with succes! But if a new folder gets added I have to manually add it to the job. And thats indeed not very nice behavor. I think Syncrify is a very cool product, but a little to little for the big jobs.
But I think with a little performance boost (native C or .NET) and a little smarter backup maintenance (also need a continious backup job or time machine kind of backup mode)
it can be every administrator's friend.
Posted by Stuart on 1/25/11 3:38 PM
I really think the developers need to automatically break top level folders into multiple backup jobs. Requiring the user to specify and keep track of additions to and deletions from the top-level folders is pretty lame.
It would not be difficult to parse the top level folder and then follow each branch to create 'pseudo folders' that then determine what gets done.
Simplistically, you could enumerate down to each leaf folder and go from there.
Add a comment to this document
Do you have a helpful tip related to this document that you'd like to share with other users?
Important: This area is reserved for useful tips. Therefore, do not post any questions here. Instead, use our public forums to post questions.
