Must be the new moon. Only one reboot after shutting down overnight. There’s nothing unusual in the event log. Just the usual stuff from the Service Control Manager about starting and stopping VSS and SymSnapService services. Also virus signature updates and routine quick scans. These occur every day. Warnings or errors are not disk-related.
Robertj, first I need to point out that I do not have a magic answer to your dilemma of why some incrementals are larger. However, I can provide I little insight into what is happening that can aid your search. Second, the power down using the case switch vs the soft power down is a nice piece of detective work and sounds like a contributor to your problem.
As you probably already know, Ghost's primary method for backing up is sector based. A base recovery point gets all used sectors on the drive. An incremental gets only those sectors that are changed since the last backup. The Vsnap.Idx is the file that tracks what has changed on the drive. If it gets deleted or Ghost detects that it has problems, a reconcile process is started with the current state of the machine vs. the last backup so that Ghost can continue to track what is changed for the next incremental. When a file or image is restored from an incremental, you must have the original base and all incrementals upto and including the one you are restoring from otherwise the restore will fail. You can optimize or consolidate incrementals to save space and reduce the size of the incremental chain.
Ghost is very good at keeping track of what is changed on the hard drive. If an incremental is larger than another there is often good reason. One of the biggest causes of change on the drive is a disk defragmenter. That is why we recommend that if you defragment your drive, you should do it immediately prior to taking a base or your incremental can easily be larger than a base. But there are also may other causes for a larger incremental.
In you list above of various things I noticed that some things were left off the list of things that can increase the size of an incremental. These other things include, but are not limited to,
- updaters for various apps, e.g. Windows, Java or Adobe or iTunes or many others;
- peer-to-peer services, e.g. BitTorent or file sharing services or any other service that cache things on you hard drive;
- any service that updates information automatically from the network or internet, e.g. a gadget to get a news feed or an email package that checks for new messages and downloads them every x minutes;
- fileshares where other machines write to and store things on your machine;
- spyware and malware that collect information for uploading;
- automatic logging, e.g. the logs the system keeps and writes out periodically;
- indexers, e.g. file indexers or Google Desktop;
- page file, e.g. any thing that causes swapping to the page file;
- browsers, e.g. any internet browsing;
- Windows itself, e.g. the registry.
And this list is by no means complete.
The best way to get an idea of what causes drive activity on a system that I know if is to get FileMon from SysInternals. It can be rather eye opening experience.
Thanks, Johan for your comments. The advice about Filemon is well taken and I shall pursue this further. Your advice concerning VSNAP.IDX certainly confirms my observations. The other thing that is clear is that a base image and its associated incremental images must always be present and available in the same folder for any restoration to succeed. However, I should make a clear statement of the problem of concern here with a view to narrowing down the scope of any ensuing investigation.
Problem Statement:
After a volume reconciliation in which the volume content is compared against the last incremental backup, the resultant backup image size does not reflect disk drive activity since the last incremental backup.
Description:
To illustrate, consider the following backup history.
Date Created Contents Size
27/07/2008 12:21 DATA (L:) 45.4 GB
28/07/2008 09:15 DATA (L:) 3.5 GB
29/07/2008 09:15 DATA (L:) 467.5 MB
29/07/2008 18:52 DATA (L:) 1.0 GB
29/07/2008 18:56 DATA (L:) 512.0 KB
29/07/2008 19:43 DATA (L:) 512.0 KB
29/07/2008 19:45 DATA (L:) 512.0 KB
29/07/2008 19:47 DATA (L:) 512.0 KB
29/07/2008 19:50 DATA (L:) 512.0 KB
29/07/2008 19:52 DATA (L:) 512.0 KB
29/07/2008 19:53 DATA (L:) 512.0 KB
29/07/2008 19:55 DATA (L:) 512.0 KB
29/07/2008 19:58 DATA (L:) 512.0 KB
29/07/2008 20:00 DATA (L:) 512.0 KB
29/07/2008 20:03 DATA (L:) 512.0 KB
30/07/2008 09:15 DATA (L:) 512.0 KB
31/07/2008 09:47 DATA (L:) 47.4 GB
01/08/2008 09:15 DATA (L:) 512.0 KB
02/08/2008 09:15 DATA (L:) 322.5 MB
03/08/2008 09:14 DATA (L:) 3.3 GB
04/08/2008 09:14 DATA (L:) 8.3 GB
05/08/2008 09:14 DATA (L:) 512.0 KB
06/08/2008 09:14 DATA (L:) 9.3 MB
A base image is created on 27/07/2008 at 12:21. The size of this image is 45.4 GB, which reflects the size of the drive content allowing for standard compression. That same day at 18:30, defragmentation of that drive takes place. This is a weekly event. On this occasion an image, 3.5 GB in size is produced on the following day (28/07/2008 09:15). The day after that (29/07/2008 09:15) an image, 467.5 MB in size is produced in keeping with the addition/modification of 466 MB (488,683,550 bytes) of data. A further backup is taken the same day at 18:52 followed by another in quick succession to check the incremental function. The first image is 1.0 GB in size, which corresponds to the addition of 1.03 GB (1,109,618,264 bytes) of data. The succeeding image is 512.0 KB in size. Starting at 19:43 ten incremental backups are taken in rapid succession to see if a large incremental image is produced. All images are 512.0 KB in size. The following scheduled backup on the following day (30/07/2008 09:15) is also 512.0 KB in size.
On 31/07/2008 at 09:47 an incremental image of 47.4 GB is produced after a volume reconciliation (starting at 09:15) comparing the drive against the previous incremental backup (or, more accurately the drive state as recorded by the previous recovery point). During the time between the two backups a total of 593 MB (622,112,768 bytes) in 17 files is recorded as created/modified. The incremental image is in fact 2.0 GB larger than the base image taken on 27/07/2008 at 12:21. During this period a net amount of 2.06 GB (2,220,369,655 bytes) of data were added to the drive. The inference that can be drawn from this is that the incremental image size is that of an equivalent base image, viz. every allocated sector on the drive has changed between incremental backups.
Since the previous incremental backup, no defragmentation has taken place and no obvious disk activity that would signify a change to every sector of the disk has been observed. Considering that a base image for this drive takes about 30 minutes to produce, that amount of activity would be readily observed.
After this the image sizes reflect the amount of activity on the disk. There is no activity between 31/07/2008 09:47 and 01/08/2008 09:15, so an image of 512.0 KB in size is produced. On 02/08/2008 at 09:15 an image of 322.5 MB in size is produced reflecting the addition of 323 MB (339,410,944 bytes) of data. There being no activity before the next backup an image of 512.0 KB in size is produced at that time (02/08/2008 09:14). The following backup image taken on 03/08/2008 at 09:14 is 3.3 GB in size due to the addition of 3.94 GB (4,238,016,512 bytes) of data.
Before the next scheduled incremental backup on 04/08/2008 at 09:14 weekly defragmentation takes place as scheduled. Two passes are made on this occasion to maximise the contiguous free space on the drive. This results in a backup image of 8.3 GB in size, within expectation. Following this there is no activity so an image of 512.0 KB in size is produced on 05/08/2008 at 09:14. The last incremental backup on 06/08/2008 at 09:14 is 9.3 MB, reflecting the addition/modification of 8.48 MB (8,896,512 bytes) of data and the removal of 6.62 MB (6,950,912 bytes) of data.
The advice about defragmenting only before a base image need not always apply as can be observed here. Regular defragmenting based on the amount of drive activity yields incremental images that are consistently smaller than the base image for the drive. It should also be pointed out that this drive contains data only; there are no applications or infrastructure supporting such applications (such as temporary file storage, etc.) located on this drive. Further, it is neither the target of any application updates, peer-to-peer services, internet feed nor writeable file sharing. Further, regular scanning shows it to be malware-free. It neither contains a page file nor supports any kind of auto-logging or indexing function. In short it is reserved for a specific set of data only, there being other drives in the system dedicated to the support of applications, browsers, etc.
In conclusion each image size can be accounted for except the one on 31/07/2008 at 09:47. So, my question is, what does Volume Reconciliation achieve? If it is restoring Ghost’s ability to track changes, why does it not implement the result of that restoration immediately?
Robert, it appears that you have found a bug. The incremental should not be the same size as a base. I will have some one contact you.
Oh, and to answer your questions:
1) "what does Volume Reconciliation achieve?" It figures out what has changed since the next backup so that when the next incremental is taken we only backup that which is changed.
2) "If it is restoring Ghost's ability to track changes, why does it not implement the result of that restoration immediately?" Because we only take backup when we are told to either by user action or by the schdeule or, if defined, by the Threatcon level.
Message Edited by Johan_Jeffery on 08-06-2008 02:22 PM
Yaay! (and there was much rejoicing)
Don't hold your breath. Look how long it took someone from Symantec to even notice this forum and recognise that this problem exists...bearing in mind I opened a case with them re: this issue (plus a couple of others) as soon as version 8 was released.
They d1ck you around for as long as they can with the phonecalls stalling until the next version *.* is released, hoping that it will just go away with it's aplication. Well it hasn't so far. They should have left this product with PowerQuest who actually knew what they were doing.
I would expect this issue to get plenty of attention in development. But I can not promise when a fix would appear in the product. Part of how quickly a fix can be done will be determine by how quickly we can got to the root cause of the problem. And as you can see this is not all that easy to reproduce. So it may take a bit of time.
As to your remark about PowerQuest. Ghost is being done by the same team that was PowerQuest and I'm grateful to be part of the team. Unfortunately we have had issues in getting information passed all the way up the line to development. We are working to solve those kinds of problems. That is why I am now involved with this forum.
We would like your experience with Ghost and all of those associated with it or any other Symantec product for that matter to be a good one. Please keep the information coming. We also appreciate your willingness to help others on this forum so that they also benefit from using Ghost.
I have shared these experiences with Symantec and have gotten nowhere, just being fobbed off with pathetic suggestions like performing defrags and backing up drives individually which defeats the object of having the simple solution this product purports to be. It's not difficult to reproduce at all. I have at least 5 servers with BESR which exhibit this issue and desktops with BESR DE and ghost also yadda yadda yadda. I've sent numerous SEAT reports and logs and tried untold suggested "workarounds" and in the end, hung up on the chap "trying" to help (after many an hour spent over numerous calls) as he was less proficient with the product than we were. Symantec need to stop letting the customers be the guinea pigs and do some realistic testing before releasing a product. This problem never occured in BESR 7 and previous or in the earlier versions of Ghost. The first question on Symantec's mind should be "right chaps so what have we changed in this release that could be causing this problem?"
[edit: removed one word per the Participation Guidelines and Terms of Service.]
Message Edited by Tim_Lopez on 11-07-2008 06:11 PM
Johan_Jeffery wrote:"as you can see this is not all that easy to reproduce."
It seems to happen quite often to quite a few of us....
Thank you StuartV.
StuartV wrote:It seems to happen quite often to quite a few of us....
That is unfortunate, but good. We have not seen this in house, but once after the product shipped. The easier it is to reproduce the quicker we can get it fixed. Do you have a recovery point set that you would be willing to let our developers look at so that they can figure out what happened?
Quickly?!?! V8 came out around March last year. Anyway, what's your email address.....and mailbox limit...I'll send you a backup set.
Correction…this year.
I have mine set to backup every night, do a base every 3rd night, and only keep 2 copies. I'm not sure I've had this problem in the last 6 days. But, if I have one, I'm willing to share it. It would be something like 300 GB, though, I think. That's one base at around 150'ish, and then an incremental that is just as large.
I'm not sure how I would get that to you. I don't foresee uploading something that big over my DSL connection.
StuartV, e-charge, I’ll have someone contact you and make the arrangements to get the recovery point sets to us. Thanks, for your willingess to help!
Johan_Jeffery wrote:Robert, it appears that you have found a bug. The incremental should not be the same size as a base. I will have some one contact you.
Oh, and to answer your questions:
1) "what does Volume Reconciliation achieve?" It figures out what has changed since the next backup so that when the next incremental is taken we only backup that which is changed.
2) "If it is restoring Ghost's ability to track changes, why does it not implement the result of that restoration immediately?" Because we only take backup when we are told to either by user action or by the schdeule or, if defined, by the Threatcon level.
Message Edited by Johan_Jeffery on 08-06-2008 02:22 PM
Tony Weiss has contacted me and I have forwarded the information he requested.
I should have phrased my questions better; in fact they are but one, viz.
Why is volume reconciliation finding that every sector on the drive has changed?
I now have the answer to that question.
Would you like to share that answer with us please Robertj?
Answer: It’s a bug.
Johan_Jeffery wrote:I would expect this issue to get plenty of attention in development... Please keep the information coming...
Perhaps the following is a clue. The problem has occurred again. Since I posted the former history the following backups have been taken for the DATA drive (L:\).
Date Created Contents Size
07/08/2008 09:14 DATA (L:\) 323.8 MB
08/08/2008 09:14 DATA (L:\) 4.3 MB
09/08/2008 09:14 DATA (L:\) 768.0 KB
10/08/2008 09:14 DATA (L:\) 768.0 KB
11/08/2008 09:51 DATA (L:\) 47.3 GB (base size)
Another incremental of baseline size was generated on 11/08/2008 at 09:51. The same thing happened for my other data drive. Both drives are IDE (PATA). However, my backup of C: (SATA drive) recorded activity from the weekly defragmentation (on 10/08/2008 at 18:30), but no large incremental.
Further, I have scheduled backup of another SATA drive (since 01/08/2008) to see if it also shows this behaviour. It does not. The curious thing then is that the problem only occurs on my IDE (PATA) drives connected via a RAID controller. The drives connected via a SATA controller do not show this problem.
I should also add that volume reconciliation only occurred for the IDE (PATA) drives. It was not observed during the backup of the SATA drives.
Perhaps the two things, baseline-sized incremental image and volume reconciliation are related and then only on IDE (PATA) drives. Over to you, Symantec.
Message Edited by robertj on 08-11-2008 01:33 AM
A large incremental kicking off definitely follows an extensive "reconciling volume data" at the beginning of a job. The only way to work around this is to start a new backup set. This occurs on SATA drives too.....as well as SAS connected drives.
Just another tidbit of info.
[edit: removed profanity per the Participation Guidelines and Terms of Service.]
Message Edited by Tim_Lopez on 11-07-2008 06:13 PM