Ghost 14 incremental backups look like full backups

Archive
1

I have been having a persistent problem with Ghost which developed and I cannot figure out why it is happening.

 

I run a P4 3 GHz using XP Home with SP3 I think. 3 GB RAM. 250 gb hard drive (now) divided into 2 partitions, C and D. The C partition is about 90 gb and D is about 144 gb. It is an old system, at least several years old. The backup drive is a FreeAgent Segate 250 gb external via USB 2.0.

 

I did backups bi-weekly (Sunday and Thursday) with a new full backup which would occur when the month changed. Naturally, a 250 gb (I set threshold at 160 gb) is not going to hold many full backups which tend to be about 60 gb). Still this seemed to work satisfactory for quite a few months though I did have an occasional problem. It was then that I learned Ghost is sector based backup so I specifically refrained from defragging the D drive which had like 60 gb of files while the C had like 15 gb which I continued to defrag almost every time I turned on the computer.

 

The problem that exists or developed is that Ghost now seems to make a full backup each time. I had to drop the Thursday backups to try and keep from overrunning the backup drive. It didn't seem to help. Ghost continued to make what seemed like [unnecessary] full backups when I took pains to not do things like defrag the D partition.

 

Info on D partition. Ok it is a game drive. Having said that, I have for all intents and purposes only one game that I play regularly (like daily). That game represents about 12 gb of files. A goodly portion of the rest of the occupied space is 3 online games that I almost never play and much of the rest are video files that I may view from time to time but don't move around. If I played all these games, I wouldn't be complaining for I would figure that changing game files were the problem but as far as I can tell, that is not the case. I have Eudora installed on D and when I check email it changes a half a dozen files. This is the only other thing that may change regularly.

 

The last thing, to date, that I have tried is to remove the backup schedule and redo it via the wizard. I had deleted the prior backup set because it was not going to fit another 60 gb into a 40 gb space which was all that was left. The full backup went as expected, but this Sunday the expected incremental backup after 2 hours was still 50+ minutes from completion at 64% and in case one wonders, yes it was saving a heck of a lot, many and many gbs.

 

I'd like to keep a backup. The idea of being able to restore the system is a comfort to me, but this insistence of Ghost to do seemingly full backups instead of incrementals is disturbing me. I expect that my 250 is too small but I'd like to know why this is happening.

 

Is this happening an artifact of XP Home? NTFS? What? If I understood why it was happening I wouldn't be quite so I suppose upset with Ghost.

 

It would also be nice if when checking (comparing the disk to the previous sector image file) that it would give an indication of where it was in doing the checking. 10%, 30% etc. not be stuck at 1% until it was done.

 

An otherwise decent piece of software.

2

RIwasa,

 

Does Ghost create any .iv2i files now or only .v2i?

 

Which defrag app are you using? I note you aren't defragging D: in between backups. But what about C: drive defrags?

 

Which antivirus app are you using?

 

Do you have WinXP System Restore turned OFF?

 

 

3

Hi RIwasa.

 

A couple of added questions. What is your full version of Ghost from Help > About? The question Brian asked about whether Ghost created a .iv2i file or .v2i is very important.

 

In between running the full and incremental backups, is the computer shut down in between those times?

 

If the computer is shut down in between these times take a look in the ROOT of each drive and look for a file called vsnap.idx. See if this file has a time stamp at near the time of last shut down or is it from some earlier time? NOTE that this is a hidden file by default so make sure Windows Explorer is configured to display hidden files.

 

Thanks

Allen

4

As far as I can tell, Ghost has been creating the .v2i for base and .iv2i for the supposed incremental backup. I took a look into one of the iv2i files and to all appearances it looked like the base file (this was when I had a base plus 2 incrementals which in total had exceeded my threshold of 160 gb).

 

I use diskeeper lite which is a separate program that came with the system when I bought it. I have also used a shareware/freeware defraggler but only on C: since diskeeper lite does not eliminate some fragmentation which is often a fragmented empty file in some symantec temp directory that is actually does not show up on directory listing.

 

My usual procedure is to use diskeeper lite on C: then if there are still fragments left, which by experience lite can't get rid of, I use defraggler which does the trick. If and when I defrag D:, which I am considering doing since this Sunday will be a new month, I use lite. It runs fast enough and pretty much does what I want. Because the backup drive does get heavily fragmented after a large backup, I do use lite on it which takes quite a while.

 

I have norton internet security version 16.0.0.125.

 

XP system restore is on and monitoring all drives C, D, and G (free agent external USB drive).

 

 

5

The full version number of Ghost is 14.0.5.34587

 

Ghost does seem to produce  v2i and iv2i backup files.

 

The computer is my home computer so it gets shut down in between the full and incremental backups unless I have a good reason to let it run.  I do not leave it running, except for special circumstances, when I leave to go to work.

 

The backup schedule was Sunday and Thursdays with a full backup on the scheduled day after the month changed with incrementals thereafter until the next month where the cycle starts again.

 

Ok, I'll take a look. I had a MS update to IE 8 just a while ago which required a reboot. I'll also do a cold boot and look again. I'll post what I find.

 

Thanks Allen and Brian.

 

6

RIwasa,

 

So Ghost is creating files correctly but the .iv2i are too large. Could you try this for 2 weeks? Defrag your partitions and then uninstall both defrag apps. Create your base Ghost image. Create incrementals twice weekly as you are doing. Don't install any new software or do any software updates. Report back in two weeks.

 

By trial and error I've found Diskeeper (even without defragging) and Norton Antivirus lead to larger incrementals. Long story.

7

Checked VSNAP.IDX

As requested.

 

C Drive

 

Created:     November 10, 2008 1:15:53 AM

Modified:    March 27, 2010 1:13:50 AM

Accessed: Novmber 10, 2008 1:15:53 AM

 

D Drive

 

Created      November 10, 2008 1:15:52 AM

Modified:     March 30, 2010 9:44:13 PM

Accessed:  November 10, 2008 1:15:52 AM

 

Shutdown via XP, turn off power main switch on power center, waited 10 seconds then powered up.

 

C Drive

 

No change

 

D Drive

 

(only change)

Modified:     March 30, 201010:42:02 PM

 

 

8

I understand what you're asking. Uninstalling defraggler is not a problem though I am not sure at the moment whether I have the diskeeper lite install file incase I want to reinstall it anyway (even if its absence solves the problem).

 

I don't  plan on installing any new software though the game I play (lord of the rings) does occasionally update its game files which happen to be on the D drive.

 

I'll have to think about it a bit. The C drive gets really fragged up seemingly pretty quickly. I have also had lite on the system forever. I was forced (iirc) to reinstall NIS because of some problem in the code updates from the server and the only way to get it to work again was to uninstall and rerun the install program. I don't think or recall if the ghost problem came after that time.

 

I have until April 1 (Thursday) to do all this. The usual schedule was Thursday and Sunday. That would normally the be start of a new backup set. I'll probably have to delete the March set just prior to doing the April backup to ensure there is enough space.

 

I will check back in a few days to see if there are any clues in the vsnap.idx file dates and times I posted.

 

Thanks for the help.

 

9

You will get differing opinions on this but I don't think most people would notice a subjective performance benefit out of only 6 monthly defragging. We've done single blind tests on people who swore by defragging benefits and their ability to tell if a defrag had been performed was as good as tossing a coin.

 

Defragging is death to incremental imaging.

10

Hi Rlwasa,

 

Along with the other things Bryan asked you to try, please try the following.

 

Run a base backup, followed by an incremental right afterwards just to establish that this works correctly and that the incremental is of expected (very small) size.

 

Then restart and/or do a full shut down of the computer and bring it back up. Once up, do another incremental backup. Is the incremental of correct size or do you have the problem with large incremental? If the first one works properly then try a second and perhaps even a third time.

 

The idea on the second test is of course to run a tightly controlled test where you know that nothing has been explicitly changed (by things like defrag, windows updates, etc, etc) in between the base and the incremental.

 

Thanks

Allen

11

I'll be preparing to try Brian's suggested test, though I'll be looking at whether I can just prevent Diskeeper Lite from being loaded at all (and not run or allow it to run) and still do the backups as he suggested. I recall a system utility that lets me decide which things get loaded which I hope will suffice to keep Diskeeper Lite's files from being loaded. Since it is probably a part of the registry, that might not work but I hope to avoid uninstalling it, if I can. I realize I may have to bite the bullet anyway and uninstall it. I might try setting incremental backups for successive nights which if they are small should be within the capacity of my backup external. If they stay small I won't get upset and if they start to balloon up again, it might happen quickly and I can report back otherwise I won't have a clue if I use my normal Sunday/Thursday schedule until Sunday for the first time (not withstanding your suggestion).

 

 Since I expect to make a full backup tomorrow (April 1) evening, I'll also try your suggested test as well. I'll be having to completely delete the previous backup set (lack of space) prior to the start of the new backup set.

 

Question: I can just tell Ghost to Run Backup Now (i.e. from the system tray icon) when I want it to do the incrementals?

 

BTW, the vsnap.idx on C still is dated March 27 while on D it is dated March 31. I am curious, if vsnap.idx is for Ghost, what date should be reflected assuming literaly no changes on the drive. The date of the last backup?

 

Thanks.

12

I have done as Brian requested and uninstalled Diskeeper Lite and Defraggler. I had defragged everything, deleted the backup set (actually only one since the "incremental" for that set looked to be as big as the base).

 

I set up a backup job that started April 1 at 12:45 AM and scheduled incremental backups on Friday (2d), Saturday and Sunday to start. I will add in Monday, Tuesday, and Wednesday later.

 

As requested by Allen, after the base backup under this new schedule, I ordered manual incremental backup shortly after the base finished. The base consisted of a 7.5 gb and 58.5 gb backup as reported by Ghost (C and D drive) and the manual incremental consisted of a 30.3 mb and 512 kb backup (C and D). I then powered down, then powered back up and ran another manual incremental - 24 mb and 512 kb backups. It appears that Ghost is operating as intended big base backup and much smaller incremental backups. The incrementals also were very fast, a few minutes as opposed to the near one hour duration of the base backup.

 

I would note that the VSNAP.IDX file on C showed a changed modified date after the shutdown and restart going from March 27 to April 1. On D, vsnap.idx showed April 1 as well but then its modified date changed with each shutdown before while the C drive remained at March 27 until this time around as noted.

 

It appears that Ghost is behaving itself producing small incremental backups. This looks promising. I guess I'll mourn Diskeeper Lite if it turns out I can't ever reinstall it.

 

Thanks, will post again after a few days of backups.

 

13

Hi Rlwasa,

 

Thanks for the update and we'll look forward to further results.

 

Allen

14

Recap: Issue was Ghost kept making full backups, even on what should be incrementals, which consumed a lot of time and backup space. The reconciliation was prolonged before starting to save (probably working as intended there).  The backup drive was not big enough to accommodate full backups multiple times in a month (new set at start of month) especially when allowable threshold was set at about 70% of capacity.

 

As requested, uninstalled Diskeeper Lite and other disk defragger (defraggler) and scheduled backups as a test that Diskeeper was forcing Ghost to do full backup due to some interference with a file Ghost references to otherwise do incremental. (not stated but seems to be the case)

 

Backups scheduled for 12:45 AM every night starting 4/1 and every night thereafter for the test.

 

Results:

 

Date                 C Drive                   D Drive

 

4/1       Base        7.5 GB                   58.5 GB

4/1       man 1      30.3 MB                  512.0 KB

4/1       man 2      24.0 MB                  512.0 KB

4/2       Inc       154.8 MB                  768.0 KB

4/3       Inc       230.8 MB                    3.0 MB

4/4       Inc       588.5 MB                    2.0 MB

4/5       Inc       180.8 MB                  768.0 KB

4/6       Inc       219.0 MB                  768.0 KB

4/7       Inc       176.8 MB                  768.0 KB

 

Base     =   Start of Recovery Set, Full backup

man 1   =   manual incremental after Base (requested by Allen)

man 2   =   manual incremental after cold boot (requested by Allen)

Inc         =   Scheduled incremental 12:45 AM every night

 

Backup drive - Seagate FreeAgent 250 gb, USB

Comp drive - 250 gb EIDE

Partitions (C 88 GB 15 used, D 144 GB, 64 GB used)

 

It would seem that uninstalling Diskeeper Lite which installed DKservices on bootup made the difference.

 

The incrementals were fast and took only 5 minutes at most, usually much less than that. This is much, much better than the usually two hours to do the "incremental" backups from before. I am thinking of setting Ghost to start a new set every week and see how this all works for another week or two - there should be enough space for 2 backups sets then Ghost management should delete the older one prior to saving the third??. My drive does not change all that much and normally I had scheduled backups every Sunday and Thursday until this full backup incremental problem manifested itself.

 

This is looking good.

 

 Thanks to Brian and Allen for their suggestion.

 

 

15

RIwasa,

 

Interesting. Thanks for the feedback.

 

My problems with Diskeeper weren't as bad as yours. I used to defrag only before a Base image. But I'd watch the Diskeeper graphic daily and although the number of fragmented files would increase every day (as expected), every few days a large chunk of data (1 GB or so) would have moved from its previous position. And the following Incremental backup would be large, as expected. I changed to another defrag app and this doesn't happen.

 

We had several computers running Diskeeper and this data movement happened with each computer.

16

Just so  you know. I had my first problem though it was not directly what this thread was about.

 

I set Ghost to make a new full backup every Thursday at 12:45 AM which was the same time as the regular backups which were incrementals. The backup started as expected but I was unpleasantly surprised to get an orangeish?, I guess its a flag, saying that it couldn't make or complete the backup.

 

The Status window now shows a blank where the new backup was supposed to be and Ghost then asked if I wanted to start the missed backup which then, when I started it, went into reconciliation. I practically shouted NO and in anger because there wasn't an obvious way to abort (actually there was which I figured out afterwards) and just shut down the computer. I just turned the power OFF. Then booted it back up.

 

I changed the backup start time for the schedule to several minutes later so that it should start normally. Which it apparently did and started making the full backup instead of doing the reconciliation step like it tried to do which got me so mad.

 

I looked in the event log and found the following entries. This was for the initial 12:45 event which was supposed to be a full backup.

 

Info 6C8F1F65: The drive-based backup job, Drive Backup of  (C:\), New Volume (D:\), has been started automatically.

Then there were several (3 I think) entries that occurred several seconds later and went thus:

 

Info 6C8F17E7: An automatic recovery point was not created because an earlier recovery point from the same job was still in progress.
Details: Operation aborted

 

At this point, the computer had been running since about 6:44 PM, so about 6 hours. So you can see the computer had not been on for a very long time.

 

After the restart and adjusting the start time to 1:00 AM, the full backup, as noted, started and so far seems to be running normally. It looks like it will take about an hour. It is about half way through as I write this. The Status window is back to what it nominally should look like.

 

These events brought up something that irritates me. The error message does not tell me squat about what went wrong. It seemed like it could not access the backup drive but then again I can't swear to that and the event log gives me no useful info.

 

I don't know why this occurred. Perhaps since I had configured the schedule as daily but left the new recovery set schedule still on monthly then yesterday changed it to do a new recovery set every Thursday etc. that Ghost got confused. I had to use the custom button to get this change to a particular day.

 

I think the event log should log whatever error message that it send to me on the screen.

 

Anyway, the backup seems to be progressing ok and barring things like this event seems to be functioning fine.

 

I have a quick question. What procedure do I follow to delete old File and Folder backups? I have backups, which are generally only a few MBs going back to Nov 2008 and would like to purge them.

 

Thanks.

 

17

Hi Rlwasa,

 

I think the problem was due to overlapping schedules for incremental and the full backup. Ideally they won't both be schedule for the same day but if they can occur on the same day it is important to stagger the schedule for both.

 

To check this, go to the backup job and select Edit Settings and click Next three times. When you get to the Backup Time window, check down near the middle of the window where it says Start a new recovery point set. What is this set to? Based on what you provided I'm guessing it is set to Weekly.

 

Just to the right of this you should see a buttom which says Custom. Click on this and you'll see that you can set a completely different time for your full backup than you have for your incremental.

 

I would set this to at least say an hour or so after the time the incremental is scheduled for. Adjust this time as necessary depending on the worst case completion time for your incremental backup.

 

This will ensure that the two events can never become due at the same time and ensure that the incremental is completed before the base starts.

 

As far as deleting file and folder backup versions. Because of the cryptic filenames used by Ghost, this can only be done safely through the Ghost interface. You really want to have this automated to the extent possible. Check in Tools > Manage Backup Destination then click Settings.

 

The number shown on that very top entry tells Ghost how many file versions to keep before removing the oldest..

 

Side note: if you are doing file and folder backups on the same drive as you do image backups, it is completely redundant and unnecessary. The image backups have you completely covered for any drive in which you do them. It is actually easier also to restore individual files from an image backup than from a file and folder backup.

 

Hope this helps.

 

Allen

18

Allen,

 

You explanation makes a lot of sense. It never occurred to me that when a new recovery set was to be made setting would cause possible conflict with the incremental schedule. I'll have to keep that in mind when I stop this test schedule and go back to a montly new recovery set with incremental not quite as frequently. I can see where even my original monthly backup scheme could run into that because eventually the day and time (since the full and incrementals were set for the same time) for both would coincide on occasion (i.e. The first of the month would also be the day for an incremental, more or less).

 

That would explain why when the 1:00 AM backup was made why the log also indicated that an automatic recovery point was not created... blah blah at 1:00:10 AM.

 

That seems to set me straight.

 

One thing that occurred during the 1:00:00 base backup that I didn't expect, not that it was a disaster, was that when the 1:00:00 completed at 1:54:59, Ghost then proceeded to delete the previous recover set (including the incrementals) with the following log entry (only the first  and last entry pasted here):

 

Info 6C8F043E: Recovery point [FreeAgent Drive]\Norton Backups\WINDOWS-N6MYA4M_C_Drive001.v2i deleted to comply with the size of recovery points to save for each drive.
Details:

 

The next seven or so entries relate to deleting the incremental recovery points ending with:

 

Info 6C8F043E: Recovery point [FreeAgent Drive]\Norton Backups\WINDOWS-N6MYA4M_D_Drive001_i008.iv2i deleted to comply with the size of recovery points to save for each drive.
Details:

 

I had set Ghost to automatically manage the recovery sets, but had thought that this would not occur until the total size of backup sets began to seriously approach the 160 GB threshold I had set. While the previous set was running about 68 GB all together excepting the File and Folder saves which are about 9 or 10 GB, I was surprised that after the 66 GB base recovery set creation on 4/8 that Ghost would then delete the prior recovery set right then. I don't have a problem as such with what Ghost did but the fact that it did it now was unexpected. Am I missing some logic in how Ghost manages things?

 

I think the basic problem about the size of incremental backups has been reasonably solved (I will mourn the loss of diskeeper lite). Thanks.

 

Russell

 

 P.S. I just checked for I reread your comment on the file and folder versions. I had it set to 6 yet I may have done that after there were more than 6, possibly because of main recovery set problems. There are obviously way more than 6 file and folder sets. What would be the best way to delete manually or otherwise the excess file and folder saves?

 

 My original schedule for image backup was twice a week so I left the file and folder saves to continue every day, preceeding the image backup by plenty of time given how quick it went. I had not thought about the overkill at the time.

 

Thanks.

 

 

19

I believe that the basic problem has been solved. Incrementals on C tend to be in the 100-300 mb range which is understandable due to it being my OS/Program drive and usually much much smaller for the D drive (until the next game expansion I guess). Given a less rigorous backup schedule (say twice a week instead of every day), my poor 250 gb drive should be quite capable of handling the backups.

 

I have thought a bit about how Ghost deleted the old recovery set when it created the new one. I suspect that my backup drive has some system files in a couple places that break up the free space into a several pieces, albeit rather large pieces. Diskeeper Lite (previously) showed me what looked like system files in a couple of scattered locations in the free space. This might account for it doing what it did.

 

I do want to mention that it appears that if I had to just plain shutdown the computer (video was not coming up) by powering down, that it seemed to affect Ghost's ability to do a fast incremental backup. Namely, I ended up with the lengthy reconcilation then full backup. The next cycle it did the proper incremental backup. So it looks like a glitch due to operator actions. I also had not shutdown the computer between the time I had to reboot and the scheduled backup.

 

I think I'm satisfied that the problem has been resolved.

 

Now if only there was an easy way to purge the backup job log of all the now non-existent file and folder backups that I deleted off the backup drive. :smileyhappy:

 

Thanks for the help.

 

20

Just a short update.

 

New recovery set started May 1.  7.8G and 58.6G no surprise there.

 

Scheduled backups Monday and Friday at 12:15 AM.

 

The C drive has been alternating between 300-400 M (May 3, 10, 17) and 1.1-1.2 G (May 7, 14).

The D drive has been incrementalling in the 1.3 - 3.3 M range.

 

No disk defrag run.

 

Quite acceptable.