Not what you are looking for? Ask the experts!
NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
In the course of testing the other v12.6 bug I just reported (system freeze due to scanning a BootCamp partition with Compressed Files ON), I noted another problem.
Evidently the new v12.6 scan engine is creating a baziillion small files -- it appears it is creating one for each file it scans. Presumably this is in aid of speeding up scans -- i.e, the engine can detect a file has been previously scanned and has not changed since.
The PROBLEM is that these files are created in such a way that Time Machine is now obliged to back them up!
So after a System Scan or an Idle Scan, the next Time Machine incremental backup takes *FOREVER*, even though the Mac has been idle and so there's not really any significant amount of "real" file changes needing backup.
I believe this is new behavior in v12.6.
This is on a MacBook Pro 17" (mid 2010), running OS X 10.8.5, fully updated, with 4GB memory.
--Bob
Replies
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
OK, this one just gets more bizarre the more I look at it.
Leave the Mac idle except for Activity Monitor and watch what happens.
As mentioned, running a scan creates a HUGE number of small files (one new file per each file scanned). The next time Time Machine fires up, it tries to back up all of those. That takes a LONG time. It has to determine that they are new, prep space to copy them, and copy them. That's what I said above. The actual amount of file data transfered is small, because the files are all small, but the directory operations involved are substantial.
But wait! There's more!
Evidently Idle Scan does not consider Time Machine's running to be something that counts as making the Mac no longer "idle".
And so, while Time Machine is furiously churning, trying to prep for such a huge backup, Idle Scan decides to fire up!
Now it's not just that you've now got two heavy consumers of disk traffic running. Nope, it's worse. Because Idle Scan is creating more files! Which means when THIS backup finishes, Time Machine will discover a whole bunch of new files were created while it was running and so it will back THEM up before completing what it considers its finished, snapshot backup!
I know that the NAV scanner is running because I can see Symdaemon fire up and consume a lot of CPU time. The disk traffic also shoots up. I know that this running of Symdaemon is for Idle Scan (as opposed to something triggered by the file operations of Time Machine), because if I do somehing like start up a small app and Quit out of it Symdaemon shuts down again right away -- having determined the system is no longer "idle" -- leaving just Time Machine's "backupd" process chruning away.
This is nasty.
Basically it means if you want to let Time Machine function in its automatic, normal manner, you'd best turn off Idle Scan in NAV Mac v12.6.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Oct 26, 2013
Time Machine is taking hours and hours to complete since Oct 16 when NAV v12.6 was installed thru Live Update.
I have NAV v12.6 installed on two Mac Book Pro's running Mavericks 10.9 build 13A603.
I decided to uninstall NAV v12.6 from one MBP. Time Machine has been working as expected since then doing backups every
hour taking only minutes instead of several hours!
I hope Norton fixes this soon!
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Sorry for the problems. Do any of you have the name/path of the file(s) that are being created? We don't see this in-house but it might have to do with archive (compression) scanning. If anybody can give the full path to the file(s) that are created, and if you can send me a couple of those files via email, that'll help us a lot.
Send me a private message if you can help us out.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Unfortunately, no. They are apparently hidden files (or embedded in file bundles or directory entry data) that are still created in a way that needs backup.
To see this problem is simple. Again, this is on OS X 10.8.5:
1) Quit out of all apps and double-check in Applications > Utilities > Activity Monitor to make sure no background tasks are chugging away (e.g., Spotlight indexing).
2) Do a Time Machine backup.
3) When that finishes, immediately do another Time Machine backup. Since nothing much has happened since the one just completed, this backup will be tiny.
4) Launch NAV Mac v12.6 and start a System Scan including Compressed files for the MAC hard drive.
5) Let that run for a while, keeping an eye on the counter of files scanned. When it reaches a suitable number -- say 50,000 files scanned -- cancel out of that System Scan.
6) Now do a 3rd Time Machine backup. Note that it transfers quite a large amount of data. And takes much MUCH longer than the 2nd backup above.
7) For this next step you will need to be logged into a Mac account that has Administrator privileges: Launch Applications > Utilities > Console. Select to view All Messages. Enter "backupd" as the Search string.
This will show the messages from the backupd process which reports on Time Machine activity. Note the info for the last 2 of the 3 Time Machine backups you just did. Backup #2 will have transfered almost no files and almost no data. Backup #3 will have transfered over 50,000 files (in this example). Divide the number of files into the total data amount backed up and you will find the average file size to be around 100 bytes). Since the only thing running in between #2 and #3 is the NAV Mac v12.6 System Scan, that's the culprit.
If you do the same thing again, while varying the amount of files scanned before you cancel out of the System Scan, the workload of files Time Machine has to back up after the System Scan *ALWAYS* tracks the number of files scanned.
If you do this same test doing a File Scan of a single folder with a modest number of very large files in it, you will see that the number of files backed up afterwards is still, always on the order of this small number of files scanned. And the total data transferred in the backup remains small. That proves the "real" files -- the large ones in that folder you just scanned -- are not what's being backed up. Some smaller file ASSOCIATED with each real file is being modified or touched in a way that causes it to be backed up.
Again, this is new behavior in NAV Mac v12.6.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Thanks for the information. We'll try this in-house, but unfortunately what Time Machine considers "changed" is sometimes not what you might expect, so we might get wildly different results.
In the mean time, can you please try running this command while doing the 3rd Time Machine backup. Open the Terminal applicaiton (Applicaitons -> Utilities) and type/copy & paste this command:
"sudo fs_usage -w backupd | grep nav"
Let it run for a while, hopefully generating some output. This will tell us what backupd thinks has changed and is backing up, and which of those files contain 'nav'. All of our temporary files created during scanning begin with this prefix.
If that does not generate any output, try again but instead with the string:
"sudo fs_usage -w backupd | grep Symantec"
Thanks,
Ryan
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Ryan,
I just sent you a PM. I've got some results files for you and just need an email address to use to send them to you.
Looks like the "Symantec" search string may have picked up some useful results.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Ryan
The terminal command -
sudo fs_usage -w backupd | grep nav
has greatly reducded the Time Machine backup!
10/31/13 12:32:32.338 PM com.apple.backupd[992]: Found 14135 files (2.09 GB) needing backup
10/31/13 12:36:26.895 PM com.apple.backupd[992]: Copied 14618 items (13.9 MB) from volume Mavericks. Linked 47011.
10/31/13 12:36:58.044 PM com.apple.backupd[992]: Backup completed successfully.
Note- I have excluded scanning of Time Machine (Backups.backupdb) and unchecked "enable compressed file scanning"
Will run another quick scan of my Mac Book Pro and try another Time Machine backup.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I had a chance to check out a recent iMAC -- also running 10.8.5 -- and I'm seeing the same nasty behavior.
I strongly recommend people disable "Idle Time Scan" in Norton Antivirus for Mac until Symantec fixes this.
Also, don't run a System Scan unless you are OK with setting aside time for the next Time Machine backup to take a *LONG* time. Think hours.
I heard back from Ryan that they think they are on the track of what's going on here. So hopefully a fix will be forthcoming soon.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I've had the same problem on my late 2011 MBP running OSX 10.9. I've removed NAV and installed iAntivirus, even though the virus definitions for it are old until I see the issue(s) have been fixed. It would be nice it Norton would give us a quick reply here when it is.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Thanks for the feedback. I have 2 MacBookPros one has NAV v12.6 and I deleted NAV from the other.
Sometimes a simple Time Machine backup is taking hours.
I hope a update/fix will be coming soon.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
If you disable Idle-Time Scan in the NAV v12.6 preferences, and don't run a System Scan unless you really need to, your Time Machine backups should go back to normal. That is, you should have no more than one last, excessively long, Time Machine backup to suffer through -- to start with, now, and then again each time you bite the bullet and do a System Scan.
I.e., you can leave traditional, live Virus Protection and Vulnerability Protection active and not run into this problem.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Thanks for the tip Bob!
I will try it (disable idle Time Scan in NAV v12.6 preferences and not doing a System Scan)
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I haven't noticed the "bazillion" files (haven't looked), but have had a similar experience. With Idle Scan enabled, Spotlight indexing never finishes. Since TimeMachine is slaved off the spotlight index, TimeMachine also never finishes. Reboting in safe mode (Command S) quickly gets a TimeMachine backup and spotlight index done. Rebooting again in normal mode gets me right back to TimeMachine and spotlight never finishing and the whole computer being sluggish.
I've left it run (not sleeping) for over a week, and still didn't finish. Tried to flush the spotlight indexes (mdutil -Ea), and left it for another week. Didn't finish indexing again.
Turning off Norton idle time scan gets spotlight index done quickly and time machine done quickly.
There's a similar thread over at the Apple support discussions... Perhaps they are related?
https://discussions.apple.com/message/23752824#23752824
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I last stated prior, I was using iAntivirus as a replacement for NAV, well I wasn't happy with the lack of updated virus definitions so I thought I would try something else. I loaded up Avast. It's free and it has updated definitions. First I've used Norton for years/decades and I had something pop up on my radar a few weeks ago. My wife brought home an external usb drive from work for me to copy onto her new laptop. The new laptop came with Webroot so I thought I would give it a try. She failed to inform me that she had a couple of viruses on her work laptop that she couldn't not get rid of. She was using Mcafee. Well Webroot found the files that contained the viruses. I thought "hey, wait a minute" I was in bootcamp that has NAV and it didn't find anything! Thats when I updated to NAV 12 on OSX and bootcamp and scanned again. NAV 12 still did not find the viruses. I scanned with Avast and found two more that NAV didn't. I don't remember what Webroot removed but Avast remove Win32:Klez-H (Wrm) and Win32:Malware-gen. Just thought I'd let you know.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I'm pretty sure the issues are related. Idle Scan apparently doesn't consider the machine "in use" just because Spotlight (or Time Machine) is running. So Idle Scan fires up. The bug that's causing Idle Scan and System Scan to create files or directory items or "touch" files in a way that forces Time Machine to back up about 100 new bytes per file "touched", likely *ALSO* creates work for Spotlight to do.
So Spotlight is running because an Idle Scan created a lot of changes Spotlight thinks it needs to index, and then WHILE Spotlight is running Idle Scan fires up AGAIN and creates still more changes.
I could easily envision a scenario where Spotlight can't catch up. Particularly since both Spotlight and Idle Scan are file system intensive apps.
--Bob
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
...and that would be?
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Thanks Bob!
Kudos for you. Your tips have worked for me thus far. My TimeMachine backups are back to "normal"
I have checked the backup times for the last several days. Time is good even after a NAV system scan with the 'idle-scan" unchecked.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Thank you for the insight which has solved a major headache for me! I can confirm that this idle-time scan issue was causing three problems on all my computers: 1) the slow backups to Time Capsule with endless preparing stages, 2) the endless and slow re-indexing by Spotlight that would take all day to complete and resume the next day, and 3) Crashplan endlessly scanning and backing up, without reaching an endpoint. Within minutes of turning off the idle time scan, Time Capsule returned to normal preparation and back-up speeds, Spotlight stopped indexing, and Crashplan finally caught up. Those tens of thousands of tiny files were overwhelming each of those three processes.
And now we just need a fix to that idle time scanning so that it can be turned back on!
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
I can confirm this issue for me as well.
My Time Machine slowed down to a CRAWL. I attempted everything according to the message boards, resetting, repairing, rebooting, re-indexing, etc., etc., etc.
Then I came across a post on the Apple boards referring to this thread.
I can say that while disabling Idle Time Scan did nothing to resolve for me, fully uninstalling Norton Internet Security completely and immediately fixed the issue.
Time Machine has no problem backing up as it always has.
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
Re: NAV Mac v12.6 creating a Baziilion small files WHICH THEN HAVE TO GET BACKED UP!
jjmbp15,
AFTER you Disable Idle-Time Scan, there will be one more, long Time Machine backup to suffer through.
Subsequent Time Machine backups should be "normal" speed, unless you initiate a System Scan or turn Idle-Time Scan back on.
At least, that's what I'm seeing.
--Bob
Replies are locked for this thread.