Why is Norton not participating in AV Comparatives testing?

---

moriseif wrote:

Hi friends and dear Tim

Why Norton does not participate in AV Comparatives institue testing?

---

It does ....

http://www.av-comparatives.org/images/stories/test/dyn/wpdt2011_2_en.pdf

Seems to be top of the list ....

Yep .AV-Comparatives has published the list of products will be tested during 2012.

The first surprise is the absence of Symantec Norton, who for reasons unknown will not be evaluated.  We expect some official explanation from Symantec.

---

Lexionline wrote:

Yep .AV-Comparatives has published the list of products will be tested during 2012.

The first surprise is the absence of Symantec Norton, who for reasons unknown will not be evaluated.  We expect some official explanation from Symantec.

---

AV-Comparatives have posted the following explanation in their forum:

http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=1060

Yes, It's pretty clear why AV-Comparatives are not testing Norton (because Symantec refused to participate in their testing).

What is not clear is why Symantec would do that.

I have been following AV-C for many years, and I believe they do a good job.

I used them as a basis for using Norton, which usually did fairly well in their tests.

de:
> just why is it that 'we' should be able to expect some 'official' explanation
> of why Symantec/Norton makes any corporate decision?

That was not a helpful response.  It was dismissive of the OP's issue.

Symantec _competes_ in the marketplace with other manufacturers for anti-virus product sales.  Anyone competing in anything should be willing to be tested against their competition. 

Or at least give a reason why not.

Now I'll take Symantec's side.  If there is a legitimate reason not to participate in the test -- and I'll say right here, right now that I am saying that's a possibility -- then Symantec has multiple blogs on their website that they can spell out their reasons.  Such a post would educate all of us.  Not just leave us in the dark.

When I look at the results of some test (not just anti-virus) I sometimes wonder if it was done for the purpose of promoting one of the products in the test.  If you follow these kinds of things long enough, you'll know what I'm talking about.  So the "testers" not being objective is one potential problem.  Or not being technically competent.  Or whatever.

But, generally speaking, one of the most common reasons for not participating in a test is concern that the results will not be favorable.  I want to be clear that I am not saying that specifically about Symantec.  But in general.

Symantec should either participate in the test or give us valid technical reasons why not. 

If they want our money in the future.

b:
> The reason is posted on AV-Comparatives forum.
> It's pretty clear.

False.  No _reason_ is given whatsoever.

Only a statement -- and it's by AV-C, not by Symantec -- that Symantec didn't want to do the File Detection Test.

AV-C points out the importance of the FDT.  A product could do well in blocking malware from websites that are visited by the user's browser, but if the malware is already on the computer _by any other means_, the FDT would be the critical test.

All,

Just wanted to let you know that we're looking in to finding the best person to respond to your concerns regarding this. I don't have the answers to your questions, but we're finding someone who does. 

Quads = the Norton real world tester on threats  

Quads

Real world, whole product testing is fine. However, if malware already exists on a computer, on-demand testing is required.

Symantec performed poorly on the last on-demand testing by AV-Comparatives. Instead of pulling out of AV-Comparatives, strengthen your on-demand scanning!!

---

FrankF wrote: but how come that Norton takes part in Virusbulletin e

---

They don't seem to take part in that either anymore.

http://www.virusbtn.com/vb100/latest_comparative/index

AV-Comparatives does carry out a Whole Product Dynamic "Real World" Protection Test and Norton was a shining example in the November/December publication. So even if it is felt that an individual test doesn't reflect the efficacy of the product, there was still the Whole Product result to show how good it is overall. It's a pity Norton won't have a place in the graph this year.

As the threat landscape has changed, our protection and repair technologies have evolved and multiplied to protect against many new threat vectors.  As a result, there is a need for a broader and more comprehensive set of tests, often called “whole product” or “real-world” tests, to measure this increased coverage and emulate the consumer experience.  Additionally Symantec has also strongly supports remediation or “clean-up” tests where a computer is infected then a security product is tested to see how well it can remove the infection from the a computer.  

Question for Tony Weiss -

Could you then tell us if there are any indepentent and reputable AV testing companies that will be testing Norton in 2012.

I really like Norton AV and have been using it for many years, but I would like some independent assurance that it is stil a leading product.

---

Tony_Weiss wrote:

As I wrote before, we are continuing to seek out and participate in a range of tests that meet these standards to measure our products’ efficacy and performance.

---

Tony, if I could make a suggestion...

I understand Symantec's objections to the AV Comparatives testing methodology.  Symantec has been shown in a poor light in the past - for things that are a consequence of poor testing procedures that highlight the ability to recognize "ancient" viruses - at the expense of both efficiency and the ability to keep slower systems safe in the real world.

If Symantec would release info on why they are refusing to participate in the AV Comparatives testing - and state that they are willing to consider rejoining the collective when those deficiencies in the testing methodology are rectified - this would go a long way towards pointing the finger where it belongs.

The problem is not AV Comparatives.  The problem is poor testing methodology.  This is an issue any Scientist understands completely.  It is common for Scientific Peer Review to analyze testing methodology when performing scientific experiments - and comment on the errors produced when that methodology is flawed.  Good Science recognizes that situation and corrects the experimental errors introduced because of poor testing methodology.

If AV Comparatives then refuses to consider altering their testing methodology so the tests accurately reflect what is needed now - not 5 years ago - we the public know where the problem lies.

If AV Comparatives alters their testing methodology so that the ability to detect "ancient" viruses is a separate item and is noted as such - along with test results both with and without Smart Definitions - I think this would go a long way towards allaying any fears from Symantec Marketing that the AV Comparatives results show Symantec in a poor light "by design".

And if AV Comparatives does nothing about this - Symantec have full recourse to saying *this* is the reason they decline to participate in AV Comparatives testing.

Call a spade a @!*##+& shovel.  The world needs more clarity - not obfuscation.

Thanks

---

Bombastus wrote:

Here is a new "real world" test, performed by AV-Test:

http://www.norton.com/avtestN360v6

Commissioned by Symantec, but still, AV-Test is considered reliable, and is the biggest independent test organization.

---

Thanks for the link.

Very interesting report but I wonder why they chose WIndows XP in January 2012 ?

Hmmm - how out of date are these people at AV-Test ?

Windows XP and Firefox 2.0 !! (Firefox is now 10.0)

---

Davec33 wrote:

Hmmm - how out of date are these people at AV-Test ?

Windows XP and Firefox 2.0 !! (Firefox is now 10.0)

---

It is intentional.  From the report's explanation of the methodology used:  "Additionally, the following applications have been installed to provide a 'vulnerable' system for the URLs that use exploits to infect the system."  The table then lists Firefox 2 along with many other obsolete applications.

---

Tony_Weiss wrote:

Symantec has long been an advocate of independent “whole product” or “real-world” tests that most closely represent the interests of consumers and utilize all of the proactive technologies provided with a product, as opposed to “static” tests that only test an individual component.  Currently AV-Comparatives does not offer a subscription focused solely on these “whole product” or “real-world” types of tests.  At this time, Symantec has chosen not to subscribe to AV-Comparatives’ testing program for 2012.  Our philosophy is to participate in tests that have high relevance and meaning for consumers, and most accurately align with their real-world need for comprehensive protection and machine cleanup from evolving online threats.  We will continue to seek out and participate in a range of tests that meet these standards to measure our products’ efficacy and performance.

---

I don't buy it: http://www.av-comparatives.org/en/comparativesreviews/dynamic-tests

What else could AV-C possibly add to their gauntlet?  (please don't say Identity Safe, registry cleaning, cookie cleaning, backup)

I switched myself and clients to Norton because of AV-C's reports--which have needed granularity over av-test.

If I provide a client any security software that doesn't have top performance documented consistently and something goes wrong, I'm subject to scrutiny and I have nothing to qualify my recommendation.  If I do have documented test results that show my recommendations are sound and something goes wrong, then it comfortably falls under "nothing is 100%".

Since Norton won't have any current documentation, I can't recommend it and I won't be considering it in 2013.  That was a shotgun to the foot, not a .22.

Thanks for the link, SoJ --- much easier to understsand than the comparatives!

Have you received any definitive reasons and links for other independent 2012 tests?

I won't buy Symantec security products if they don't participate in the tests.  This tellsme they are afraid their product won't stand up to the competition, which scares me.  Good-bye Symantec!