• All Community
    • All Community
    • Forums
    • Ideas
    • Blogs
Kudos1

New Feature for Norton Internet Security 2010 - Download Insight

Why Download Insight?

Downloading malicious software, typically when tricked into doing so, is becoming the primary way malware infects people’s computers. Nearly every threat today is unique in some way and is designed to evade detection putting tremendous pressure on the traditional signature-based approach. By the time a signature is written for a particular malware variant,  it has already changed itself and as far as the signature is concerned it is an “unknown” file. Whether the signatures are on the disk or in the cloud, they are usually not fast enough to keep pace with modern threats.

The approach we are taking with Download Insight is to build a cloud-based reputation system. This system has knowledge of millions of applications and individual files across the globe and determines the reputation of each one using statistical methods.  This approach is the perfect complement to signatures—it is tailor-made for making decisions about unknown executables whereas signatures excel at telling you about something that is already known (like an existing virus or Trojan). We call our reputation-based intelligence "Quorum".

At a high-level, Insight will contact the Quorum server and ask for the reputation of the package. Based on the reputation, the package will be allowed to sit on the disk and execute, or will be deleted and removed from the computer.


Why is my cloud better than your cloud?

Cloud based scanning is the new buzz word in the security industry and a few security vendors are using the term although they often mean something very different.


The obvious advantage of cloud scanning is that the turnaround time for a definition to be available is extremely fast – as soon as a definition is available in the cloud, it is available to the user. Note that this approach still requires you to actually have seen the threat before in order to make a signature, a questionable assumption to make given the thousands of new threats produced every day.

What we have done with Quorum is to build a system that analyzes the reputation of the new software and files across the Internet and then calculates a reputation score for each of them. This system receives feeds from tens of millions of customers that anonymously participate in the Norton Community Watch program. Quorum automatically starts working on calculating the reputation score as it becomes aware of new files.

Now this is powerful – we have a system that can receive knowledge of new files worldwide and use a Symantec “secret sauce” algorithm to calculate the reputation score automatically! This information is immediately available to Download Insight through the cloud, but quite a bit different than just moving the old signature model to the cloud.


How is the reputation score of a file determined?

A reputation score is calculated using a complex algorithm based on various parameters. Remember, the main feed in to the Reputation system is the information received from the Norton Community Watch program. 

Here’s a list of a few parameters that are used to calculate the reputation score:
  -    How many instances of a particular file are seen?
  -    How long has that file been around?
  -    From which URLs were they downloaded?
  -    What is the basic health of the system that is submitting the data?
  -    Which software vendor does the file belong to?

These parameters are fed into a complex algorithm that determines the score of an application or file. As we continuously receive new information – the score of a file can change over time.


Download Insight in action

Download Insight monitors when new files are downloaded, and once the download is complete it goes into action.  From a user’s point of view, it should be straight-forward as there are basically two “flows”:

1) Save the downloaded File
This is the flow where the user chooses to save the application to a folder on the computer.


  1.    Download Insight observes that the file download from the Internet is complete.
  2.    It calculates the SHA256 hash of that file and immediately asks the Quorum online servers for a reputation score.
  3.    Based on the reputation score, Download Insight will:
         a.    Delete the application if the reputation score is at a “Bad” level and display a notification to the user.
         b.    Allow the file to persist if the reputation score is “Good” and display a corresponding notification.
         c.    Provide additional information when the score for the file is still being evaluated.

Here is what the notifications can look like depending on the reputation score:


Figure 1 


The “View Details” link for each notification provides more information from our Quorum servers. Here are a few examples:


  1.    Prevalence – How widely used is this file is in the Norton Community? It can range from very few instances to millions of machines.
  2.    Age – How long has this file been around?
  3.    Reputation Rating – What does Norton think of this file? It provides an indication on how trustworthy the file is.
  4.    URL – This provides the website from which this file was downloaded.


 Figure 2



While each individual item listed above is useful in itself, it becomes powerful when combined to build a picture of how trustworthy a particular file is.

Let’s try to draw an analogy – say you want to buy a new HD camera. Typically what you would do is try to find more information about it on the internet. After the research if you find that it is a popular camera and the camera itself has been available in the market for a long time then that builds credibility for that camera and your chances of buying it might be higher – or we can say that its “reputation” is good.

At the same time, if you come across a brand new camera that was released last week and very few folks out there have tried it out – you may say that you’d like to wait and see how this camera pans out – and your chances of buying it right away could be lower - or in other words its “reputation” would be considered lower.

Something similar can be applied to software applications as well.

2) Run the downloaded File
The second user flow where Download Insight participates is the time when you run the application downloaded from the Internet – it could be right after you download the application or couple of days later when you choose to install the application.

If the reputation of the file was still being evaluated (yellow notification in Figure 1), Norton will alert the user with a dialogue that provides the information showed in Figure 2 and has recommendation on what the user can do with application. It looks like:


 Figure 3



We can treat this category as “currently being monitored” – every time an application with a yellow reputation score is launched, we re-query the reputation server to see if it has any new information on this particular file.
Both the notification (figure 1) & dialog (Figure 3) can be disabled or made more active via the feature settings depending on the user’s level of interest.


Summary

For the 2010 product line, we’re introducing a new reputation-based means of protecting our customers against unknown malware called Quorum. Quorum has been in the works for several years now and is designed specifically to protect against today’s breed of unknown malware.  Even better, Quorom provides useful intelligence on all files, good or bad, that we make available to our customers through Download Insight and other features in 2010.  Download Insight brings you this information when you need it the most—right before you install a downloaded file.  We think the result will not only be better protection, but a great experience overall for our customers.


Q & A

Q: Is Download Insight similar to Norton Insight from the 2009 product line?

A: Reputation scoring based on the Quorum backend intelligence is a leap forward in the functionality of Norton Insight. In the Norton 2009 product line, we leveraged our Norton Community program to identify the “good guys” (files, executables). With our 2010 release, we have taken it to a next level where we identify who the “bad guys” are and provide more protection and intelligence to our users.


Q: What browsers do we support currently?

A: Currently Download Insight supports Internet Explorer 6.0 & above and Firefox 3.0 & above. We have plans to extend this functionality to more browsers in the future.


Q: How long does it take to retrieve the reputation score?

A: The amount of data sent and received for getting the reputation score is small, making the response time very fast. For a normal operation the delay will not be perceived by users.


Q: Can a reputation score change for the file I have already downloaded?

A: Yes, a reputation score can change as we continue to receive more information from Norton Community Watch and Quorum. The next time Download Insight asks for this information (e.g. run the file scenario above) we will fetch a new, updated reputation score if it is available.


Q: Is any personal information sent to or stored by Symantec?

A: No, the queries only include the file hash, and no personally identifiable information is submitted or stored by Symantec. This also applies to any Norton Community Watch submissions.



(view in My Videos)

Comments

Kudos0

Hi, Viral,

Like the colourful Notifications!

If I understand you correctly, each time to Program on the computer is Run, File(s) will be Submitted to symantec?  If so, does this apply to the Programs already installed on the User's computer that has not been Downloaded and Installed when N.I.S. 2010 was installed?

If the Download Insight Detects a File as Bad, or turns Bad, will Virus Definitions be Created and Released for that File?

Thursday, November 21, 2013: The THREATCON was changed to Level 1: Normal | Tue., Nov. 05, 2013: Zero-Day Vulnerability: Microsoft Security Advisory 2896666 | Saturday, November 09, 2013: Cyber-Criminals Serve Up A Veritable Smorgasbord Of Threats For South Koreans | Wednesday, October 09, 2013: New Internet Explorer Zero-Day Targeted In Attacks Against Korea And Japan [C.V.E.-2013-3897]
Kudos0

this is so stupid! i am a user who once a file is downloaded to my desktop is always rite clicked on and scanned BEFORE it is clicked on to install.

also 2010 is a step back to the bloat days as compared to the slimmed down and very nice 2009 Norton anti virus as i only use your antivirus product.

also not trying in anyway to be pejudiced but many users would probably get a warmer feeling about your product with clear english speaking commetators. i realize the gentleman must know his stuff as he is your project manager for this part of nav 2010 i believe i heard him say. but i add this part knowing you are selling worldwide yet so many user have complained about computer makers and software makers using offshore support it doesn't help your product. just one mans opinion, nothing more. i clearly understood what was spoken but it was a tad harder than it should be.

thank you,

chevysales

Kudos0
"I am investigating taking automatic action on these "Bad" files..."
We're looking at automatically quarantining files when launched if their reputation level has changed to “Bad” from when they were downloaded and when they are launched. Right now we show an alert and ask.

"Programs that are already on the machine before the product is installed will not be checked when they are launched."
I’m speaking specifically of Download Intelligence, they are still processed by all our other protection engines (Auto-Protect, SONAR2, etc.). They are not analyzed by Download Intelligence because we did not identify them as a downloaded file.
Principal Software EngineerNorton Research Labs
Kudos0

It seems that there are multiple layers of protection in NIS, which is undoubtedly good. I, personally, don't care what protects me, as long as I am safe.

But, as seen in NIS 2010 beta forum, it happens that whatever protects me in NIS is over zealous. It just deletes the file without asking user's permission. I agree that we have to get rid of bad file, but not necessarily of the file which is simply unknown. If there will be too many false positives users will opt out from Norton Community Watch making the whole project less efficient. 

Sever issue mdturner mentioned, is very important to reliability of cloud based security approach. At the same time, I hope, falling back on old, signature based solution, should be enough till problems are resolved . It should be, however, very short time. 

Norton for a long time was chastised by security geeks (see wilderssecurity.com) for being too heavy on system resources. It is not entirely true what Viral says in video that "if you wanted security you had to sacrifice performance'. In 2007 or 2008 there were solutions as effective as Norton, or better, without being heavy on the system.

AFAIK new Norton is much better. Reliability of cloud based approach to security is crucial as most of the competition will most likely go in the same direction. 

Win7 64-bit build 7100 NIS 2010 - 17.0.0.136
Kudos0

Hi, Garret_Polk, 

"I am investigating taking automatic action on these "Bad" files...": What do you mean? 

"Programs that are already on the machine before the product is installed will not be checked when they are launched."  Why is this?  I thought that it would check every Program on your computer, whether they had been Installed via Norton Download Insight on your computer because Anti-Virus Scans from the previous Product might have Not Detected a Threat/Threats on your computer - and that S.O.N.A.R. Missed - that Download Insight would Detect.

Thanks for getting back to us!

Thursday, November 21, 2013: The THREATCON was changed to Level 1: Normal | Tue., Nov. 05, 2013: Zero-Day Vulnerability: Microsoft Security Advisory 2896666 | Saturday, November 09, 2013: Cyber-Criminals Serve Up A Veritable Smorgasbord Of Threats For South Koreans | Wednesday, October 09, 2013: New Internet Explorer Zero-Day Targeted In Attacks Against Korea And Japan [C.V.E.-2013-3897]
Kudos0

I agree that most people would feel the same way - "don't care as long as I am safe" but I go back to my previous comment. When systems that are protecting you prove to be unreliable/unavailable what is the point of a cloud based system? You may well be better off with the protection being back on your own system. It is all very well making protection systems lighter on your PC by moving some of the protection into the clouds but I remain to be convinced that this sort of compomise actually keeps us as safe as we would wish to be.

We look forward to the time when the Power of Love will replace the Love of Power. Then will our world know the blessings of peace. ~William Ewart Gladstone
Kudos0

Download Insight really should be compatible with Google Chrome by now.

Kudos0
Given the serious issues with Symantec Servers over recent days, how can we be confident in something that relies on, what has been of late, unreliable servers. To date we have had very little in the way of updates as to what the issues where or when they will be fully resolved.
We look forward to the time when the Power of Love will replace the Love of Power. Then will our world know the blessings of peace. ~William Ewart Gladstone
Kudos0

Hi, I'm one of the developers on Download Insight. I'll see if I can answer these questions.

Download Insight watches for executable files downloaded by Internet Explorer and Firefox. When the download has completed the hash of the file is computed and used as a query to our cloud servers. The file is never sent. If you don't have an Internet connection we won't be able to connect to the cloud, but you won't be able to download the file either. Programs that are already on the machine before the product is installed will not be checked when they are launched.

When you launch a file that was "watched' by Download Insight we make another query to our cloud to see if the data has changed. If we get no response from the cloud, say you have no Internet connection, we assume the file is unknown and will show an alert. If we are able to query and the file is now "Bad" we don't take automatic action but show "Bad"-type information on the alert.

I am investigating taking automatic action on these "Bad" files, so this behavior may change in a later beta build.

Principal Software EngineerNorton Research Labs
Kudos0

If I understand this correctly, you have to connected to the internet for all this new functionality, correct?

So what is the new technology / systems doing for the users who may be offline from the internet for some periods of time?  A company field service employee for example.

Also, if you download a "currently being monitored" file, go offline and install the program.  Later, Norton decides this is a "Bad" file.  When the user goes back online now does Norton remove the installation file and the installed program or what?

Win7 x32 SP1
Kudos0

Granted that this technology is new and that the initial releases may have some bugs,but  I feel that Norton has strayed into the area of unnecessary bells and whistles that don't quite work properly, the idle time FS scans are iffy, the number of trusted files in the scan results don't match the number of trusted files in Insight,  and on and on.  I would like to see less GUI and more server reliability, scan reliability, fewer patches, and fewer installation errors.

To me, adding the download and installation stats page is relatively pointless, particularly when installations are no longer reported due to a glitch in one of the updates.  It worked in November, but not since.  Since that is directly tied into the Insight components, one can't help but wonder what else isn't working.

Under certain circumstances profanity provides relief denied even to prayer.Mark Twain
Kudos0

Why Download Insight?

Downloading malicious software, typically when tricked into doing so, is becoming the primary way malware infects people’s computers. Nearly every threat today is unique in some way and is designed to evade detection putting tremendous pressure on the traditional signature-based approach. By the time a signature is written for a particular malware variant,  it has already changed itself and as far as the signature is concerned it is an “unknown” file. Whether the signatures are on the disk or in the cloud, they are usually not fast enough to keep pace with modern threats.

The approach we are taking with Download Insight is to build a cloud-based reputation system. This system has knowledge of millions of applications and individual files across the globe and determines the reputation of each one using statistical methods.  This approach is the perfect complement to signatures—it is tailor-made for making decisions about unknown executables whereas signatures excel at telling you about something that is already known (like an existing virus or Trojan). We call our reputation-based intelligence "Quorum".

At a high-level, Insight will contact the Quorum server and ask for the reputation of the package. Based on the reputation, the package will be allowed to sit on the disk and execute, or will be deleted and removed from the computer.


Why is my cloud better than your cloud?

Cloud based scanning is the new buzz word in the security industry and a few security vendors are using the term although they often mean something very different.


The obvious advantage of cloud scanning is that the turnaround time for a definition to be available is extremely fast – as soon as a definition is available in the cloud, it is available to the user. Note that this approach still requires you to actually have seen the threat before in order to make a signature, a questionable assumption to make given the thousands of new threats produced every day.

What we have done with Quorum is to build a system that analyzes the reputation of the new software and files across the Internet and then calculates a reputation score for each of them. This system receives feeds from tens of millions of customers that anonymously participate in the Norton Community Watch program. Quorum automatically starts working on calculating the reputation score as it becomes aware of new files.

Now this is powerful – we have a system that can receive knowledge of new files worldwide and use a Symantec “secret sauce” algorithm to calculate the reputation score automatically! This information is immediately available to Download Insight through the cloud, but quite a bit different than just moving the old signature model to the cloud.


How is the reputation score of a file determined?

A reputation score is calculated using a complex algorithm based on various parameters. Remember, the main feed in to the Reputation system is the information received from the Norton Community Watch program. 

Here’s a list of a few parameters that are used to calculate the reputation score:
  -    How many instances of a particular file are seen?
  -    How long has that file been around?
  -    From which URLs were they downloaded?
  -    What is the basic health of the system that is submitting the data?
  -    Which software vendor does the file belong to?

These parameters are fed into a complex algorithm that determines the score of an application or file. As we continuously receive new information – the score of a file can change over time.


Download Insight in action

Download Insight monitors when new files are downloaded, and once the download is complete it goes into action.  From a user’s point of view, it should be straight-forward as there are basically two “flows”:

1) Save the downloaded File
This is the flow where the user chooses to save the application to a folder on the computer.


  1.    Download Insight observes that the file download from the Internet is complete.
  2.    It calculates the SHA256 hash of that file and immediately asks the Quorum online servers for a reputation score.
  3.    Based on the reputation score, Download Insight will:
         a.    Delete the application if the reputation score is at a “Bad” level and display a notification to the user.
         b.    Allow the file to persist if the reputation score is “Good” and display a corresponding notification.
         c.    Provide additional information when the score for the file is still being evaluated.

Here is what the notifications can look like depending on the reputation score:


Figure 1 


The “View Details” link for each notification provides more information from our Quorum servers. Here are a few examples:


  1.    Prevalence – How widely used is this file is in the Norton Community? It can range from very few instances to millions of machines.
  2.    Age – How long has this file been around?
  3.    Reputation Rating – What does Norton think of this file? It provides an indication on how trustworthy the file is.
  4.    URL – This provides the website from which this file was downloaded.


 Figure 2



While each individual item listed above is useful in itself, it becomes powerful when combined to build a picture of how trustworthy a particular file is.

Let’s try to draw an analogy – say you want to buy a new HD camera. Typically what you would do is try to find more information about it on the internet. After the research if you find that it is a popular camera and the camera itself has been available in the market for a long time then that builds credibility for that camera and your chances of buying it might be higher – or we can say that its “reputation” is good.

At the same time, if you come across a brand new camera that was released last week and very few folks out there have tried it out – you may say that you’d like to wait and see how this camera pans out – and your chances of buying it right away could be lower - or in other words its “reputation” would be considered lower.

Something similar can be applied to software applications as well.

2) Run the downloaded File
The second user flow where Download Insight participates is the time when you run the application downloaded from the Internet – it could be right after you download the application or couple of days later when you choose to install the application.

If the reputation of the file was still being evaluated (yellow notification in Figure 1), Norton will alert the user with a dialogue that provides the information showed in Figure 2 and has recommendation on what the user can do with application. It looks like:


 Figure 3



We can treat this category as “currently being monitored” – every time an application with a yellow reputation score is launched, we re-query the reputation server to see if it has any new information on this particular file.
Both the notification (figure 1) & dialog (Figure 3) can be disabled or made more active via the feature settings depending on the user’s level of interest.


Summary

For the 2010 product line, we’re introducing a new reputation-based means of protecting our customers against unknown malware called Quorum. Quorum has been in the works for several years now and is designed specifically to protect against today’s breed of unknown malware.  Even better, Quorom provides useful intelligence on all files, good or bad, that we make available to our customers through Download Insight and other features in 2010.  Download Insight brings you this information when you need it the most—right before you install a downloaded file.  We think the result will not only be better protection, but a great experience overall for our customers.


Q & A

Q: Is Download Insight similar to Norton Insight from the 2009 product line?

A: Reputation scoring based on the Quorum backend intelligence is a leap forward in the functionality of Norton Insight. In the Norton 2009 product line, we leveraged our Norton Community program to identify the “good guys” (files, executables). With our 2010 release, we have taken it to a next level where we identify who the “bad guys” are and provide more protection and intelligence to our users.


Q: What browsers do we support currently?

A: Currently Download Insight supports Internet Explorer 6.0 & above and Firefox 3.0 & above. We have plans to extend this functionality to more browsers in the future.


Q: How long does it take to retrieve the reputation score?

A: The amount of data sent and received for getting the reputation score is small, making the response time very fast. For a normal operation the delay will not be perceived by users.


Q: Can a reputation score change for the file I have already downloaded?

A: Yes, a reputation score can change as we continue to receive more information from Norton Community Watch and Quorum. The next time Download Insight asks for this information (e.g. run the file scenario above) we will fetch a new, updated reputation score if it is available.


Q: Is any personal information sent to or stored by Symantec?

A: No, the queries only include the file hash, and no personally identifiable information is submitted or stored by Symantec. This also applies to any Norton Community Watch submissions.



(view in My Videos)
Kudos0
I totally agree with you.  They also need to rip things like the optimize feature out.  Vista and 7 defragment automatically and so it will become less important as time goes by and more and more people move to the newer MS operating systems.  I would rather they concentrate on reliability and improving their detection and removal capabilities of rootkits.  How many people come to the forum with rootkits that NIS can't see and can't remove.  I am starting to lose faith in the tool.  What is the point of knowing all the programs that have been installed and having a well defragmented hard drive if you can't get rid of rootkits.  I am paying for detection and removal...
Kudos0

I'm a small developer and I'm now getting my installation exe download blocked by Download Insight.

How do I get it unblocked before I go out of businesss?

Kudos0

 What an awesome  explaination!

Thanks!

If you believe you will receive whatever you ask in prayer