Chocolate-Covered PIFTS, X-Files & Mea Culpas
Author: davecole27 Employee Posted: 12-Mar-2009 | 9:12PM · 9 Comments
Ever had one of those days?
You really intended for things to go right, but they didn’t. And they didn’t simply whimper out of existence, but left large, smoking holes in the living room carpet. That was this Tuesday.
In truth, it began on Monday evening around 4:30 p.m. Pacific when we released a patch, the infamously named “pifts.exe,” which left the building before we signed it. This is a process that has worked like clockwork for many years, but due to a simple mistake, the patch wasn’t signed and was instead sent out to Norton AntiVirus and Norton Internet Security customers using the ‘06 & ’07 versions of our products. The patch also inadvertently went out to some people who still had old versions of LiveUpdate & LiveUpdate Notice lingering on their PC (which, yes, should have been removed when the original product was removed; we’ve since fixed this, details are here). The unsigned PIFTS patch is what resulted in the firewall alerts, and we can certainly understand why people were concerned when they saw an unknown application attempting to communicate to the Internet. Personally, I’d be concerned too.
At this point, we’ve explained what PIFTS is, from both a technical and a layman’s perspective. It’s a patch that is used for diagnostic purposes to determine the state of a product and will then let us know if the product can receive upgrade messages, for example. This type of information is useful for when we plan migration of our customers to new operating systems, such as Windows 7. We can use it to plan for support calls, server loads, and a whole raft of things that you would expect us to do in advance of a major new release or an OS migration. On the other hand, PIFTS.exe does not collect or send any personal information—we don’t need this information to provide protection, and while we might really want to know who the next American Idol is, we’re betting that our guess is as good as yours, no matter what sort of insider information you might have on your hard drive. PIFTS.exe also does not make any changes to your PC, not even if you ask it politely. It’s just a patch that we forgot to sign before sending out, plain and simple, and which also went to a group of people who did not really need it. We pulled it back as soon as we realized our error, but not before a lot of people received an admittedly suspicious looking (yet harmless) alert.
And while it might be exotic to think that it sends information to Africa, the reality is much more mundane. It communicates back to our servers in North America. In fact, there isn’t even a decent Moroccan restaurant within driving distance to any of our data centers. Once we have this information, we use it in aggregate (again, we have no personally identifiable information) to plan roll-outs of new releases and product notices. No X-Files style plot, we promise.
Now this brings us to the Norton User Forums. Before I get into the topic of deleting posts and such, it’s important that you understand the background of our Forums. Some might believe the corporate myth that behind our Forums are legions of faceless automatons processing posts and replying to questions. Instead, the forums were started by a small group of people inside Symantec who wanted to begin a more direct dialogue with our customers and wanted to give customers a place to communicate with each other as well. There was not a lot of funding for the project and many were concerned that we would be overwhelmed by negative feedback, but we were given the blessing of the organization and we launched early last spring. By all measures, the Forums have been a success. Not only have we helped a lot of people (and allowed a lot of people to help other people), but we’ve learned a lot and had fun along the way.
We're open to discussing tough topics; we've seen many posts about non-Symantec security software that is recommended by users to other users. Those threads are not removed, as much as it pains us to see the commentary. We’ve taken on other sensitive topics, for example Firefox 3 compatibility issues last year. The forums were instrumental in helping us resolve patching issues for the updated browser that were causing some people pain while they were trying to surf the web. Even though it became a little heated, we never removed any of the threads because none were spam, no matter how unflattering they might be. We’ve prided ourselves from the outset in the Forums that we have not overreacted to people who wanted to bash Norton or those who referenced the competition. Our feeling has been that you have to take the good with the bad, and that it’s better to have the dialogue at our house than somewhere else on the Internet. If you take a look around on the forums, I’m sure you’ll see plenty of examples of this.
So let’s time warp to this past Monday around 8:44 p.m. EDT when we saw a post entitled “What is PIFTS.exe?” By 10:20 p.m. EDT, the thread had 18 replies, all from newly created accounts, and about 4,500 views. To give you some perspective, the announcement about the Norton Internet Security 16.5 update has received ~4,200 views in the 10 days it has been posted. Something smelled funny, so we made the thread read-only for the time being. Odd things continued to happen for the next few hours and culminated with a flurry of new users making massive amounts of posts, some obscene and some pretty humorous, for the better part of 6.5 hours. None of these posts had any real substance and were removed so as to not offend people and obscure the useful information that is on the Forums. We’ve got a sense of humor (Chocolate in your PIFTS, anyone?), but this was clearly the work of a group of people spamming the forums.
During this time, we know that some legitimate posts on the PIFTS topic from new community members were mistakenly removed by our moderators. We apologize to anyone whose post was lost in the process. We were not trying to cover anything up, but it was our attempt to clean up the mess left by the spammers and scrub the gunk they left on the forums. If you think we’re afraid of covering up the discussion or allowing opposing viewpoints on our Forums, check out the current thread on the topic.
There have been a number of remarks that we should have said something sooner about the unsigned patch, mysterious alerts and the deleted posts on the forums. We don’t disagree with you—if we were to go back in time we certainly would have posted what we knew about the series of events as quickly as possible instead of waiting as long as we did. I hope this provides some of the additional background on what happened and why we reacted how we did.
I know full well that those who want to believe this is somehow part of a plot to collect personal information for a Bernie Madoff-style ponzi scheme will continue to do so. However, for those of you who are willing to believe that we can make mistakes, we hope that you will understand that this was only a clumsy series of events initiated by honest intentions. Mea culpa, and apologies for all the confusion.
Comments
Phil_D replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
I appreciate your openness in posting this information on the blog. It reflects the same openness that I have seen on the Norton Community Forums over the past 9 months.
"We don’t disagree with you—if we were to go back in time....." You can't be more honest than that.
It is commendable that Symantec / Norton is willing to allow all points of view on their products and service. Even topics that may be less than flattering to the company are left in place for public viewing and comment.
I'm sure that most folks realize that the majority of the commotion was caused by a few individuals who do not reflect the true spirit of the Forums and its members.
Keep up the good work.
Dch48 replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
dbrisendine replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
Dch48 replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
davecole replied on Permalink
Chocolate-Covered PIFTS, X-Files & Mea Culpas
Ever had one of those days?
You really intended for things to go right, but they didn’t. And they didn’t simply whimper out of existence, but left large, smoking holes in the living room carpet. That was this Tuesday.
In truth, it began on Monday evening around 4:30 p.m. Pacific when we released a patch, the infamously named “pifts.exe,” which left the building before we signed it. This is a process that has worked like clockwork for many years, but due to a simple mistake, the patch wasn’t signed and was instead sent out to Norton AntiVirus and Norton Internet Security customers using the ‘06 & ’07 versions of our products. The patch also inadvertently went out to some people who still had old versions of LiveUpdate & LiveUpdate Notice lingering on their PC (which, yes, should have been removed when the original product was removed; we’ve since fixed this, details are here). The unsigned PIFTS patch is what resulted in the firewall alerts, and we can certainly understand why people were concerned when they saw an unknown application attempting to communicate to the Internet. Personally, I’d be concerned too.
At this point, we’ve explained what PIFTS is, from both a technical and a layman’s perspective. It’s a patch that is used for diagnostic purposes to determine the state of a product and will then let us know if the product can receive upgrade messages, for example. This type of information is useful for when we plan migration of our customers to new operating systems, such as Windows 7. We can use it to plan for support calls, server loads, and a whole raft of things that you would expect us to do in advance of a major new release or an OS migration. On the other hand, PIFTS.exe does not collect or send any personal information—we don’t need this information to provide protection, and while we might really want to know who the next American Idol is, we’re betting that our guess is as good as yours, no matter what sort of insider information you might have on your hard drive. PIFTS.exe also does not make any changes to your PC, not even if you ask it politely. It’s just a patch that we forgot to sign before sending out, plain and simple, and which also went to a group of people who did not really need it. We pulled it back as soon as we realized our error, but not before a lot of people received an admittedly suspicious looking (yet harmless) alert.
And while it might be exotic to think that it sends information to Africa, the reality is much more mundane. It communicates back to our servers in North America. In fact, there isn’t even a decent Moroccan restaurant within driving distance to any of our data centers. Once we have this information, we use it in aggregate (again, we have no personally identifiable information) to plan roll-outs of new releases and product notices. No X-Files style plot, we promise.
Now this brings us to the Norton User Forums. Before I get into the topic of deleting posts and such, it’s important that you understand the background of our Forums. Some might believe the corporate myth that behind our Forums are legions of faceless automatons processing posts and replying to questions. Instead, the forums were started by a small group of people inside Symantec who wanted to begin a more direct dialogue with our customers and wanted to give customers a place to communicate with each other as well. There was not a lot of funding for the project and many were concerned that we would be overwhelmed by negative feedback, but we were given the blessing of the organization and we launched early last spring. By all measures, the Forums have been a success. Not only have we helped a lot of people (and allowed a lot of people to help other people), but we’ve learned a lot and had fun along the way.
We're open to discussing tough topics; we've seen many posts about non-Symantec security software that is recommended by users to other users. Those threads are not removed, as much as it pains us to see the commentary. We’ve taken on other sensitive topics, for example Firefox 3 compatibility issues last year. The forums were instrumental in helping us resolve patching issues for the updated browser that were causing some people pain while they were trying to surf the web. Even though it became a little heated, we never removed any of the threads because none were spam, no matter how unflattering they might be. We’ve prided ourselves from the outset in the Forums that we have not overreacted to people who wanted to bash Norton or those who referenced the competition. Our feeling has been that you have to take the good with the bad, and that it’s better to have the dialogue at our house than somewhere else on the Internet. If you take a look around on the forums, I’m sure you’ll see plenty of examples of this.
So let’s time warp to this past Monday around 8:44 p.m. EDT when we saw a post entitled “What is PIFTS.exe?” By 10:20 p.m. EDT, the thread had 18 replies, all from newly created accounts, and about 4,500 views. To give you some perspective, the announcement about the Norton Internet Security 16.5 update has received ~4,200 views in the 10 days it has been posted. Something smelled funny, so we made the thread read-only for the time being. Odd things continued to happen for the next few hours and culminated with a flurry of new users making massive amounts of posts, some obscene and some pretty humorous, for the better part of 6.5 hours. None of these posts had any real substance and were removed so as to not offend people and obscure the useful information that is on the Forums. We’ve got a sense of humor (Chocolate in your PIFTS, anyone?), but this was clearly the work of a group of people spamming the forums.
During this time, we know that some legitimate posts on the PIFTS topic from new community members were mistakenly removed by our moderators. We apologize to anyone whose post was lost in the process. We were not trying to cover anything up, but it was our attempt to clean up the mess left by the spammers and scrub the gunk they left on the forums. If you think we’re afraid of covering up the discussion or allowing opposing viewpoints on our Forums, check out the current thread on the topic.
There have been a number of remarks that we should have said something sooner about the unsigned patch, mysterious alerts and the deleted posts on the forums. We don’t disagree with you—if we were to go back in time we certainly would have posted what we knew about the series of events as quickly as possible instead of waiting as long as we did. I hope this provides some of the additional background on what happened and why we reacted how we did.
I know full well that those who want to believe this is somehow part of a plot to collect personal information for a Bernie Madoff-style ponzi scheme will continue to do so. However, for those of you who are willing to believe that we can make mistakes, we hope that you will understand that this was only a clumsy series of events initiated by honest intentions. Mea culpa, and apologies for all the confusion.
Message Edited by davecole on 03-26-2009 03:37 PMMrZero replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
I tip my hat to Symantec. The effort made by the staff to be active and respond quickly to user concerns as well as general support questions should be adopted by all (no I wont name companies...) ;)
NIS 2009 is by far the best AV/Security product you have produced. Thanks Symantec!
huwyngr replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
tukangnggame replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
Floating_Red replied on Permalink
Re: Chocolate-Covered PIFTS, X-Files & Mea Culpas
"NIS 2009 is by far the best AV/Security product you have produced."
I personally don't agree with this statement 100%, but I Agree with it somewhat; there are some things that annoy me with N.I.S. 2009. Anyway, just wait until Future Products. ;)