This is a re-post of a topic I started on the forum some weeks ago. However, following the upgrade to the community forum, my original topic has been moved to the archive and the screenshot images have been lost. So I’ll start again.
The issue started on May 10th when I noted that Norton’s Product Maintenance task frequently failed to complete. I know from an issue some years ago (also DNS related) that some Norton background tasks need to access the server stats.norton.com. On checking I found that my ISP’s DNS was regularly failing to resolve stats.norton.com. Further investigation revealed that stats.norton.com is now hosted by Microsoft’s Azure cloud services and access managed by Azure Traffic Manager. My ISP diagnosed that the issue occurs when its DNS attempts to refresh its cached records from the trafficmanager.net authoritative nameservers via IPv6 but everything works fine if refreshed via IPv4.
This is a detailed explanation of the path to the server:
-
The Norton domain stats.norton.com is now hosted by Microsoft Azure cloud services. Access to the Azure hosts is managed by Azure’s Traffic Manager system. Therefore, the DNS record for stats.norton.com is now an alias (CNAME) that points to stats.norton.com.trafficmanager.net.
-
Stats.norton.com.trafficmanager.net is a further CNAME that resolves to one of two end-points, either mue1-bit-ipv6-pip.eastus.cloudapp.azure.com, with an IP address of 52.188.144.172, or muw1-bit-ipv6-pip.westus.cloudapp.azure.com, with an IP address of 13.64.142.149.
-
To ensure effective traffic management between the two endpoints, the CNAME record for stats.norton.com.trafficmanager.net has a short Time-to-Live (TTL), requiring downstream DNS servers to regularly refresh their cached copy of the record by reference to the trafficmanager.net nameservers.
Using the Powershell command Resolve-dnsname, this screenshot shows the correct resolution of stats.norton.com. The lookup is fully resolved to the westus Azure server and the authority section is that for Microsoft Azure host:
However, the DNS lookup often fails to fully resolve, resulting in this response. Stats.norton.com.trafficmanager.net is not resolved and the authority section is for trafficmanager.net, not the Azure endpoint.
The failure is due to the downstream DNS server failing to correctly refresh its stats.norton.com.trafficmanager.net record from a trafficmanager.net nameserver if accessed via IPv6. Trafficmanager.net publishes 4 nameservers, as per:
Resolving those names reveals that each of the nameservers has an IPv4 and an IPv6 address, as in this example of tm2.edgedns-tm.info:
Attempting to resolve stats.norton.com.trafficmanager.net via the IPv4 address works correctly but fails via the IPv6 address, as shown in this screenshot of attempts with the addresses for nameserver tm2.edgedns-tm.info:
As can be seen, via IPv4, the correct Answer section is returned. Via IPv6, only the Authority section is returned. The same problem is seen with any of the four nameservers.
Further confirmation of the issue of the trafficmanager.net nameservers not returning the correct response if queried via IPv6 can be seen in these tests using the NSLOOKUP command with the -debug switch. (Test results are shown as text extracts as the the responses are too lengthy to be captured in a screenshot.)
Firstly, querying the nameserver via its IPv4 address:
PS C:\WINDOWS\system32> Nslookup -debug stats.norton.com.trafficmanager.net 13.107.206.240
The response ends with this:
Got answer:
HEADER:
opcode = QUERY, id = 4, rcode = NOERROR
header flags: response, auth. answer, want recursion
questions = 1, answers = 1, authority records = 0, additional = 0
QUESTIONS:
stats.norton.com.trafficmanager.net, type = A, class = IN
ANSWERS:
-> stats.norton.com.trafficmanager.net
canonical name = muw1-bit-ipv6-pip.westus.cloudapp.azure.com
ttl = 30 (30 secs)
Got answer:
HEADER:
opcode = QUERY, id = 5, rcode = NOERROR
header flags: response, auth. answer, want recursion
questions = 1, answers = 1, authority records = 0, additional = 0
QUESTIONS:
stats.norton.com.trafficmanager.net, type = AAAA, class = IN
ANSWERS:
-> stats.norton.com.trafficmanager.net
canonical name = muw1-bit-ipv6-pip.westus.cloudapp.azure.com
ttl = 30 (30 secs)
Name: stats.norton.com.trafficmanager.net
In the final two parts of the response, both questions get the Answer record correctly resolving the canonical name (CNAME) stats.norton.com.trafficmanager.net to muw1-bit-ipv6-pip.westus.cloudapp.azure.com. The response header shows the responses are Answer records.
Secondly, querying the nameserver via its IPv6 address:
PS C:\WINDOWS\system32> Nslookup -debug stats.norton.com.trafficmanager.net 2620:1ec:bda:f00::f0
The response ends with this:
Got answer:
HEADER:
opcode = QUERY, id = 4, rcode = NOERROR
header flags: response, auth. answer, want recursion
questions = 1, answers = 0, authority records = 1, additional = 0
QUESTIONS:
stats.norton.com.trafficmanager.net, type = A, class = IN
AUTHORITY RECORDS:
-> trafficmanager.net
ttl = 30 (30 secs)
primary name server = tm1.dns-tm.com
responsible mail addr = hostmaster.trafficmanager.net
serial = 2003080800
refresh = 900 (15 mins)
retry = 300 (5 mins)
expire = 2419200 (28 days)
default TTL = 30 (30 secs)
Got answer:
HEADER:
opcode = QUERY, id = 5, rcode = NOERROR
header flags: response, auth. answer, want recursion
questions = 1, answers = 0, authority records = 1, additional = 0
QUESTIONS:
stats.norton.com.trafficmanager.net, type = AAAA, class = IN
AUTHORITY RECORDS:
-> trafficmanager.net
ttl = 30 (30 secs)
primary name server = tm1.dns-tm.com
responsible mail addr = hostmaster.trafficmanager.net
serial = 2003080800
refresh = 900 (15 mins)
retry = 300 (5 mins)
expire = 2419200 (28 days)
default TTL = 30 (30 secs)
Name: stats.norton.com.trafficmanager.net
In this response, the final two questions receive Authority records rather than Answer records, as confirmed by the record counts in the response headers.
Interestingly, the incorrect response returned from the authoritative nameserver is also reflected by downstream recursive DNS servers if accessed via IPv6. The Google DNS resolves correctly if accessed via its IPv4 address of 8.8.8.8. However, if accessed via IPv6 using address 2001:4860:4860::8888, the response returns only the Authority record, as shown here:
Not all DNS providers are impacted by this issue, both Google DNS and Cloudflare DNS consistently resolve stats.norton.com OK if queried via their IPv4 addresses. My current workaround is to set my router to use Google IPv4 DNS instead of my ISP DNS. Previously I bypassed DNS with a mapping for stats.norton.com in my Windows hosts file.
After feedback from my ISP, I raised a support case with Norton support. An update some weeks ago was that the case had now been escalated from “L2” to “L3”, but the first level call agents were not able to give me any further details of the investigation. Ultimately, I would think Norton need to raise this with Microsoft as the issue would appear to be with the Azure Traffic Manager nameservers.
I received an email from Norton Support saying they tried unsuccessfully to contact me by phone and that no voicemail was available. That doesn’t make sense as I have voicemail and there is no missed call on my incoming call log. I contacted Support again, checked that they do have my correct phone number and booked a call-back but again no call was received.
A further related discovery is from this Traffic Manager FAQs web page (which is dated June 3rd, later than the date in May when this issue started):
https://learn.microsoft.com/en-us/azure/traffic-manager/traffic-manager-faqs
It includes this section:
Does Traffic Manager support IPv6 endpoints?
Traffic Manager doesn’t currently provide IPv6-addressable name servers. However, Traffic Manager can still be used by IPv6 clients connecting to IPv6 endpoints if the client’s recursive DNS server supports IPv4. A client doesn’t make DNS request directly to Traffic Manager. Instead, the client uses a recursive DNS service. An IPv6-only client sends requests to the recursive DNS service via IPv6. The recursive service must then be able to contact the Traffic Manager name servers using IPv4. Traffic Manager responds with the DNS name or IP address of the endpoint.
The statement that Traffic Manager doesn’t provide IPv6-addressable name servers conflicts with reality. All four trafficmanager.net nameservers do have IPv6 addresses, as follows:
Tm1.dns-tm.com has the IPv6 address 2603:1061:0:f00::f0
Tm2.dns-tm.com has the IPv6 address 2620:1ec:8ec:f00::f0
Tm1.edgedns-tm.info has the IPv6 address 2a01:111:4000:f00::f0
Tm2.edgedns-tm.info has the IPv6 address 2620:1ec:bda:f00::f0
If Microsoft’s understanding is that they don’t support IPv6 nameservers for Traffic Manager, maybe those nameservers are not properly configured for IPv6 operation and that explains why, when addressed via IPv6, they return Authority records and not Answer records when asked to resolve stats.norton.com.trafficmanager.net.
Again, Norton Support need to pursue this with Microsoft Azure Support.
If a Norton forum moderator is monitoring this topic, I’d be grateful if they could pursue it with Norton support to obtain some feedback as to the current state of play of my support case.
My setup is a Dell Optiplex 5090, with Windows 11 Pro 23H2, running Norton 360 version 22.24.5.6.