Message-ID: <00a301c3de02$de6c0d70$4605bb0a@corp.ad.timeinc.com>
From: jlevitsk at joshie.com (Joshua Levitsky)
Subject: Re: Symantec AntiVirus and AOL

Symantec AntiVirus 8.1.1 build 319 addresses the problem below. It is now 
available from Symantec.

--
Joshua Levitsky, MCSE, CISSP
System Engineer
Time Inc. Information Technology
[5957 F27C 9C71 E9A7 274A 0447 C9B9 75A4 9B41 D4D1]


----- Original Message ----- 
NOTE: This is not an exploit. This is not a vulnerability. This is simply a 
bug that makes management of clients more difficult / broken. Posting this 
to hopefully bring a bug in to the open that may not have been discovered 
yet at companies other than my own. If this was a flaw that allowed an 
exploit or if this allowed a system compromise then I would not post this. 
If anyone knows of other virtual adapters that cause this problem then I 
would appreciate emails listing the product and the MAC address that the 
product uses.


--------------------------- 


I have a question for all of you about something I noticed with Symantec 
AntiVirus Corporate Edition. The SAV CE client uses a GUID (unique 
identifier) that is generated based on the MAC address of the Ethernet 
adapter on the machine SAV CE is running on. This GUID is then used by the 
SAV CE Parent Server that the managed client checks in to. If more than one 
machine has the same MAC address then in the SSC (Symantec System Console) 
you will only see 1 machine even if 50 machines have the same GUID and check 
in to the Parent. If you think that your SSC is showing less clients than it 
should then you should possibly read this document to see if this affects 
you. What I noticed was that this problem was not in my corporate image at 
first but then it happened, and finally enough machines have been replaced / 
reimaged that I noticed I was short on clients in the SSC.


What is a GUID? Where is the GUID found? A GUID is a Global Unique ID. Well 
at least it is supposed to be unique. It is found in the key below. It is 
generated using a Microsoft library that is part of RPC. It is based on the 
MAC address of an Ethernet adapter and your current IP address. Is the IP 
address enough to make it unique even when the MAC is the same? I do not 
know. I have been unable to find the answer. If the IP is enough to make 
this value unique then remediation of this issue is as simple as deleting 
the GUID key on all workstations that are not showing up in the SSC.


[HKEY_LOCAL_MACHINE\SOFTWARE\Intel\LANDesk\VirusProtect6\CurrentVersion]
"GUID"=hex:3b,13,fe,fd,4c,6f,cc,4c,89,5b,82,1f,2b,c4,a0,43


How does SAV CE pick which adapter to make a GUID from? It picks the first 
thing that looks like an Ethernet adapter by going through the binding order 
in windows.


How can multiple machines end up with the same GUID? Imagine a corporate 
standard image that might include the AOL client which adds a hidden adapter 
to the system. Now when you ghost that image to a new machine and the 
machine goes through sysprep it will add the Ethernet card of the machine 
you are putting your image on. Since the AOL adapter never goes away then 
the AOL adapter of course comes before the real Ethernet adapter. This means 
that if the AOL adapter was selected by SAV CE in the master image then no 
new GUID will generate because the GUID only changes when an adapter is 
added or removed. Since the AOL adapter never leaves then it has a good 
chance of coming before real adapters in the binding order. If the AOL 
adapter was not selected in the master image by the LocalMAC registry key 
then a GUID should be generated when you image a new target machine because 
SAV CE sees that the old adapter leaves the machine and then it will look to 
the binding order to pick a new adapter. SAV CE hopes to get a real Ethernet 
card and a GUID should be generated based on the MAC address. So this 
problem happens most when the AOL Adapter is watched by SAV, and that never 
leaves so LocalMAC will always watch it and a new GUID never is generated.


How can you see machines with duplicate GUIDs if you think they are not 
showing up in your SSC? Delete the key below using whatever you have like 
SMS / NetOctopus / etc. The GUID will regenerate and maybe you won't have 
duplicate GUIDs, but if you are generating the GUID from the AOL MAC address 
then the potential for duplicates continues to exist. Note that you want to 
probably clear out all the host records in the SSC if you do this because 
you will see duplicates because the GUID is how the SSC tracks records, and 
it takes 30 days for a stale record to purge out of the SSC with the current 
version of SAV CE.


[HKEY_LOCAL_MACHINE\SOFTWARE\Intel\LANDesk\VirusProtect6\CurrentVersion]
"GUID"=hex:3b,13,fe,fd,4c,6f,cc,4c,89,5b,82,1f,2b,c4,a0,43


How can you know that you are using the AOL MAC address instead of the real 
one? It would seem that AOL 7.x, 8.x, and 9.x all use 00-03-8a-00-00-15 as 
the MAC address for the virtual adapter. This aparently is the pseudo-VPN 
adapter used when you connect to AOL over IP. Deleting this key and 
rebooting will simply result in the same MAC address being tracked because 
the AOL adapter will still come first in the binding list. The binding list 
is from Windows binding order, and not anything from Symantec.


[HKEY_LOCAL_MACHINE\SOFTWARE\Intel\LANDesk\VirusProtect6\CurrentVersion]
"LocalMAC"=hex:00,03,8a,00,00,15


Can't I just change the binding order in Properties for My Network Places in 
the Advanced Options? No. The AOL adapter does not appear there for AOL 7.x, 
8.x and 9.x. The adapter does not exist like it did in AOL 6.x. The only way 
to remove the adapter is to uninstall AOL, and even then if you re-install 
AOL it appears that it can come first again in the binding order. You will 
get a new GUID, but since it is based on the AOL MAC addres then it is 
possibly you will get a duplicate GUID again.


Doesn't the Parent Server resolve GUID conflicts and tell a client to make a 
new one if there is a conflict? No. Because the GUID is based on the MAC 
address, and the MAC address should never be the same from machine to 
machine, the Parent Server does not do conflict resolution of GUIDs. I am 
filing this idea as a RFE however with Symantec because if the server simply 
did this then it would fix issues with other adapters.


Is this problem AOL's because of the adapter? No. AOL is creating a virtual 
adapter just like a VPN client might. In concept what AOL is doing is fine. 
The potential for this problem with other pre-loaded software on a master 
image is great if that software makes a virtual adapter. In fact if your 
company pre-loads VPN software or such then I would encourage you to look at 
the LocalMAC value listed above and see if it matches your real Ethernet 
adapter that you can see by going to Start -> Run and typing "cmd /k 
ipconfig /all"


What versions of SAV CE does this affect, and what operating systems? It 
affects everything from NAV 7.61 through SAV 8.1.1 that I've looked at, and 
has the same problem on Windows 2000 and XP. It appears to be a universal 
problem. (Windows NT and 98 were not looked at because they are dead 
products.)


Is this problem Symantec's because they bind to a VPN adapter? Sort of but 
not really. Symantec could have tested for more things like default route or 
adapter able to reach the Parent server when doing the figuring out, but 
they seem to have a pretty good detection system. It would seem to me that 
the burden is on Symantec to put in an exclusion for the AOL MAC address 
just like any other adapter found to cause similar problems.


How critical is this? It appears to be more of an annoyance than anything 
else. If you turn on debugging on your clients then you will see the clients 
check in with "mommy" and it seems like policies are being communicated from 
the Parent servers to their children. So it makes it so that you can't 
really tell if your environment is protected. Depending on how you view this 
to be critical will make it more or less important to you.


What can you do to fix this? I opened a ticket with Symantec Platinum 
support a week ago. They said they have not seen other customers with this 
issue. They are working on it though and understand the issue, but I think 
they might not be 100% on board with this being something for Symantec to 
fix. If you find that this issue affects you then I encourage you to open a 
ticket with Symantec so that they can find out if this affects more people 
than just my 10,000 clients. While 10,000 is a nice big number, it is only 
one company so it is understandable that Symantec would wonder why no other 
incidents were filed. If you have this problem then it is urgent that you 
make a ticket. If you do not have support and it would cost you money to 
make a ticket then please just email me how many clients you have and what 
company you are with, and I will tell Symantec so you do not have to pay for 
support on this issue.


Should you be angry with Symantec about this? No. Seriously. There's no way 
they could have seen this bug. I just wanted to post this document out there 
so that perhaps more feedback could come in to the Symantec support system 
so that perhaps the issue would be given the proper treatment when it gets 
to the programming staff as a bugfix.


Should you be angry with AOL about this? No. Seriously. They make a VPNish 
adapter to tunnel you in to the network. They do everything in a pretty 
legitimate way, and because they use the same MAC address from version to 
version it is pretty simple to code an exclusion for that MAC address. One 
could argue that AOL should be using unique MAC addresses, but imagine all 
the MAC addresses that would be burned up by every single install of AOL on 
every OS using up a MAC address. It would be very wasteful.

Powered by blists - more mailing lists