I am detecting and blocking outbound fragmented TCP packets with the DF flag set at my perimeter firewall. The only time this occurs is when Comodo Internet Security is updating. The IP addresses are traceable to comodo.com so I am making an educated guess that that these packets are originating from the Comodo Internet Security application. Is this a bug or a feature ?
pf firewall logs attached
Stephen
[attachment deleted by admin]
Hi Stepen,
Can you tell us a bit about your network setup?
I don’t see this behavior on a network capture here.
Dell Optiplex GX620 → Netgear GS108T switch → pfSense 2.0RC1 firewall (on PCEngines ALIX 2d3) → Draytek Vigor 120 ADSL modem
Model Name: GS108T
Model Name: Hardware Version 1.0.0.1
Boot Version: V1.0.0_02
Software Version: 3.0.4.7
The only fragmented packets detected are those directed at comodo.com and cachefly.net when CIS updates. It may be possible that the GS108T is fragmenting the packets - connecting my computer directly into the pfSense firewall with a crossover cable will show if that is an issue. I shall try a crossover cable at the earliest opportunity.
Stephen
With my PC connected directly to the pfSense firewall with a crossover cable the fragmented TCP packets were still being logged on the firewall. That excludes the Netgear GS108T from being the culprit.
Stephen
Hi i think there is a device with a smaller MTU on the path between your PC and the update servers.
To be sure we’d need a packet capture with tcpdump/wireshark to see where fragmentation starts.
And possibly where the PMTU is taking place, are you blocking PMTU at the perimeter?
Run either:
ping -f -l (mtu packet size) destination - From the Windows box
or
ping -s (mtu packet size) destination - From the Linux box
Start with a packet size or 1472 (this plus 28bytes overhead) gives the optimal mtu of 1500 bytes. The mtu size will depend on your connection type.
ping -n 1 -f -l 1465 comodo.com
Pinging comodo.com [91.199.212.176] with 1465 bytes of data:
Packet needs to be fragmented but DF set.
Ping statistics for 91.199.212.176:
Packets: Sent = 1, Received = 0, Lost = 1 (100% loss),
ping -n 1 -f -l 1464 comodo.com
Pinging comodo.com [91.199.212.176] with 1464 bytes of data:
Reply from 91.199.212.176: bytes=1464 time=56ms TTL=48
Ping statistics for 91.199.212.176:
Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 56ms, Maximum = 56ms, Average = 56ms
1464 is my limit. Nothing shown on the firewall logs.
As you think it’s not CIS and I know it’s originating from my machine I have been into device manager and deleted the (hidden) remnants of AVG, McAfee and Symantec antivirus/firewall drivers that should have been deleted when the relevant products were uninstalled but have been accumulating over the last 6 years. I’ve had one CIS update so far and there has been nothing on the pfsense firewall logs. Hopefully the issue is resolved.
Just checked the firewall logs. Still getting fragmented packets with the DF flag set (to download.comodo.com and vip1.lon1.cachefly.net).
zip file attached with the pcap output of wireshark for the duration of the CIS update event (this includes other traffic) and the pfsense output for same time period showing the fragmented packets with DF flag set.
[attachment deleted by admin]
Yes but the traffic to your pfsense also has DF flag set… I don’t assume it needs to frag on the LAN is this still the crossover link? are you running jumbo-frames on the switch?
Seems DF bit is set by default on TCP traffic, only way to disable that is to disable PMTU discovery in the system registry but that will probably brake connections for your setup.
Jumbo frames are not being used on the switch. (The crossover cable was only used to eliminate the switch).
I have located the port mirroring facility on the GS108T switch. If the mirroring works correctly then using wireshark on another machine should determine if the packets “on the wire” are fragmented (or not) with the DF bit set. I’ll try this.
In that case mirror the FW port so the traffic has already been handled by the switch once.
pfsense firewall/gateway → Netgear GS108T switch → CIS PC
Netgear GS108T switch
port 4: pfsense firewall/gateway : mirrored to port 8
port 7: CIS PC
port 8: syslog server/wireshark monitor
Attached are syslog and wireshark pcap files that end with two fragmented DF flag set events. Checking the complete syslogs for a six hour period only shows comodo.com directed packets being blocked for being fragmented with the DF flag set.
[attachment deleted by admin]
There is a part missing here, if you look at the first src port the firewall reports blocked it’s 1058.
Only thing in the trace is FIN,ACK for that socket.
Same for 1059 and 1162 doesn’t show at all in the capture.
The wireshark log showed everything that went through the (mirrored) HWFW switch port for the period of that log. If anything is missing then either it was sent outside the log period or it never made it to the switch port.
I could ping the CIS machine from a remote machine on the internet with the same response as pinging from the local subnet. I was not detecting any blocked ICMP packets at the HWFW.
I left my Mozilla Thunderbird mail client running and detected the same fragmented packets with DF flag set when using IMAP with SSL/TLS when the mailbox was checked automatically. This was reproducible. So I uninstalled CIS. The symptoms disappeared. So something on my PC was causing the problem. I had forgotten that I have Spybot S&D (and SpywareBlaster) installed, so maybe there was an interaction there.
Given up on this topic. Pity as I only want a firewall to stop applications getting out, but I do not want one of the other firewalls with an overly simplified interface. Currently using Agnitum’s Outpost free which does what I want but is hobbled - no problems in more than 24 hours.
It looked based on the ports used as traffic that had originated before the capture, only the last part of the session showed up and was possibly flagged as DF the FIN,ACK’s don’t reach the destination because your HWFW drops them so the PC still thinks the session is half-open.
I could ping the CIS machine from a remote machine on the internet with the same response as pinging from the local subnet. I was not detecting any blocked ICMP packets at the HWFW.Well next to ping (icmp echo, request/reply) there are a bunch of other ICMP message types that are used for error control, so e.g. if you send out a UDP packet to a DNS server and that server has no DNS service active the system will send you a ICMP port unreachable message so to know it's unreachable you need to allow ICMP error traffic in to your local network else your system will keep resending etc...
Same goes for PMTU detection you send out TCP and it could happen that a return ICMP message appears or tries to appear but is blocked by your HWFW leaving your PC in the dark of the smaller MTU size requested.
I left my Mozilla Thunderbird mail client running and detected the same fragmented packets with DF flag set when using IMAP with SSL/TLS when the mailbox was checked automatically. This was reproducible. So I uninstalled CIS. The symptoms disappeared. So something on my PC was causing the problem. I had forgotten that I have Spybot S&D (and SpywareBlaster) installed, so maybe there was an interaction there.Are you saying once CIS is installed it forces your system to use DF? and when you uninstall it the problem disappears?
Given up on this topic. Pity as I only want a firewall to stop applications getting out, but I do not want one of the other firewalls with an overly simplified interface. Currently using Agnitum's Outpost free which does what I want but is hobbled - no problems in more than 24 hours.If you wish to keep CIS you can also set your MTU size on you system smaller to fit the size needed, you will only lose a bit performance on large downloads of course in that case.