UDP Port Scan on HP all-on-one printers [CLOSED]

Having had some time now to think about what’s going on, given the Wireshark data, I’m starting to believe that there’s a difference between textbook application (where everything works, perfectly) and real-world engineering (where things break). Rumor has it that there is a difference.

So, why, in this case, would using separate ports per query make sense?

Two things to keep in mind: its UDP traffic, and its SNMP, which is used to monitor presumably simple devices.

UDP is an “unreliable” protocol, in that it doesn’t have error detection and retry methods. There’s no guarantee the packet gets to the destination, or if it gets there, that any kind of answer will come back.

SNMP is a common data structure used as an overlay on all sorts of hardware with all sorts of capabilities. In its original incarnation, the device hardware was little more than some decoder logic, a hardwired UDP capability, and a register that could be queried and set by a UDP packet.

SNMP can make a very complicated and physically distributed “device” appear as a single entity. In reality, there may be a single controller device, and very distinct and physically distant subdevices. Consider a cable TV distrubution, with a neighborhood controller node, and the individual household interfaces at each home. One SNMP “device”, but physically distributed.

To query that device, the SNMP manager software will go thru and walk its data structure list. Simple to do. Textbook stuff.

Enter the real world. Things break, wires get unplugged, response times can differ from physical device to physical device.

If the query goes out by TCP, then the controller device has to have more smarts than simple decoder logic. That costs, so that won’t do at all. So UDP it is.

Query packet goes out to the controller device, to report the value of register foo. Okay, so where’s the answer. UDP doesn’t guarantee an answer, or even a timely answer. One device that’s damaged, destroyed, or frozen can’t hold up the queries to everything else. The port the query got sent from can’t be reused until you get an answer back. Answers don’t have tags saying what the answer is about. Query “show me register A”, wait wait wait wait, back comes “15”. If you’re lucky and everything is working.

So, real world requires one unique port per query. The time lag from query-out till answer-back is part of the diagnostics. No answer is a fail, slow answer is a problem. Atypical value, like “2” instead of “15”, may or not be a problem. That depends on the device. Look the number up in the specifications, as translated into the SNMP Management Information Base (MIB), and see what it says.

If the device MIB has a lot of registers to be queried, then it’s a lot of ports and a lot of timeout checking to make sure everything is working properly.

In the case here, the HP printer MIB seems to be large as in several hundred entries. Being directly attached on the LAN, probably with an on-board CPU doing its own diagnostics, the printer can respond almost instantly to any SNMP query that comes in. SNMP answers come back as a flood.

With this kind of SNMP description, it would seem that any SNMP capable device produced these days, especially that has its on on-board CPU, is going to reply similarly. That would seem to imply that CFP is going to complain about any SNMP capable device that has a MIB over a certain size, that has device management software installed on the PC where CFP is installed.

I won’t characterize this as a CFP bug, but a change in the flood and scan checking to ignore LAN traffic on UDP 161 would seem to an obvious solution.

And, my apologies to the HP code writers. They got it right. Real world engineering beats textbook code, if things are going to work reliably.

Nice reply, Grue.

As a developer of ERP software, I’m constantly getting smacked in the forehead with real world. You would hope that programming would be black and white but there’s a lot of grey there too.

It sounds like at least you have a target now. That’s half the battle!

If you need more data from Wireshark, let me know. What I sent was less than one minute of capture. Imagine how much traffic in 24 hours:>)

Thanks…

Firstly, great reply. comprehensive, logical and, above all else, easy to follow.

An alternative method would be to excluse specific IPs on the local segment from the UDP flood checking. Which would be quicker, checking the IP or checking the destination port?

Ewen :slight_smile:

Being a windump/tcpdump person, I don’t have that much practice with Wireshark as an application. After installing it, and playing around some, I’ve found that it comes with some very nice tools. One in particular being a command-line version, called tshark (assuming the standard install directory c:\program files\wireshark\tshark.exe). Command line stuff is great, as you can set it up, and just let it run. It just takes a few more steps.

First, find out what interface you want to run against. Enter “tshark -D”. This will give a numbered list of adapters that tshark will run against. Typically some generic is first, and the Ethernet adapter is second. The number is the interface identifer to monitor, eg " -i 2".

For a long term monitor, try this as a command line

tshark -i 2 -S -w c:\capture\this -b files:5 -b filesize:1024 “host 192.168.1.50”

tshark command options are in the html file tshark.html, in the install directory. Summarizing the options here:

-i 2 presumably the Ethernet interface is the second device in the -D list
-S show the capture, so you can have some idea of what’s going on
-w c:\capture\this write packet information to a file “this”
-b files:5 -b filesize:1024 creates a file buffer ring, of 5 files to a max filesize of 1024 kbytes
“host 192.168.1.50” the capture filter, with the quotes, in tcpdump syntax. In this case, capture all traffic to or from the printer. The filter specification is the last parameter, and having multiple words with spaces, has to be in quotes to keep it all together.

Fire that up in a command window, and let it run. Using a file buffer ring, you won’t overrun your disk space, and you’ll have only the most recent traffic for the printer.

The capture file are binary, the first 80 bytes or so of each packet. A “tshark -r” can read the capture and can apply selection filters to pull out particular packet information. A “Wireshark -r” should do the same thing.

To see what’s going on long-term, this is probably about the best way to find out.

Quicker? That would depend on how the CFP code is written. Checking an IP address is the simpler method, but just an IP address check opens a lot of doors for things to get spoofed thru. If an IP address where there is an already established outbound, then the opportunity to bypass the checking gets narrowed considerably. If this SNMP traffic is indeed the port scan trigger, then there is already an established outbound (each response is to a PC initiated query, with a corresponding “open” UDP port awaiting a reply). That would mean the CFP flood and port scan check isn’t looking for matching traffic, but simply comparing packet intervals. Not a bug, but not an entirely reliable method either. Another case of the real-world complicating the code.

As an aside, I’m presuming the LAN is a 100Base-T. The CFP flood thresholds, now that I look at them a little more closely, seem consistent with 10Base. One of the LANs on dayjob is getting upgraded to 1000Base, so the threshold numbers would need to be adjusted accordingly. Maybe thresholds set per interface adapter would make sense, although multiple interfaces are not entirely consistent with a desktop environment.

Grue155,

I ran the command line test after reinstalling Comodo to match up times for the UDP port scan block. There’s a block in the log:

Date/Time :2007-10-31 19:19:30
Severity :High
Reporter :Network Monitor
Description: UDP Port Scan
Attacker: 192.168.1.50
Ports: 2828, 3084, 4364, 62731, 62987, 63243, 63499, 63755, 64011, 64267, 64523, 64779, 65035, 65291, 12, 268, 524, 780, 1036, 1292, 1548, 1804, 2060, 2316, 2572, 10352, 59478, 65374, 65535, 21899, 33548, 6242, 65024, 9026, 17027, 9312, 3723, 5631, 35324, 63344, 23902, 2242, 52224, 52428, 52428, 65419, 35669, 33772, 5356, 17803
The attacker has been temporarily blocked

And the attached Wireshark file being written to during the block. Look at the log around frame 9370 (time was 19:19:29:983989000).

There was an earlier block, contained in the same file, around frame 3644.

Date/Time :2007-10-31 19:17:25
Severity :High
Reporter :Network Monitor
Description: UDP Port Scan
Attacker: 192.168.1.50
Ports: 35602, 35858, 36114, 36370, 36626, 36882, 37138, 37394, 37650, 37906, 31762, 32018, 32274, 32530, 32786, 33042, 33298, 33554, 33810, 34066, 34322, 34578, 34834, 35090, 35346, 49152, 21331, 17805, 20720, 5631, 35116, 63344, 18059, 33632, 9448, 19853, 35312, 8264, 19851, 35592, 51158, 7232, 30526, 63344, 16582, 57347, 5631, 35324, 63344, 829
The attacker has been temporarily blocked

Hopefully the information is helpful.

Using the default settings on the Attack and Detection dialog except that I reduced the time to leave the port blocked down to 1 minute, I’m getting a block about every two minutes.

[attachment deleted by admin]

Oh my. 5 minutes worth of capture data, and it fills up a full 1025k file, with just under 10 000 packets. For the benefit of anybody considering downloading the file, that’s what you get.

Two things stand out…

One, the HP monitor software is insane. It doesn’t seem to have an “off” switch. It queries and queries and queries and queries and queries. Just as fast as it can. And the printer, being a good little printer, answers every one of them. This kind of behaviour, I would classify as an HP bug against their director/manager software. It’s not a bug with the printer.

And two… Congratulations, it looks like you’ve tripped a CFP bug. I did a check of what ports are actually being used, and saw a rollover at frames 4167 and 4168. The highest port number in use is 5000, and rolls back around to 1025. CFP reporting ports above 5000, and below 1025, is just not right. I’m inclined to believe this is some kind of data overrun, and the ports listed in the CFP port scan log are actually some kind of memory dump. I need to sit back and think a bit more, but right now I’m strongly suspecting there’s going to be a PR attached to this topic.

Thank you for the data!

Quick thought, have you tried using packet thresholds of 300 packets/sec or higher for the flood and port scan checking? There’s some cutoff point where CFP shouldn’t log these scans, and its going to be a fairly high number, or some special value like 0 or -1.

I did try bumping the numbers up in the UDP Flood fields when I first began reporting, but in reality, I’m really not sure which ones to change. Is it just the UDP Flood Traffic Rate and Port Scan probing rate?

For the moment, I’d suggest treating the two settings as being the same thing. Once some kind of threshold gets established, then one of the values can be tweaked up or down to see what happens. I don’t know what interaction there is, if any, between the two settings. In trying to find something that works, it’s simpler to just treat them as one variable.

I’m on the end of my day, so I’ll have to pick this up tomorrow.

I just did an eyeball check for HP updates for “HP Photosmart 2575”. That shows there are a bunch of software updates, 6 listed as critical, 2 of which are diagnostics and performance related. The descriptions are remarkably uninformative about what the update is about. I don’t know if the PC installed software is at current version, so that would be something to check. At worst, opening up a trouble report with HP about excessive SNMP traffic might give some solutions from their perspective.

In reading back early in the topic, Soya mentioned the max threshold number supported is 2000. I’d suggest starting there, and moving the number down by factors of 2 (binary search) to find where the problem starts occuring. If the 2000 doesn’t turn off the warnings, then there is another CFP issue that may need a PR. The Wireshark capture is showing SNMP traffic in the 150/sec range.

I haven’t finished my eyeball of the CFP ports scan, but one things stood out very quickly: the ports differ by 256, in ascending sequence. Expressed in 16 bit hex format, that looks like either Unicode strings, or a memory management “reuse trap” pattern.

I make sure I’m up to date with the software. Plus, HP has as service running that checks as well. Speaking of which, while I’m typing this, a new one popped up for install.

I set the fields to 300 last night before getting off the computer for the night and there weren’t any port scans when I checked this evening. That’s about 20 hours. That being said, I rebooted my computer and upon startup a port scan showed up. But just one.

I’ll leave it like it is for a bit and see what happens. I’ll install the HP update now:>)

Having had some time to examine that set of ports listed, and having a hunch this could be a data overrun, I rewrote the port numbers in hex (knowing that port numbers are unsigned 16-bit integers). That list of ports becomes the following.

0b0c 0c0c 110c f50b f60b f70b f808 f90b fa0b fb0b fc0b fd1f fe0b ff0b 000c 010c 020c 030c 040c 050c 0606 070c 080c 090c 0a0c

2870 e856 ff5e ffff 558b 830c 1862 fe00 2342 4283 2460 0e8b 15ff 89fc f770 5d5e 08c2 cc00 cccc cccc ff8b 8b55 83ec 14ec 458b

That second line has a strong resemblence to x86 machine code. However, a search comparison of the files in the install directory doesn’t find any matches. So if it is excutable code, it is either dynamically produced, or has been modified by the program loading process. I haven’t tried doing any disassembly to confirm that is actually machine code, as I don’t have the tools to properly do a disassembly.

If nobody has any objections, I’ll put this in as a Comodo support ticket, and refer the description back to this topic. It’ll probably be this weekend before I get the opportunity to lodge the ticket.

Go ahead. We’re out of ideas. FYI the devs won’t reply until version 3 is finalized. Even then, I have a feeling that most tickets will be responded to upgrade to the new version to see if the issue is resolved.

I’ve lodged two support tickets on the things that have turned up. For anybody who’s interested:

Subject: Legitimate SNMP traffic identified as packet flood
Ticket: WUA-285063

and

Subject: Machine code exposed in report
Ticket: XXX-898548

Both tickets have pointers back to this forum topic.

Grue155,

Thanks again for all the time you spent on this. Hopefully the issue will get resolved.

CFP 3.0 RC1 got released a few days ago. And, as Soyabeaner points out, the most likely ticket feedback to a 2.4 problem is going to be “upgrade”.

This kind of SNMP packet storm would be a great test for 3.0 RC1, and it would give a bit of incentive to get issues resolved so far as CFP is concerned. At worst, it’s some new support ticket numbers, and at best, everything works as it should. Game?

Yeah, I can give 3.0 a shot.

Out of town until this weekend. I’ll see about downloading and installing then.

BTW, I set the UDP traffic and port probing rates to 350 and it seems t have stopped the port scan blocking.

There was a reply to the tickets in the email I looked at today. Same message for both tickets:

Date: Wed, 21 Nov 2007 17:57:00 +0000 Message-ID: Subject: [SUPPORT #XXX-898548]: Machine code exposed in report From: "Comodo Support" Reply-To: desktopsupport@ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit

Hi,

Sorry for the inconvenience caused.

Now You can download the latest release 3.0.13.268 from http://personalfirewall.comodo.com and uninstall the existing, restart the computer and install the latest.

It will fix your issue.

For more info, Please do refer our forum…
https://forums.comodo.com/feedbackcommentsannouncementsnews/comodo_firewall_pro_3_has_been_released-t14915.0.html

Regards
Malcolm
Technical Support

Looks like Soyabeaner’s earlier comment was pretty much square on.

If there’s no objection, I’ll go ahead in the next couple of days and mark this topic as closed. If v3 doesn’t resolve the problem, then we’ll have another go in the new v3 help forum.

Hi Grue,

I’ve just gotten 3.0 installed (3.0.13.268) and haven’t seen the port scan issue yet, although I haven’t been running it that long yet. So far I like it.

I am monitoring a VPN issue for 3.0 though.
https://forums.comodo.com/help_for_v3/cant_connect_with_xps_vpn_client_help-t15241.0.html

It specifies XP VPN, but I cannot connect using Nortel Extranet VPN.

I’ll try a couple more things and then comment in the 3.0 thread.

Thanks,

Alrighty, I’ll go ahead an mark this as closed, for CFP v2.4