DNS Benchmarking and DOS Detection

[Guest]

[ZBTSI]

I recently downloaded a couple of software tools, the intended purpose of
which is to benchmark certain parameters of DNS system performance and
security.  I think that it's great to have such tools available, and I would
certainly encourage their use for the intended purpose.

Nevertheless, one of the first things I noted during the use of these
tools is that they generate a considerable amount of network activity
for the duration of the test.  For the moment, and for lack of a better
term, I would suggest that these tools repetitively "bombard" the
servers with lookup requests, then extract certain characteristics based
on the nature of the responses.

Since I employ a fairly high-speed Internet connection, it immediately
occurred to me that the people who operate such servers, or run the
network I'm on, may or may not take a dim view of such network
activity; especially in light of all the DOS'ing that's going on nowadays.
Although my intentions are certainly benign, and the benchmarking purely
diagnostic in nature, might these tests not raise a red flag here and there
in the course of their use?

Further, if this network activity exceeds certain detection thresholds,
and subsequently triggers intervention on the part of an ISP, could
the results of such benchmarking be negatively affected?

With this in mind, I made a cursory examination of my ISP's terms and
conditions document, and discovered a particularly relevant section of their
network management policy, which I now reproduce here for your perusal:

Description of Network Management Practices, Performance, and Commercial Terms (Residential & Small Business Broadband Internet Access Services)

[relevant ISP] employs certain practices on a case-by-case and as-needed basis to protect its network and its customers against distributed Denial of Service (“DDOS”) attacks.  These practices (which could include limiting traffic to DNS and DHCP servers) could be triggered if [relevant ISP] detects traffic levels that significantly exceed certain baselines; the applicable thresholds are not disclosed here, in order to ensure that these security practices remain effective and cannot be deliberately circumvented.

I am curious to know how those that run DNS systems view the network
activity generated by such tests.  Similarly, I would be interested in hearing from
users who have considerable knowledge of the nature of DOS attacks, and
might be able to comment on the issue of ISP countermeasures, and the
potential influence of such countermeasures, on benchmarking tests.

Regards.

[Radaghast]

I would say, the amount of traffic generated by utilities like namebench or dnsbench is tiny in comparison to something like a DoS or DDoS attack. Moreover, an attack, for the most part, tends to be sustained, whereas the traffic generated by utilities are short lived.

To put it in perspective, most modern browsers support DNS-prefetching and in some cases link pre-fetching. if one visits a site with a large number off-site links, it's easy to generate dozens of DNS requests as well as generating additional traffic for link fetching.

It's also worth looking at the statistics for some of the well known DNS servers. For example you can look at the root name server statistics - C. ROOT - SERVERS .NET to get some idea of how many queries per second they're handling and compared to something like Google public DNS, these numbers are quite low.

I might also mention that Google advocate the use of namebench for DNS testing in the FAQ for the public DNS servers

[ZBTSI]

Hi there Radaghast.

I would say, the amount of traffic generated by utilities like namebench or dnsbench is tiny in comparison to something like a DoS or DDoS attack. Moreover, an attack, for the most part, tends to be sustained, whereas the traffic generated by utilities are short lived.

You're probably right about that, and your point is well taken.

Still, I wonder where some ISPs draw their line in the sand. Do they meter such
traffic on a network or server level, or is there even a difference nowadays?
And if they do throttle one's DNS usage, is it done on a dynamic, adaptive basis
(w DNS queries in x Seconds results in a limit of y queries for the next z minutes)?
Or do you just have to "spring the trap" once to get permanently throttled?

While not entirely analagous, I recently faced a similar situation with a large U.S.
West-coast ISP regarding my access to outbound port 25. In my case, their
network management system was neither dynamic nor adaptive, but certainly
reactive. One day, completely out of the blue, I received a mysterious e-mail
from the ISP, essentially stating "We've recently noticed an increase in use of
port 25 from your IP address, so we've blocked it. Don't even ask to get it back."

No, I wasn't spamming, nor was my machine infected.  I have a feeling that
the message was generated by an automaton, without any human intervention
required.

To put it in perspective, most modern browsers support DNS-prefetching and in some cases link pre-fetching. if one visits a site with a large number off-site links, it's easy to generate dozens of DNS requests as well as generating additional traffic for link fetching.

I really wasn't aware of this fact.  Guess I haven't been keeping up on current
events.  Thanks for that insight.

It's also worth looking at the statistics for some of the well known DNS servers. For example you can look at the root name server statistics - C. ROOT - SERVERS .NET to get some idea of how many queries per second they're handling and compared to something like Google public DNS, these numbers are quite low.

It looks like they handle about 20,000 to 25,000 queries per second.  
I'll admit, that's a lot.  One of the benchmarking tools is configurable.
So.. if you tell it to do 5000 lookups, and it does that over the course
of a couple minutes, even though the total number of queries can
be quite high, the load on any given server won't be all that great,
will it?

On the other hand, if your ISP is metering the number of UDP
connections to port 53 from a given IP address over a specific
period of time, that might raise a flag in their network management
software.

I might also mention that Google advocate the use of namebench for DNS testing in the FAQ for the public DNS servers

Personally, I never pay much attention to what Google advocate, or not.
Yet, I find this interesting.

Thanks again for your post!

Regards.

[Radaghast]

Hi there Radaghast.

You're probably right about that, and your point is well taken.

Still, I wonder where some ISPs draw their line in the sand. Do they meter such
traffic on a network or server level, or is there even a difference nowadays?
And if they do throttle one's DNS usage, is it done on a dynamic, adaptive basis
(w DNS queries in x Seconds results in a limit of y queries for the next z minutes)?
Or do you just have to "spring the trap" once to get permanently throttled?

I have no doubt ISPs employ some kind of 'throttling' technique on the number of queries per second, whether that's by using iptables rate limits or by using third-party software (I don't think Bind does this?) but the limit is going to be pretty high, especially for a large ISP. I also suspect the counter to DoS/SSoS  attacks, would be in countering amplification, by limiting outbound traffic.

As an aside, I mentioned in my earlier post that the statistics for the root servers were quite modest, well take a look at the numbers in the bottom right hand corner on this website

While not entirely analagous, I recently faced a similar situation with a large U.S.
West-coast ISP regarding my access to outbound port 25. In my case, their
network management system was neither dynamic nor adaptive, but certainly
reactive. One day, completely out of the blue, I received a mysterious e-mail
from the ISP, essentially stating "We've recently noticed an increase in use of
port 25 from your IP address, so we've blocked it. Don't even ask to get it back."

No, I wasn't spamming, nor was my machine infected.  I have a feeling that
the message was generated by an automaton, without any human intervention
required.

This really doesn't surprise me. Did you, by any chance, have a dynamic IP address?

It looks like they handle about 20,000 to 25,000 queries per second.  
I'll admit, that's a lot.  One of the benchmarking tools is configurable.
So.. if you tell it to do 5000 lookups, and it does that over the course
of a couple minutes, even though the total number of queries can
be quite high, the load on any given server won't be all that great,
will it?

It will take a lot more than a simple benchmarking utility like these. Normally, performance testing on DNS servers, which predominately run Bind, is done by using things like resperf or dnsperf.   

On the other hand, if your ISP is metering the number of UDP
connections to port 53 from a given IP address over a specific
period of time, that might raise a flag in their network management
software.

It might, but again, the number of queries would have to be substantially greater than the number generated by these simple tools. Personally, if my ISP contacted me in regard to testing DNS in this way, I'd go elsewhere.

[ZBTSI]

I have no doubt ISPs employ some kind of 'throttling' technique on the number of queries per second, whether that's by using iptables rate limits or by using third-party software (I don't think Bind does this?) but the limit is going to be pretty high, especially for a large ISP. I also suspect the counter to DoS/SSoS  attacks, would be in countering amplification, by limiting outbound traffic.

Sounds reasonable to me. Thanks for the clarification.

As an aside, I mentioned in my earlier post that the statistics for the root servers were quite modest, well take a look at the numbers in the bottom right hand corner on this website

Wow! Is that number for real?  It was hovering around 400,000 q/sec when I
viewed the page.  I actually have an account with OpenDNS, and have used
their servers, but I never noticed the counter on that page.

This really doesn't surprise me. Did you, by any chance, have a dynamic IP address?

Yes, that's correct.  It was a "sticky" dynamic address, which didn't change
very often. Whenever I wanted to, I could force it to change, but I rarely
had occasion to do so.

It will take a lot more than a simple benchmarking utility like these. Normally, performance testing on DNS servers, which predominately run Bind, is done by using things like resperf or dnsperf.  

That's good to know.  I'll feel much better about using the benchmarking
tools now.  

It might, but again, the number of queries would have to be substantially greater than the number generated by these simple tools.

Again, this is reassuring news. I really don't wish to get noticed in
that way by my ISP again.

Personally, if my ISP contacted me in regard to testing DNS in this way, I'd go elsewhere.

I guess that's always an option. Just how attractive that option
might be may depend upon where you live. Here in my part of
the U.S., broadband cable and DSL seem to reign, both run by
large corporations. How their network management policies might
differ is unclear. I suppose one might consider some of the public
and private wireless options becoming available. And, if it's even
still being marketed, there's satellite, which historically has been
spendy, plus you need a dish on the house.

In any case, at least in my area, the services offered seem to market
themselves.  No one has ever beaten a path to my door to try to
sell me any kind of Internet access whatsoever, and to see advertising
is rare as well.  We seem to be locked into an unhappy scenario in
which we need the ISPs more than they need us.  Yet, it' still nice
to know there are options out there.

Thanks for replying to my post.  See you in the next thread.

Regards.

[Tags:]