Author Topic: How to block "Agents" when they without "Agents"(Empty)?  (Read 310 times)

Offline darkbear

  • Newbie
  • *
  • Posts: 16
How to block "Agents" when they without "Agents"(Empty)?
« on: September 14, 2022, 04:29:28 AM »
I see some log in my server and their Agents are empty

235.47.1.13 - - [14/Sep/2022:08:13:29 +0800] "\n" 400 3542 "-" "-"

how can I block them? thanks

Offline linux4me

  • Comodo Member
  • **
  • Posts: 32
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #1 on: September 14, 2022, 11:16:35 AM »
I created a couple of custom rules and used the following to block empty user agents:

Code: [Select]
# Block empty user agents.
    SecRule &REQUEST_HEADERS:User-Agent "[at]eq 0" "id:1,msg:'Blocked empty user agent for USERAGENT_IP',phase:2,severity:WARNING,capture,nolog,auditlog,logdata:'%{TX.0}',t:none,deny,status:406"
    SecRule REQUEST_HEADERS:User-Agent "^$" "id:2,msg:'Blocked empty user agent for USERAGENT_IP',phase:2,severity:WARNING,capture,nolog,auditlog,logdata:'%{TX.0}',t:none,deny,status:406"

You will need to set the "id" values to some not used by other custom rules.

Offline darkbear

  • Newbie
  • *
  • Posts: 16
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #2 on: September 14, 2022, 09:57:19 PM »
Thank you so much, if this is my first rule, I can use 1, 2 for id? thanks
« Last Edit: September 14, 2022, 10:28:39 PM by darkbear »

Offline linux4me

  • Comodo Member
  • **
  • Posts: 32
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #3 on: September 15, 2022, 10:53:23 AM »
Quote
if this is my first rule, I can use 1, 2 for id?

Yes, as long as you don't have another rule using the same ID.

Offline darkbear

  • Newbie
  • *
  • Posts: 16
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #4 on: September 17, 2022, 09:24:59 AM »
Yes, as long as you don't have another rule using the same ID.
Thank you so much

Offline darkbear

  • Newbie
  • *
  • Posts: 16
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #5 on: September 17, 2022, 09:50:40 PM »
Yes, as long as you don't have another rule using the same ID.
May I ask you one more question?
how can I block user-agent by keyword? because I find multi version(they always change user-agent version of masscan) of masscan is scanning my server, but I hate them. how can I block "masscan" or "zgrab" by only those 2 keyword?
Is that "Blocked Agents" support like *masscan* to block all user-agent with masscan keyword? or only add new rules to do that?

Thank you so much.
« Last Edit: September 17, 2022, 09:52:32 PM by darkbear »

Offline linux4me

  • Comodo Member
  • **
  • Posts: 32
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #6 on: September 18, 2022, 11:36:55 AM »
If you're using Comodo's plugin--I no longer do, so I can't look--there may be a way for you to add the keywords to the user agent blacklist via the UI, so you won't need to use the following, but here is how I used to do it using a text file blacklist and whitelist:

  • Create a file with the filename "userdata_bl_agents" using a simple text editor for the blacklist, including each keyword from the user-agent strings you want to block on a separate line.
  • Create a file with the filename "userdata_wl_agents" using a simple text editor for the whitelist, including each keyword from the user-agent strings that you want to whitelist.
  • Save the two files to /var/lib/mod_security/ (or the location of your choice, if you alter the path in the custom rules below).
  • Add the rules below.

In your case, user_bl_agents would look like this:
Code: [Select]
masscan
zgrab

A sample whitelist (user_wl_agents) might be:
Code: [Select]
bingbot
duckduckbot
facebook
google
msnbot
twitter

The custom rule:
Code: [Select]
# Block User Agents from your list in /var/lib/mod_security/userdata_bl_agents (blacklist)
# that are not in your /var/lib/mod_security/userdata_wl_agents (whitelist)
# Note: You switched from "drop" to "deny" because mod_http2's multiplexing
# gives you an error about being unable to close the socket.
SecRule REQUEST_HEADERS:User-Agent "[at]pmFromFile /var/lib/mod_security/userdata_bl_agents" \
"id:3,msg:'Blacklisted by userdata_bl_agents|%{tx.0}|%{tx.mode}',phase:2,chain,severity:WARNING,capture,nolog,auditlog,logdata:'%{TX.0}',deny,status:406,t:'none'"
SecRule REQUEST_HEADERS:User-Agent "![at]pmFromFile /var/lib/mod_security/userdata_wl_agents" "t:'none'"

After adding the rules, or updating the whitelist or blacklist in the future, you'll need to restart Apache for the changes to take effect.

You can test a user-agent from a terminal session by using curl:
Code: [Select]
curl -A "the user-agent string to test" https://thedomaintotest.com

The beauty of this approach is that you can get as radical with it as you like; for example, you could add the keywords "crawl", "spider", and "bot" to the blacklist as long as you whitelist any user-agents that contain those terms--like googlebot--that you want to allow.

Offline darkbear

  • Newbie
  • *
  • Posts: 16
Re: How to block "Agents" when they without "Agents"(Empty)?
« Reply #7 on: September 18, 2022, 11:03:09 PM »
Thank you for your help, I'm using directadmin's Comodo WAF.

Thanks for your solutions.

Thank you so much

 

Free Endpoint Protection
Seo4Smf 2.0 © SmfMod.Com Smf Destek