Working via command line

willdashwood · January 1, 2015, 2:03pm

Hello,

I’m new to using the free Comodo ModSecurity rules but so far I’m very impressed We used to use the free, delayed Atomic Security rules but we got a lot of false positives and found we had to disable more than we enabled! These rules seem a lot more specific and so far very few false positives reported.

I’m planning to deploy them across a large number of servers and so it would be easier if I could do a lot of the configuration required via the command line. I’ve been able to work out most things but does anyone know how I can do the following via the command line?

[ol]- Set “Schedule Rules Update” to daily

Enable all rules. By default it seems there are quite a few disabled.
Disable specific rules
List all rules that are currently disabled[/ol]

They’re all CentOS/CloudLinux cPanel servers by the way.

Also can I ask why some rules are disabled by default? Are these rules which have been flagged as potentially triggering false positives?

Many thanks,
Will

willdashwood · January 1, 2015, 4:02pm

I’ve figured out 2, 3 and 4. The WHM plugin references the following file:

/var/cpanel/cwaf/etc/yml/exclude.yml

So I can amend that file as required and then just save changes in the GUI to have them applied. Or could also amend the following file which is what ModSecurity uses.

/var/cpanel/cwaf/etc/httpd/global/zzz_exclude_global.conf

oleg.tsygany · January 1, 2015, 4:51pm

Hi Will

I’ll try to answer your questions one by one

How to set “Schedule Rules Update” to daily?
In fact plugin interface just create cronjob for this task, so we can do the same.
Let assume Comodo Web Application Firewall installed to /var/cpanel/cwaf folder.
In this case update script will be located at /var/cpanel/cwaf/scripts/updater.pl
To run it at 12:00 AM (every midnight) we should add line
0 0 * * * /var/cpanel/cwaf/scripts/updater.pl 2>&1
to crontab file (in cPanel located at /var/spool/cron/root).

Enable all rules. By default it seems there are quite a few disabled. Also can I ask why some rules are disabled by default? Are these rules which have been flagged as potentially triggering false positives?
Yes some of rules disabled by default.
These are Bruteforce rules - disabled as experimental. Additionaly they used persistent storage which is broken in current version of mod_security.
Also disabled Incoming rules (related to attacks targeting OSVDB flagged resource) - because COMODO is not responsible for OSVDB and Outgoing - preventing reveal of information about server software - because of load to server I assume. Of course all these rules can be turned on by curious customer, but she/he should keep in mind these rules are experimental.

To enable all rules we should remove them from exclude list.
In fact there are two exclude file types. First are ended with .yml and used as database for plugin interface. Second are .conf files and contain exclude directives for Apache web-server.
First files reside in /var/cpanel/cwaf/etc/yml folder, second in /var/cpanel/cwaf/etc/httpd and subfolders.
To add/remove exclude changing of second type is enough. But in this case our changes will not be shown in plugin interface.

So if revealing changes in plugin interface is not important we can just edit Apache config file /var/cpanel/cwaf/etc/httpd/global/zzz_exclude_global.conf removing all rule IDs.

Otherwise let rename file /var/cpanel/cwaf/etc/yml/exclude.yml and next rules update will do the job.
Note. File /var/cpanel/cwaf/etc/yml/exclude.yml is database file containing list of globally disabled rules.
For domain-related excludes see files /var/cpanel/cwaf/etc/yml/DOMAIN:PORT_exclude.yml.

Disable specific rules
Again we can add rule ID to Apache exclude /var/cpanel/cwaf/etc/httpd/global/zzz_exclude_global.conf and restart Apache, or add rule to /var/cpanel/cwaf/etc/yml/exclude.yml and perform rules update.
Well adding rule to YAML is somewhat tricky. Maybe it’s great idea to write CLI interface for adding/deleting/listing excludes…
It’s better to study existing exclude.yml as example before. File structure is following:
Category_name:
Group1_name:
rule1_id: 0
rule2_id: 0
Group2_name:
rule3_id: 0
rule4_id: 0
So to add exclude we should add rule id in corresponding group/category…

List all rules that are currently disabled

It’s as easy as

cat /var/cpanel/cwaf/etc/yml/exclude.yml | grep ‘: 0’

Or look in Apache exclude list located at /var/cpanel/cwaf/etc/httpd/global/zzz_exclude_global.conf

I hope this long post will be useful to you
Will be glad to answer any questions you have.

Best regards, Oleg

willdashwood · January 1, 2015, 5:57pm

Thanks Oleg, very good explanation. It’s certainly of use to me and I’m sure others.

_0M0D0 · January 15, 2015, 4:47pm

Is there a FULL list of commands that you can share for Comodo WAF, for those of us who only use commands and do not work with cPanel?

Thank you!

oleg.tsygany · January 16, 2015, 9:19am

Sure

Command-line utilities located in script directory inside of CWAF install tree.
uninstall_cwaf.sh - self-explanatory
update-client.pl - script to update client
update-client - run client update
update-client -h - display script help
update-client -v - display plugin and rules version

updater.pl - script to update rules
updater -h - display script help
updater -v - display plugin and rules version, web-platform type
updater - run update in console mode (do not update catalog )
updater -w - run update in web-plugin mode (update catalog, rules scheme etc )

Also we planning to develop CLI utility to manage:
Exclude list for rules

add rule to global/per-domain exclude list
remove rule from exclude list
get list of excluded rules globally/per-domain
Disabled domains
disable mod_security for domain
get list of disabled domains
enable mod_security for domain

willdashwood · January 16, 2015, 9:21am

Hi Oleg

A proper CLI utility would be fantastic, especially as it then makes it easier to automate.

Thanks,
Will

vadim · January 16, 2015, 9:56am

We’ve added this feature to our TODO list, but we have a long wish list, so I can’t say you when it will be ready.

_0M0D0 · January 19, 2015, 5:24pm

oleg.tsygany post:6:

Sure

Command-line utilities located in script directory inside of CWAF install tree.
uninstall_cwaf.sh - self-explanatory
update-client.pl - script to update client
update-client - run client update
update-client -h - display script help
update-client -v - display plugin and rules version

updater.pl - script to update rules
updater -h - display script help
updater -v - display plugin and rules version, web-platform type
updater - run update in console mode (do not update catalog )
updater -w - run update in web-plugin mode (update catalog, rules scheme etc )

Also we planning to develop CLI utility to manage:
Exclude list for rules

add rule to global/per-domain exclude list

remove rule from exclude list

get list of excluded rules globally/per-domain
Disabled domains

disable mod_security for domain

get list of disabled domains

enable mod_security for domain

Thank you!

Those are great news.

vadim · February 3, 2015, 10:25am

New command-line utilities from Client version 2.3:

cwaf-cli.pl - new command-line tool for protection rules management:

Turn on/off all protection rules (mod_security) for domain.
Enable/disable rules by ID for domain.

Usage:

`./cwaf-cli.pl [arguments]

Arguments:

-h, --help - this help message
-g, --loglevel - set loglevel (1 - 10)
-v, --version - show client version
-l, --domain_list - show list of domains

Exclude rules:

-d, --domain - set domain for exclude operation (global exclude list if not specified)
-ea, --exclude_add [rule_ID1 rule_ID2…] - add rules to exclude list
-ed, --exclude_del [rule_ID1 rule_ID2…] - remove rules from exclude list
-el, --exclude_list - show list of excluded rules

Disable/enable mod_security for domains:

-dd, --disable_domain [domain1 domain2…] - disable mod_security for domains
-nd, --enable_domain [domain1 domain2…] - enable mod_security for domains
-dl, --disabled_list - show list of disabled domains

`

Examples:

Global disable of the rules by IDs: 230000, 230010

./cwaf-cli.pl -ea 230000 230010

Enable rule ID 210700 for domain “mydomain.com:8080”

./cwaf-cli.pl -ed 210700 -d mydomain.com:8080

Notes:

Command-line utilities located in script directory inside of CWAF install tree.
Domain name should be specified as it looks in plugin or result of “–domain_list” command

_0M0D0 · March 16, 2015, 9:12pm

I have recently installed COMODO WAF in a Debian box. Configured everything and its running perfectly.

I want to test the command-line utility (cli). So I run the command (using “mydomain.com” as an example, I am actually typing the correct domain in my Debian):

./cwaf-cli.pl --dd mydomain.com

I get the output:

disabling domains:mydomain.com
no domain found: mydomain.com

I have 3 domains installed in my server. So I want to check if they all are listed with the command “./cwaf-cli.pl --domain_list” and this is the output:

list of all domains:

As you can see, the list is blank. However, I already tested a SQL injection and CWAF indentified it meaning that it is working.

Why I cannot display the list of domains installed in my server, and most important of all why I cannot disable a domain when I specify in the command-line?

akabakov · March 17, 2015, 8:48am

Unfortunately now you can get domain list if you use some WHM-system such as Plesk, cPanel etc. only.

To exclude domain in standalone mode you need create file
/<path_to_cwaf>/cwaf/etc/httpd/domains/domain.name.conf which contains:

SecRule SERVER_NAME “(?:..)?www.domain.name(?::80|:443)?|(?:..)?ftp.domain.name(?::80|:443)?|(?:..)?ipv4.domain.name(?::80|:443)?|(?:..)?mail.domain.name(?::80|:443)?|(?:..)?webmail.domain.name(?::80|:443)?|(?:..)?ns.domain.name(?::80|:443)?|(?:.*.)?domain.name(?::80|:443)?” “phase:1,nolog,noauditlog,allow,ctl:ruleEngine=Off,id:10001”

We are going to add domain exclude possibility for standalone mode in the neatest future.

oleg.tsygany · March 17, 2015, 1:32pm

Try attached script on your risk
There is ‘-f’ option now available to force apply domain.

[attachment deleted by admin]

_0M0D0 · March 17, 2015, 6:21pm

Does the following mean that the “nearest future” is already here?

Please advice.

oleg.tsygany · March 17, 2015, 8:01pm

Hi

Just extract this script to CWAF script folder and run:

./cwaf-cli.pl -f --dd mydomain.com

This will exclude domain even if it not found by plugin.

This also will work for rules disabling/enabling for domain, f.e.

./cwaf-cli.pl -f --ea 213440  -d mydomain.com

Will exclude rule with id 213440 for domain mydomain.com.

oleg.tsygany · March 19, 2015, 12:08pm

Yes, it’s released in CWAF 2.5

_0M0D0 · March 19, 2015, 3:14pm

GRRRRRREAT! I have just updated using the “update-client.pl”. It updated just fine.

I have used the -f option to disable mod_security for a specific domain and it worked!
Thank you for the quick solution to this.

I have two questions and one suggestion for this new version.

1. Question #1:
As mentioned, I have used -f to disabled a domain. I know the domain is listed in the file cwaf/etc/yml/disabled.yml. There I can see the DISABLED domains.
How can I see the ENABLED domains list? Is there a YML file I can access to see the available domains, since “cwaf-cli.pl --domain_list” doesn’t work?:

list of all domains:
Not available for standalone installation

2. Question #2:
How can I, after initial install of CWAF, disable mod_security for ALL domains and start manually adding the domains I only want to enable mod_security to?

3. Suggestion:
Currently, for me the option “-nd” doesn’t make any sense (e.g. cwaf-cli -nd domain.com). This option is used to Enable Domain to use mod_security. What would make sense is to use instead the option “-ed” (Enable Domain). I understand this option cannot be used since “-ed” is already being used by the exclude rule commands.

So, my suggestion is… why not to simplify the command options and use D for domains and X for exclude rules? So the commands will be more organized and would make more sense:

The output of the following command would be: cwaf-cli.pl -h

Arguments:
-h, --help         - this help message
-g, --loglevel     - set loglevel (1 - 10)
-v, --version      - show client version
-l, --domain_list  - show list of domains
-f, --force_domain - apply domain even if it not found

 Exclude rules:
  -d, --domain - set domain for exclude operation (global exclude list if not specified)
  -xa, --exclude_add [rule_ID1 rule_ID2...] - add rules to exclude list
  -xd, --exclude_del [rule_ID1 rule_ID2...] - remove rules from exclude list
  -xl, --exclude_list  - show list of excluded rules

 Disable/enable mod_security for domains:
  -dd, --domain_disable [domain1 domain2...] - disable mod_security for domains
  -de, --domain_enable [domain1 domain2...] - enable mod_security for domains
  -dl, --disabled_list  - show list of disabled domains

As you can see the “Exclude rules” commands start with X and the following letter denotes the action (A for add, D for del, L for list).
“Disable/enable mod_security for domains” commands start with D and the second letter is the action (D for disable, E for enable, L for list).
This way each command will be easier to remember, commands are more organized (hence they make more sense now), and standalone installation will be more user friendly.

This is my humble opinion and suggestion. :azn:

oleg.tsygany · March 19, 2015, 4:26pm

Hi

Thank you for warm words.

This is a tricky part. CWAF plugin get domains information form web-panel (cPanel, Plesk, DirectAdmin).
To acquire such information for standalone mode we have to scan Apache configuration files/directories.
This is far beyond our abilities for a moment
So right now we assume user knows his domains and can provide names with ‘-f’ options.

Did I already said ‘this is a tricky part’ ? Yeah, this is tricky part too
Net flooded with messages describing how to disable mod_security for certain domain, but no one for reverse task.
I’m afraid to enable mod_security for row of domains we should disable all other domains leaving only required.
Also manual solution proposed on Atomicorp Wiki (adding lines to vhost files):
Mod security - Atomicorp Wiki
This can work (of course adding lines ‘SecRuleEngine On’ instead of ‘SecRuleEngine Off’), if your system support vhosts files per domain.

Accepted fully
Changed command mnemonics to new one.

_0M0D0 · March 24, 2015, 9:18pm

oleg.tsygany post:18:

Hi

Thank you for warm words.

This is a tricky part. CWAF plugin get domains information form web-panel (cPanel, Plesk, DirectAdmin).
To acquire such information for standalone mode we have to scan Apache configuration files/directories.
This is far beyond our abilities for a moment
So right now we assume user knows his domains and can provide names with ‘-f’ options.

Did I already said ‘this is a tricky part’ ? Yeah, this is tricky part too
Net flooded with messages describing how to disable mod_security for certain domain, but no one for reverse task.
I’m afraid to enable mod_security for row of domains we should disable all other domains leaving only required.
Also manual solution proposed on Atomicorp Wiki (adding lines to vhost files):
Mod security - Atomicorp Wiki
This can work (of course adding lines ‘SecRuleEngine On’ instead of ‘SecRuleEngine Off’), if your system support vhosts files per domain.

Accepted fully
Changed command mnemonics to new one.

Great! Thanks for the excellent support and listening to your happy users!