Common CSF/LFD False Positives and How to Stop The Notifications
CSF Notifications 101:
- Common Notifications from CSF/LFD
- Common CSF/LFD False Positives and How to Stop The Notifications
It is worrisome when you receive a notification from the firewall regarding a suspicious process, especially for those processes that you do not readily recognize. That is why KnownHost has compiled a list of some common alerts that are mostly false positives and provided the instructions for stopping the notifications. This list is specific to cPanel servers running CSF/LFD firewalls.
The good thing is that you don’t have to continue to receive these alerts. In fact, it is best to disable them so that when alerts are sent from the firewall, your inbox won’t be inundated with false positives making you less likely to notice those important alerts that could indicate something more serious.
To stop the notifications, all you need to do is add either the executable, the command, or the user (depending on the type of notification received) to the firewall’s process ignore file located at /etc/csf/csf.pignore and then restart the firewall. You can either do this from SSH or via WHM.
To determine whether it is best to ignore the user, the executable, or the command, you should know a little bit about the process. If the process is one of those that will under run under its own user, such as postgres, then it should be fine to add the postgres username to the ignore file. The executable is usually best to whitelist when considering the executable versus the command, but if the executable changes often, you may be better to whitelist the command. One such case of this is with SpamAssassin (see spamd below).
Ignoring Processes Via SSH
You would first log into the server as the root user via SSH:
ssh root@hostname -p2200
Be sure to replace hostname in the previous command with your actual IP address or hostname (I prefer the IP just in case the A record is not set or set incorrectly for the hostname).
Next, edit the process ignore file:
nano /etc/csf/csf.pignore
For some of these alerts, like the ‘spamd child/mailman’ alert, an entry already exists in the file, but is disabled. In this case, just locate the entry and remove the leading ‘#’. If an entry does not exist in the csf.pignore file, you will need to scroll to the bottom of the file and add it there. Then, you will hit Ctrl X + y + Enter to exit and save.
Lastly, restart the firewall so that these edits take effect:
csf -ra
Now, you should no longer receive notifications regarding the false positive.
Ignoring Processes Via WHM
Login to WHM at the following URL (replace hostname with your actual hostname or your server IP):
https://hostname/whm
Search and select the option “ConfigServer Security & Firewall” from WHM
Locate and click “csf.pignore – Process Tracking” if you are using an older version of CSF/LFD. If your version is current, you will select the csf.pignore file from the drop-down menu located under the ‘lfd’ tab as shown below and then click “Edit”:
Add the line containing the executable, user, or command to the bottom of the list. In some cases, the entry may already exist but be commented out. If so, just locate the entry and remove the leading ‘#’ to enable the process ignore feature for it.
Now click “Change” and restart lfd using the “Restart lfd” button
The csf.pignore File and Common Processes That Trigger False Positives Alerts
spamd child
This particular notification is so common that CSF/LFD actually provides a disabled entry for it. It isn’t ignored by default, so one must configure the firewall to ignore it using the instructions above. For this process, just locate it in the list and remove the leading ‘#’, then restart the firewall. The line you will look for in the /etc/csf/csf.pignore file is as follows:
#cmd:spamd child
You just need to change it as follows:
cmd:spamd child
Now restart the firewall.
Jailshell
The following alert is one that you may not necessarily want to disable completely. The following is a cron executed via a user’s crontab that is using jailed shell. You may actually want to know about this process in case there is occasionally and error with the cron completing. Otherwise, it may be that you expect the process to take a while and this process is legitimate, but is causing your mailbox to be flooded with alerts. In that case, you would likely want to ignore the process.
cmd:jailshell (user) [init] ell -c /usr/local/bin/php /path/to/file/index.php cron
\\
Passenger
The following process is one that you would want to configure the firewall to ignore. The Passenger process is actually due to the installation of the Apache module Passenger. This module would be installed for serving NodeJS, Ruby, and Python applications via Apache. You would simply add the command to the end of the file and restart the firewall to disable this alert.
exe:/usr/bin/node
OR
Aug 4 23:34:43 xander lfd[4367]: *User Processing* PID:4251 Kill:0 User:scriptkittie Time:72087 EXE:/usr/bin/node CMD:Passenger NodeApp: /home/scriptkittie/tools
Litespeed
Depending on whether Litespeed is installed on CentOS or on Cloudlinux determines which entries should be added to the process ignore file.
CentOS servers with EasyApache 3:
cmd:lsphp
exe:/usr/local/bin/lsphp
CentOS Servers with EasyApache 4:
cmd:lsphp
pexe:^/opt/cpanel/ea-php\d\d/root/usr/bin/lsphp
pexe:^/usr/local/lsws/bin/lshttpd.*
Cloudlinux with EasyApache 3:
cmd:lsphp
exe:/usr/selector/lsphp
Cloudlinux with EasyApache 4:
cmd:lsphp
pexe:^/opt/alt/php.*/usr/bin/lsphp
Cloudlinux servers with CageFS enabled:
cmd:lsphp
pexe:^/opt/cpanel/ea-php\d\d/root/usr/bin/lsphp\.cagefs
ruby
If you install Ruby on your EasyApache 4 cPanel server, it will cause firewall alerts unless you configure the firewall to ignore it. You will add the executable below to accomplish this:
exe:/opt/cpanel/ea-ruby24/root/usr/bin/ruby
awstats
This is another entry that exists by default in the process ignore file, though disabled. It simply needs to be enabled by removing the leading ‘#’ from each entry in the csf.pignore file and then issuing a firewall restart:
pcmd:/usr/bin/perl /usr/local/cpanel/3rdparty/bin/awstats\.pl.*
pcmd:/usr/bin/perl /usr/local/cpanel/base/awstats\.pl.*
nagios
If you install Nagios on your server, the following can be used to stop the firewall process alerts regarding it:
user:nagios
exe:/usr/sbin/nrpe
cmd:/usr/sbin/nrpe -c /etc/nrpe.cfg -d
MailScanner
MailScanner is yet another default entry that is disabled, but can be enabled by removing the leading ‘#’ and restarting the firewall.
pcmd:MailScanner:.*
cpanellogd
This is another one that the firewall foreseen as a legitimate false positive and has added the disabled entry for us. We just need to remove the leading ‘#’ from the entry to enable it and restart the firewall.
pcmd:/cpanellogd - (http|ftp) logs for .*
mailman
Mailman is another one that could be one of the false positives that the firewall developers predicted we’d want to ignore, so they’ve added the entry for us. Just locate the following entries and remove the leading ‘#’s from each and restart the firewall:
pcmd:/usr/local/cpanel/3rdparty/bin/python /usr/local/cpanel/3rdparty/mailman/bin/qrunner.*
pcmd:/usr/local/cpanel/3rdparty/bin/python /usr/local/cpanel/3rdparty/mailman/bin/mailmanctl.*
nginx
If you’ve installed Nginx on your server, you will need to manually add the process ignore for it. Your path may differ from the one below depending on how you installed the service as there is no support for Nginx at this time via cPanel, but the process is the same. Add the executable, command, or user and restart the firewall.
exe:/usr/local/sbin/nginx
memcached
Memcached is often manually installed as no official cPanel support exists for it yet, though it is currently in the Experimental EA4 repositories. Adding the following to the csf.pignore file and issuing a firewall restart should stop the LFD notifications concerning it:
exe:/usr/bin/memcached
rsync (via backup plugins)
Rsync is often used by backup plugins to back up a site. Personally, I would prefer a cPanel backup over a site backup made via plugin for many reasons, but, if for some reason you decide you must continue to use the backup plugin, you may find that you want to disable alerts regarding the rsync process. If so, you can add the following to the csf.pignore file and restart the firewall to accomplish this:
exe:/usr/bin/rsync
redis
Redis caching can be ignored via the firewall by adding the following entries to the csf.pignore file. Do note that, if you’re Redis installation is using different specifications, that you may need to actually examine the alerts sent and add that command to the csf.pignore file instead of the command below:
exe:/usr/bin/redis-server
cmd;/usr/bin/redis-server 127.0.0.1:6379
sftp
Whether or not to add SFTP to the process ignore file is more or less a matter of personal preference. If you’d rather be alerted every time one of your users on your server is uploading/downloading files via SFTP, then you wouldn’t want to add this to the process ignore file. On the other hand, if you feel that the user wouldn’t have SFTP access if they weren’t already trusted, then you would add the following to the process ignore file and restart the firewall to ignore these alerts (ensure that you replace user, not usr, with the actual cPanel user’s name):
exe:/home/virtfs/user/usr/libexec/openssh/sftp-server
cmd: /usr/libexec/openssh/sftp-server
Node
If you’ve installed NodeJS on your server, you will need to configure the firewall to ignore it by adding the following executable to the bottom of the csf.pignore file and then restarting the firewall:
exe:/usr/bin/node
Python
If you have any Python applications running via Apache’s Passenger module, then you will likely start getting these false positives. You can ignore them by adding the command to the process ignore file and restarting the firewall.
Command Line: python /opt/cpanel/ea-ruby24/root/usr/share/passenger/helper-scripts/wsgi-loader.py
PHP-FPM
PHP FPM for a specific user can be ignored via the following (be sure to replace username below with the actual cPanel username), though you will want to check the user’s resource usage and make sure no abuse is occurring for their sites if these are sudden:
cmd:php-fpm: pool username
Before adding this line to the ignore file, you would need to check the scripts running under the user and confirm whether or not those scripts should be using so much resources and are legitimate or not.
A reason to whitelist a user’s PHP-FPM processes would be if the user has backup scripts that take a long time to run. I would advise only whitelisting such processes as long as certain conditions are met, including 1) runs frequent malware scans on the site, and 2) uses a web application firewall.
Varnish
Varnish caching daemon can often be ignored using the following entry, though your path may be different depending on the 3rd party installation instructions you used to install it:
exe:/usr/sbin/varnishd
Postgres
Depending on the age of your server, you may or may not need to add Postgres to the firewall’s process ignore file. You can check using the following command:
grep postgres /etc/csf/csf.pignore
If you don’t get any output from that command, or you get output that is preceded with ‘#’, then you will need to enable postgres to be ignored by either removing the existing ‘#’, or by adding the entry.
exe:/usr/bin/postgres
ClamAV
ClamAV is a malware scanner plugin in cPanel. If you choose to enable this, you will also need to ignore the process in the firewall. Since ClamAV runs under its own user, you can ignore the user:
user:clamav
If you’d rather ignore the executable or command, they are as follows:
exe::/usr/local/cpanel/3rdparty/bin/freshclam
cmd:/usr/local/cpanel/3rdparty/bin/freshclam --quiet --no-warnings
ElasticSearch
ElasticSearch is yet another common installation on cPanel servers that can cause false positives. This service also runs under its own user and can be added to the process ignore file via the username:
user:elasticsearch
If you prefer to use the executable or command, they are listed below for reference:
exe:/usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
cmd:/usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
Any Third Party Software That You Have Installed
Centovacast isn’t really a very common software that we see these types of alerts for. It is an example of a 3rd-party software installations that would cause LFD notifications but can be ignored via the firewall to stop such notifications. Here are the executable lines that had to be whitelisted in a recent Centovacast installation that caused such alerts to be sent:
exe:/usr/local/centovacast/liquidsoap/bin/liquidsoap
exe:/usr/local/centovacast/sbin/cc-comet
exe:/usr/local/centovacast/sbin/cc-web
exe:/usr/local/centovacast/sbctrans2/sc_trans
exe:/usr/local/centovacast/shoutcast2/sc_srv
exe:/usr/local/centovacast/sbin/cc-control
exe:/usr/local/centovacast/sbin/cc-appserver
The LFD alert you receive will contain the executable and the command line values that you would need to add to the process ignore list.
System Integrity has detected modified file(s)
This mostly is one of the false positives results of updates that have updated packages. One must first determine if this is the case before ignoring the email.
Please see the next section regarding md5sum comparison failures for more information on how to deal with these types of alerts.
The following list of files have FAILED the md5sum comparison test
This is not always a false positive, so please don’t automatically ignore this alert! This mostly is a false positive and the results of updates that have updated packages. One must first determine if this is the case before ignoring the email.
An example email may be one such as the following, which one may receive following an automatic cPanel update:
The following list of files have FAILED the md5sum comparison test. This means that the file has been changed in some way. This could be a result of an OS update or application upgrade. If the change is unexpected it should be investigated:
/usr/bin/ea-php56: FAILED
/usr/bin/ea-php70: FAILED
/usr/bin/ea-php71: FAILED
/usr/bin/ea-php72: FAILED
/bin/ea-php56: FAILED
/bin/ea-php70: FAILED
/bin/ea-php71: FAILED
/bin/ea-php72: FAILED
/usr/local/bin/ea-php56: FAILED
/usr/local/bin/ea-php70: FAILED
/usr/local/bin/ea-php71: FAILED
/usr/local/bin/ea-php72: FAILED
You must determine if these binaries were changed due to updates or via some other means. You can search the yum update log and the cPanel update logs for entries suggesting updates to these binaries just prior to these alerts having been sent.
The yum update log is located here:
/var/log/yum.log
The cPanel update logs are located here:
/var/cpanel/updatelogs/
The csf.fignore file and Legitimate Directory Content Alerts
CSF/LFD also contains a csf.fignore file that can be accessed the same way that the csf.pignore file is. This file’s purpose is to list directories that the LFD directory watching feature should ignore. For example, a recent cPanel update temporarily changed where PHPMyAdmin temporary files were stored. The case reads as follows:
Internal case CPANEL-23314 is open to address an issue in cPanel & WHM version 76 where accessing phpMyAdmin as a cPanel user leads to the creation of pma_template_compiles_$user files in the system’s /tmp directory. While this doesn’t lead to any direct issues with cPanel & WHM itself, it’s contrary to the behavior seen in applications such as Horde and Roundcube where temporary files are stored in the /home/$user/tmp/ directory.
Because CSF/LFD monitors the /tmp directory for suspicious files, this change resulted in false positives alerts with the subject “Suspicious File Alert” referencing files located under the directory matching the pattern “/tmp\/pma_template_compiles_*” where * represents the cPanel account username that accessed PHPMyAdmin via cPanel.
Since all of these files were being created under a directory used solely for them, and because nothing else requiring monitoring existed under that directory, ignoring the entire directory recursively was the best option. Doing so meant adding the process to the csf.fignore file like so:
echo "/tmp\/pma_template_compiles_*" >> /etc/csf/csf.fignore; csf -ra
This command added the process and restarted the firewall with the new rules.
Another similar case involved Litespeed’s stats plugins dumping to a dedicated /tmp directory /tmp/lshttpd/.rtreport. In this case, the following directory was ignored:
/tmp/lshttpd/.rtreport*
The csf.ignore File and Legitimate IP Blocks
This file is useful for IPs that you don’t want to bypass closed ports but you don’t want to block, either. For example, and PCI scan provider. You will need to add the IPs for the PCI Compliance testing company in your firewall to ensure they are not blocked for port scanning when scanning your site. Adding their IPs to csf.ignore instead of csf.allow ensures that they do not get blocked, but still can’t bypass closed ports, which would result in a failure of the PCI scan. If you were to add their IPs to csf.allow instead, they could still become blocked by certain LFD checks and would be able to bypass closed ports, which would likely lead to inevitable PCI scan failure.
Other Ignore Files
Other Ignore files that can be used in the case of far less common false positives include the following:
File | Purpose |
csf.logignore | regex to match logs to be ignored by LOGSCANNER |
csf.mignore | list of users and local IPs to be ignored by the RT_LOCALRELAY_ALERT |
csf.rignore | list of rDNS domains to be ignored by LFD process tracking such as bots |
csf.signore | list of files that LF_SCRIPT_ALERT will ignore |
csf.suignore | list of usernames that are ignored during the LF_EXPLOIT SUPERUSER check |
csf.uidignore | list of user ID’s (UID) that are ignored by the User ID tracking feature |
Here is some information that is useful when reading the table above:
- LOGSCANNER This feature will send out an email summary of the log lines ofeach log listed in /etc/csf/csf.logfiles
- RT_LOCALHOST_RELAY Feature that triggers for excessive email sent via /usr/sbin/sendmail or /usr/sbin/exim
- LF_SCRIPT_ALERT Alerts when a limit of how many cwd= path emails are sent within an hour is exceeded.
These are far less common and you are not likely to need to edit any of these listed in the table.
Other False Positives
There are obvious false positives, too, such as the login notifications that the firewall sends when you log in as the root user via SSH or or when you su from a cPanel user to the root user. You may also receive an alert regarding a block resulting from an authentication failure that you yourself are responsible for if you have forgotten a password.
Another common false positive results when an active site administrator or developer is working in many different services at once and has more connections open than is allotted via the firewall’s connection limit, CT_LIMIT. This can cause the administrator to become blocked for no apparent reason while actively working on the sites. Though the firewall recommends a setting of 300 for this limit (via the csf.conf file), a more conservative but sane restriction of 200 concurrent connections from a single IP is likely suitable for most.
The CT_LIMIT may also cause false positives when used in conjunction with sites that use Cloudflare if the Cloudflare IPs have not been added to the csf.ignore file. On busier sites, concurrent connections from Cloudflare proxy/CDN IPs may easily exceed this limit and result in a false positive blocks against Cloudflare IPs. This can cause sites to have trouble loading and is why KnownHost configures your firewall accordingly to permit common services such as Cloudflare.
Conclusion – CSF/LFD False Positives
Your firewall is a piece of software that must be configured to work accordingly. The approach taken with CSF/LFD is essentially a whitelist approach. You whitelist allowed processes, and then it sends warnings regarding any other processes that are found running and not on the whitelist (listed in the process ignore file). So, if you have recently installed a new service or daemon, and then receive an alert about this process that you know is legitimate, you may be able to safely whitelist it. If the process is one that you do not recognize, have it investigated.
Honestly, it is best to have your server support investigate any that you receive because it is common for malware to name itself after normally legitimate processes to try to hide itself. For this reason, if you receive an alert that you are not 100% sure about, ask us! An example would be a malicious Perl process running as /usr/bin/http. If you are just now starting to receive alerts, but Apache has been running on your server for months, then this would definitely be something to inquire support about.
Remember, KnownHost is here to help!