Today we had an issue were a server was continuously getting requests from IP addresses in a particular country. The requests were obviously automated and searching for PHP pages that didn’t exist within the application we were running.
However this did mean that apache was continually trying to process and log these requests, causing the server to slow.
We knew fail2ban should do what we wanted but weren’t quite sure how to implement it. So here is a quick run through of how to implement a fail2ban policy for continuous requests for pages that don’t exist.
First install fail2ban. Our setup is Debian so we just used apt-get
sudo apt-get update
sudo apt-get install fail2ban
In the current version of fail2ban, filters are added to the filter.d folder. The default install directory is /etc/fail2ban. We used the following commands to create a new filter
sudo nano apache-404.conf
The configuration of the filter requires a regex that will cause a fail and an optional regex that can be used to ignore requests. We just wanted a regex to block requests so we only needed the fail regex.
failregex = ^ - .* "(GET|POST|HEAD).*HTTP.*" 404 .*$
This regex does a match on any get, post or head request that returns a 404 e.g.
XX.XXX.XXX.XXX - - [1/Jan/2020:12:30:00 +0000] "GET /url/path/file.php HTTP/1.1" 404 465 "-"
We also eventually refined it to only fail on requests for unknown PHP pages by adding it to the regex
failregex = ^ - .* "(GET|POST|HEAD).*php.*HTTP.*" 404 .*$
Save the file and then we can add it into the fail2ban configuration.
Now create a file to append configuration to the default configuration. The current configuration is stored in the jail.conf file and is automatically overridden or appended by jail.local
sudo nano jail.local
Add the following lines:
[apache-404] # label
enabled = true # enable the filter
port = http,https # ports to enable filter on
filter = apache-404 # name of the filter to use
logpath = /var/log/apache2/*access.log # the log file to do the regex check on
maxretry = 3 # number of failed before banning
bantime = 600 # how long to band for in seconds
ignoreip = # exempt IP addresses
We are getting the filter to check the apache access log but you could just as easily get to check the error log or any other log for a match. You can also easily update the number of allowed fails and how long to ban for.
Restart fail2ban and you should be ready to go. You can check the status of the filter using the following command:
sudo fail2ban-client status apache-404
If you called your filter something different then you exchange the name in the above command. This should show you a count of bans and currently banned IP addresses.