Spot attacks in web server logs and block them

Updated November 2026

Every public web server is probed all day by automated scanners. Most probes fail, but the same access log that records them also shows the one that worked: a 200 response to /.env, a POST to a PHP file in an upload folder. This guide shows how to read the log, find those lines quickly, and block the noisy sources.

Read an access log line

nginx and Apache both default to the "combined" format. nginx defines it as $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"; Apache's %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i" gives the same layout. Split on spaces, the useful fields are:

Combined log format fields (awk numbering)
Field	Contains	Example
$1	Client IP address	`203.0.113.45`
$6	Method, with a leading quote	`"GET`
$7	Path and query string	`/.env`
$9	Status code	`404`
$10	Response size in bytes	`153`

Default locations: /var/log/nginx/access.log for nginx, /var/log/apache2/access.log on Debian and Ubuntu, /var/log/httpd/access_log on RHEL-family systems. Malformed requests (raw TLS bytes sent to port 80, for example) shift the fields, so treat awk counts as a guide. If the server sits behind a proxy or CDN, $1 is the proxy's address unless you have configured the real client IP (nginx real_ip, Apache mod_remoteip).

What attacks look like

203.0.113.45 - - [03/Nov/2026:04:12:09 +0000] "GET /.env HTTP/1.1" 404 153 "-" "Mozilla/5.0"
203.0.113.45 - - [03/Nov/2026:04:12:10 +0000] "GET /.git/config HTTP/1.1" 404 153 "-" "Mozilla/5.0"
198.51.100.7 - - [03/Nov/2026:04:15:31 +0000] "POST /wp-login.php HTTP/1.1" 200 4521 "-" "Mozilla/5.0"
198.51.100.9 - - [03/Nov/2026:04:20:02 +0000] "GET /index.php?page=../../../../etc/passwd HTTP/1.1" 200 512 "-" "curl/8.5.0"
192.0.2.33 - - [03/Nov/2026:04:21:44 +0000] "POST /wp-content/uploads/2026/10/x.php HTTP/1.1" 200 18 "-" "Mozilla/5.0"

Secret file hunting: /.env, /.git/config, /.aws/credentials, /wp-config.php.bak, /backup.sql. A 404 is harmless noise; a 200 means the file was served.
Login guessing: many POSTs to /wp-login.php, /xmlrpc.php, /administrator/ or your own login URL from one address.
Path traversal: ../ or its encoded form %2e%2e%2f in the path or query string, often aimed at /etc/passwd.
Webshell use: requests (what a web shell is) to .php files inside upload, image or cache folders. Real sites rarely run PHP from there; a 200 with a tiny response is a strong sign of compromise.
Exploit strings: ${jndi: (Log4Shell), () { (Shellshock), UNION SELECT, <script>, or a user agent naming a scanner such as sqlmap or Nikto.

One-liners for quick triage

Run these as root (log files are usually not world-readable). Replace the path with your own log.

LOG=/var/log/nginx/access.log

# busiest client addresses
awk '{print $1}' "$LOG" | sort | uniq -c | sort -rn | head -n 20

# addresses collecting the most 404s (scanners)
awk '$9 == 404 {print $1}' "$LOG" | sort | uniq -c | sort -rn | head -n 20

# probes for secrets, admin tools and traversal
grep -Ei '"[A-Z]+ [^"]*(/\.env|/\.git/|wp-login\.php|xmlrpc\.php|phpmyadmin|/cgi-bin/|\.\./|%2e%2e|/etc/passwd)' "$LOG" | tail -n 50

# DANGER: sensitive files that were actually served
awk '$9 == 200 && $7 ~ /(\/\.env|\/\.git\/|\.sql$|\.bak$|wp-config\.php)/' "$LOG"

# PHP requested inside upload-type folders (possible webshell)
grep -Ei '"(GET|POST) [^"]*/(uploads|images|files|tmp)/[^" ]*\.(php[0-9]?|phtml|phar)' "$LOG"

# Log4Shell-style strings anywhere in the line
grep -F '${jndi:' "$LOG"

# include rotated and compressed logs
zgrep -Eih 'wp-login\.php|xmlrpc\.php' /var/log/nginx/access.log* | awk '{print $1}' | sort | uniq -c | sort -rn | head

A hit with status 200 is an incident, not noise. If /.env or /.git/ returned 200, assume every password and API key in it is stolen and follow Leaked credentials. If a PHP file in an upload folder answers, follow Website hacked.

Close the most common leak in the web server itself. For nginx, inside each server block:

# deny dotfiles such as .env and .git, but keep ACME challenges working
location ~ /\.(?!well-known/) {
    deny all;
}

For Apache, in the virtual host or main config:

<LocationMatch "/\.(?!well-known/)">
    Require all denied
</LocationMatch>

Test and reload: sudo nginx -t && sudo systemctl reload nginx or sudo apachectl configtest && sudo systemctl reload apache2 (httpd on RHEL).

See the big picture with GoAccess

GoAccess is a free log analyzer that turns an access log into a live terminal dashboard or a single HTML report: top addresses, paths, 404s, status codes and user agents. Install it with sudo apt install goaccess (Debian, Ubuntu) or from EPEL on RHEL-family systems.

# interactive terminal report (q to quit)
sudo goaccess /var/log/nginx/access.log --log-format=COMBINED

# HTML report from current and rotated logs, read from stdin
sudo sh -c 'zcat -f /var/log/nginx/access.log* | goaccess - --log-format=COMBINED -o /root/report.html'

The report contains visitor IP addresses. Keep it private; do not write it into the public web root.

Block repeat offenders with fail2ban

fail2ban reads logs and adds temporary firewall bans. Its stock jail.conf already has web jails, including nginx-http-auth, nginx-limit-req, nginx-botsearch, apache-auth, apache-badbots, apache-noscript and apache-botsearch. Do not edit jail.conf; enable jails in a .local file:

# /etc/fail2ban/jail.d/web.local
[nginx-http-auth]
enabled = true

[nginx-botsearch]
enabled = true

[nginx-limit-req]
enabled = true

Defaults are 5 matches within 10 minutes for a 10-minute ban. For probes that the stock filters miss, add a small filter for the access log:

# /etc/fail2ban/filter.d/web-probes.conf
[Definition]
failregex = ^<HOST> -[^"]*"(?:GET|POST|HEAD) [^"]*(?:/\.env|/\.git/|/\.aws/|/phpmyadmin|/wp-config\.php|\.\./)
ignoreregex =

# add to /etc/fail2ban/jail.d/web.local
[web-probes]
enabled  = true
port     = http,https
filter   = web-probes
logpath  = /var/log/nginx/access.log
maxretry = 2
findtime = 10m
bantime  = 1h

# check how many lines the filter matches before enabling it
sudo fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/web-probes.conf

sudo systemctl restart fail2ban          # RESTARTS fail2ban
sudo fail2ban-client status web-probes
sudo fail2ban-client set web-probes unbanip 203.0.113.45

Remove any path your site really uses (for example /phpmyadmin if you run it on purpose). Ban only on patterns no real visitor sends, and add your own office addresses to ignoreip so you cannot ban yourself.

nginx's limit_req module caps how fast one address can hit a location. Rejected requests get a 503 unless you set limit_req_status.

# in the http { } block
limit_req_zone $binary_remote_addr zone=login:10m rate=10r/m;

# in the server { } block
location = /wp-login.php {
    limit_req zone=login burst=5 nodelay;
    limit_req_status 429;
    # repeat your normal PHP handling here (include fastcgi_params; fastcgi_pass ...;)
}

That allows 10 requests a minute per address, plus a burst of 5. A person logging in sends 2 or 3; a password-guessing script sends hundreds and gets 429 errors. nginx also notes each rejection in its error log, where the nginx-limit-req jail turns repeat offenders into bans. For Apache, combine fail2ban with a WAF, or put the login page behind an IP allow-list or HTTP authentication.

Web application firewalls

A WAF (web application firewall) inspects each request and blocks those matching attack rules. Two free engines run the same rule set:

ModSecurity v3 (libmodsecurity): A library with a separate connector for nginx (ModSecurity-nginx); ModSecurity v2 remains available as an Apache module. The project now lives under the OWASP GitHub organization; Trustwave's sponsorship ended on July 1, 2024. Check your distribution's package search for the module.
Coraza: An OWASP project written in Go that supports ModSecurity's rule language and the OWASP Core Rule Set. It runs as a Caddy plugin, a Go library, a proxy-wasm extension (for Envoy) and, in preview, an HAProxy SPOE agent.

The OWASP Core Rule Set (CRS) supplies generic rules against SQL injection, cross-site scripting, path traversal and similar attacks. It uses anomaly scoring: each matching rule adds points (a critical match adds 5), and the request is blocked when the total reaches the inbound threshold, which CRS suggests should end up at 5. Paranoia level 1 of 4 is the baseline with the fewest false positives.

Start safe: the recommended ModSecurity config ships with SecRuleEngine DetectionOnly, which logs without blocking. CRS's own advice is to block with very high anomaly thresholds at first and lower them step by step.
Watch the audit log for a week and write exclusions for your application's false positives before you tighten.
Keep CRS updated like any other software; new rules follow new attack techniques.

A WAF buys time, but it does not replace patching the application. For host-wide detection and automatic responses (for example Wazuh's active response, which runs scripts such as firewall blocks when a rule fires), see Intrusion detection.