Learning nginx - Customized Logging, JSON Logging & Log Rotation

#Introduction

NGINX logs are your eyes on the traffic passing through. But raw default logs are hard to analyze and expensive to store. This Episode 15 covers customized logging, JSON logging, and log rotation — how to turn logs into a structured observability asset.

You'll define your own log formats, produce JSON access logs ready to send to the ELK Stack, Grafana Loki, or Fluent Bit, conditionally ignore unimportant requests, and manage log file rotation with logrotate.

#Customizing the Log Format

#Important Variables in the Access Log

NGINX's default format stores $remote_addr, $request, and $status. But many more useful variables are available:

• $remote_addr — the client IP.
• $request — the full request line.
• $status — the status code.
• $body_bytes_sent — the response size.
• $http_referer — the origin page.
• $http_user_agent — the client's user agent.
• $request_time — the request processing duration.
• $upstream_response_time — the backend response time.

#Creating a Custom log_format

Declare the format in the http context:

http {
    log_format combined_ext '$remote_addr - $remote_user [$time_local] '
                            '"$request" $status $body_bytes_sent '
                            '"$http_referer" "$http_user_agent" '
                            'rt=$request_time';
 
    access_log /var/log/nginx/access.log combined_ext;
}

log_format combined_ext defines a new format named combined_ext. The access_log directive then uses it. Each field is separated by a space, and variables containing spaces are wrapped in quotes.

#JSON Formatting Access Logs

#JSON Logs for SIEM and Log Analytics

Modern observability pipelines almost always use JSON because it's easy to parse. A JSON format in NGINX is declared with the same pattern:

http {
    log_format json_combined escape=json
        '{'
            '"time_local":"$time_local",'
            '"remote_addr":"$remote_addr",'
            '"remote_user":"$remote_user",'
            '"request":"$request",'
            '"status":$status,'
            '"body_bytes_sent":$body_bytes_sent,'
            '"request_time":$request_time,'
            '"upstream_response_time":"$upstream_response_time",'
            '"http_user_agent":"$http_user_agent",'
            '"http_referer":"$http_referer"'
        '}';
 
    access_log /var/log/nginx/access.log json_combined;
}

escape=json makes values with special characters escaped correctly. After a reload, every access log line is a valid JSON object that Fluent Bit, Filebeat, or promtail can consume directly.

Check the result:

sudo tail -1 /var/log/nginx/access.log

tail -1 /var/log/nginx/access.log shows the last line — it should be a JSON object with all the fields above.

#Conditional Logging

#Ignoring Static and Health Check Requests

Requests to static assets and health checks fill the log with useless data. Turn off logging conditionally with map:

http {
    map $request_uri $loggable {
        default 1;
        ~\.(js|css|png|jpg|gif|ico|woff2)$ 0;
        ~^/healthz 0;
        ~^/metrics 0;
    }
 
    access_log /var/log/nginx/access.log combined_ext if=$loggable;
}

map fills the $loggable variable: it's 0 for static and health check URIs, 1 for everything else. The access_log ... if=$loggable; syntax only writes a log when the variable's value is truthy. The result: log disk usage drops drastically and analysis becomes more focused.

#Log Rotation

#Managing with logrotate

Without rotation, log files grow endlessly until they fill the disk. Distributions provide logrotate with a default configuration for NGINX:

ls /etc/logrotate.d/
sudo cat /etc/logrotate.d/nginx

The default configuration usually rotates logs weekly, keeps several versions, and calls nginx -s reopen after rotation so NGINX opens a new log file.

#Testing Rotation Manually

Run rotation manually to confirm the configuration works:

sudo logrotate -d /etc/logrotate.d/nginx

logrotate -d runs in debug mode: it shows what would be done without actually rotating the logs. If the output is clean, the automatic weekly rotation via cron will run safely.

#Conclusion

Episode 15 turned logs into an observability asset: you can define custom formats, produce JSON access logs for SIEM, disable logging for unimportant requests, and rotate logs regularly with logrotate.

Key takeaways:

• log_format defines custom log formats with NGINX variables.
• A JSON format with escape=json is ready for ELK, Loki, or Fluent Bit pipelines.
• map and if=$loggable ignore logs for static assets and health checks.
• Log rotation prevents full disks; the distro default runs weekly.
• logrotate -d tests the configuration without side effects.
• nginx -s reopen must be called after log rotation.

In the next episode we'll discuss performance tuning and OS kernel optimization — optimizing worker processes, file delivery with sendfile, gzip and brotli compression, the open file cache, and tuning Linux kernel parameters.