Secure RSyslog Configuration - A Comprehensive Guide#

In modern infrastructure, centralized logging is essential for security monitoring, troubleshooting, and compliance. However, a logging system itself can become a security liability if not properly secured. This guide provides a comprehensive approach to hardening RSyslog deployments with encryption, authentication, and robust queue management.

Architecture Overview#

A secure logging infrastructure consists of multiple clients (log sources) sending logs to a central server over encrypted channels. Our configuration implements:

1
graph LR
2
    A[Web Servers] -->|TLS Encrypted| C[Central RSyslog Server]
3
    B[Application Servers] -->|TLS Encrypted| C
4
    D[Database Servers] -->|TLS Encrypted| C
5
    C --> E[(Structured Log Storage)]
6
    C --> F[SIEM/Analytics]

This architecture ensures logs are:

Encrypted during transit
Authenticated using X.509 certificates
Properly organized by source and facility
Protected against data loss with queue management
Rate-limited to prevent denial of service

Security Controls Matrix#

Control Point	Implementation	Purpose
Transport	TLS 1.3	Data encryption in transit
Authentication	X.509 Certificates	Client/server mutual authentication
Authorization	File permissions	Access control for logs
Integrity	Message signing	Tamper detection
Availability	Queue management	Reliability during outages
Rate Limiting	Session/connection limits	DoS protection

Server Configuration#

The server configuration establishes a TLS-enabled listener and organizes incoming logs by type and source:

1
# Load required modules
2
module(load="imtcp")
3
module(load="imuxsock") # system log input
4
module(load="imjournal") # systemd journal input
5

6
# Enable TLS for secure transport
7
module(load="gtls"
8
    StreamDriver.Name="gtls"
9
    StreamDriver.Mode="1"
10
    StreamDriver.Authmode="x509/name"
11
    DefaultNetstreamDriver="gtls"
12
)
13

14
# TLS settings
15
global(
16
    DefaultNetstreamDriver="gtls"
17
    DefaultNetstreamDriverCAFile="/etc/rsyslog.d/certs/ca.pem"
18
    DefaultNetstreamDriverCertFile="/etc/rsyslog.d/certs/server-cert.pem"
19
    DefaultNetstreamDriverKeyFile="/etc/rsyslog.d/certs/server-key.pem"
20
)
21

22
# TCP Input configuration with TLS
23
input(
24
    type="imtcp"
25
    port="6514"
26
    StreamDriver.Name="gtls"
27
    StreamDriver.Mode="1"
28
    StreamDriver.Authmode="x509/name"
29
)
30

31
# Set file permissions and ownership
32
$FileCreateMode 0640
33
$FileOwner syslog
34
$FileGroup adm
35
$DirCreateMode 0750
36

37
# Separate log files by facility
38
template(name="SecureApacheError" type="string" string="/var/log/remote/httpd/%HOSTNAME%/error.log")
39
template(name="SecureApacheAccess" type="string" string="/var/log/remote/httpd/%HOSTNAME%/access.log")
40

41
# Rules for sorting logs
42
if $syslogfacility-text == 'local3' then ?SecureApacheError
43
if $syslogfaculty-text == 'local4' then ?SecureApacheAccess
44

45
# Rate limiting to prevent DoS
46
module(load="imtcp" MaxSessions="1000" RateLimit.Interval="1" RateLimit.Burst="20000")

Key Security Features#

TLS Encryption: All connections use TLS with X.509 certificate authentication
Non-Standard Port: Using port 6514 instead of the standard 514 to avoid common scanning
Strict File Permissions: Explicit file and directory permissions (0640/0750)
Rate Limiting: Connection limits to prevent denial of service attacks
Structured Log Organization: Logs organized by host and type

Client Configuration#

The client configuration securely forwards Apache logs to the central server:

1
# Load required modules
2
module(load="imfile")
3
module(load="gtls")
4

5
# TLS Configuration
6
global(
7
    DefaultNetstreamDriver="gtls"
8
    DefaultNetstreamDriverCAFile="/etc/rsyslog.d/certs/ca.pem"
9
    DefaultNetstreamDriverCertFile="/etc/rsyslog.d/certs/client-cert.pem"
10
    DefaultNetstreamDriverKeyFile="/etc/rsyslog.d/certs/client-key.pem"
11
)
12

13
# Apache Error Log Input
14
input(
15
    type="imfile"
16
    File="/var/log/httpd/error_log"
17
    Tag="httpd-error-default:"
18
    StateFile="stat-httpd-error"
19
    Severity="info"
20
    Facility="local3"
21
    PersistStateInterval="1000"
22
    ReadMode="2"
23
)
24

25
# Apache Access Log Input
26
input(
27
    type="imfile"
28
    File="/var/log/httpd/access_log"
29
    Tag="httpd-access-default:"
30
    StateFile="stat-httpd-access"
31
    Severity="info"
32
    Facility="local4"
33
    PersistStateInterval="1000"
34
    ReadMode="2"
35
)
36

37
# Queue configuration for reliability
38
main_queue(
39
    queue.filename="fwdRule1"
40
    queue.maxdiskspace="1g"
41
    queue.saveonshutdown="on"
42
    queue.type="LinkedList"
43
    queue.timeoutEnqueue="0"
44
    queue.maxFileSize="100m"
45
)
46

47
# Forward to central server with TLS
48
action(
49
    type="omfwd"
50
    Target="log.example.com"
51
    Port="6514"
52
    Protocol="tcp"
53
    StreamDriver="gtls"
54
    StreamDriverMode="1"
55
    StreamDriverAuthMode="x509/name"
56
    queue.type="LinkedList"
57
    queue.filename="fwdRule1"
58
    queue.maxdiskspace="1g"
59
    queue.saveonshutdown="on"
60
    action.resumeRetryCount="-1"
61
    action.resumeInterval="30"
62
)

Key Client Features#

File Monitoring: Continuously monitors Apache log files for changes
Binary-Safe Reading: ReadMode="2" ensures proper handling of binary data
Reliable Delivery: Disk-assisted queues with persistence across restarts
Certificate Authentication: Mutual TLS authentication with the server
Automatic Recovery: Configurable retry parameters for network outages

Implementation Steps#

1. Certificate Generation#

First, create a certificate authority (CA) and generate server/client certificates:

1
# Create directories
2
mkdir -p /etc/rsyslog.d/certs
3

4
# Generate CA certificate
5
openssl req -newkey rsa:4096 -x509 -sha256 -days 365 -nodes \
6
    -out /etc/rsyslog.d/certs/ca.pem \
7
    -keyout /etc/rsyslog.d/certs/ca-key.pem
8

9
# Generate server certificate
10
openssl req -newkey rsa:4096 -nodes \
11
    -out /etc/rsyslog.d/certs/server.csr \
12
    -keyout /etc/rsyslog.d/certs/server-key.pem
13

14
# Sign server certificate
15
openssl x509 -req -in /etc/rsyslog.d/certs/server.csr \
16
    -CA /etc/rsyslog.d/certs/ca.pem \
17
    -CAkey /etc/rsyslog.d/certs/ca-key.pem \
18
    -CAcreateserial -out /etc/rsyslog.d/certs/server-cert.pem

Repeat the certificate generation process for each client, creating unique client certificates.

2. Set Permissions#

Secure your certificates with strict permissions:

1
chown -R syslog:syslog /etc/rsyslog.d/certs
2
chmod -R 0500 /etc/rsyslog.d/certs
3
chmod 0400 /etc/rsyslog.d/certs/*.pem

3. Create Log Directories#

Set up properly structured log directories:

1
mkdir -p /var/log/remote/httpd
2
chown -R syslog:adm /var/log/remote
3
chmod -R 0750 /var/log/remote

4. SELinux Configuration (if enabled)#

Apply correct SELinux contexts:

1
semanage port -a -t syslogd_port_t -p tcp 6514
2
semanage fcontext -a -t var_log_t "/var/log/remote(/.*)?"
3
restorecon -R /var/log/remote

5. Validate Configuration#

Check your configuration for errors:

1
rsyslogd -N1 -f /etc/rsyslog.conf

6. Restart RSyslog#

Apply the configuration:

1
systemctl restart rsyslog

Monitoring and Maintenance#

Certificate Management#

Implement these practices for certificate security:

Rotate certificates every 90 days
Use strong key sizes (4096 bits for RSA)
Set up automated renewal processes
Implement certificate expiration monitoring

Create a certificate expiration checker:

1
echo "0 0 1 * * root /usr/local/bin/check_cert_expiry.sh" > /etc/cron.d/cert_check

Queue Monitoring#

Watch for queue buildup which might indicate issues:

1
watch -n 60 'du -sh /var/lib/rsyslog'

Connection Verification#

Verify encrypted connections are properly established:

1
tcpdump -i any port 6514 -nn | grep "TLS"

Security Best Practices#

Network Security#

Implement network segmentation to isolate logging infrastructure
Use a dedicated management network for rsyslog traffic
Apply strict firewall rules to limit access to logging servers

System Hardening#

Regularly apply security patches to RSyslog and system
Run RSyslog with minimal privileges
Implement filesystem integrity monitoring
Configure host-based firewalls on all logging servers

Operational Security#

Implement log rotation and archiving policies
Set up log backup procedures
Conduct regular security audits of logging infrastructure
Document your logging architecture and security controls

Troubleshooting Common Issues#

Certificate Problems#

If you encounter TLS connection failures:

Verify certificate paths and permissions
Check certificate expiration dates
Ensure hostname verification is properly configured
Validate CA trust chain

Queue Issues#

If logs are not being delivered:

Check disk space on queue storage
Verify queue permissions
Inspect queue status with rsyslogd -d
Review retry parameters

Connection Failures#

For general connectivity issues:

Check firewall rules on both client and server
Verify network routing between systems
Ensure the correct port is being used
Test basic connectivity with netcat or similar tools

Advanced Features#

Log Filtering and Processing#

RSyslog can perform advanced processing on logs before forwarding:

1
# Filter out sensitive information
2
:msg, contains, "password=" stop
3

4
# Transform messages
5
template(name="JSONFormat" type="string" string="{\"message\":\"%msg%\",\"host\":\"%HOSTNAME%\",\"severity\":\"%syslogseverity-text%\",\"facility\":\"%syslogfacility-text%\",\"timereported\":\"%timereported:::date-rfc3339%\",\"timegenerated\":\"%timegenerated:::date-rfc3339%\"}\n")

High Availability#

For mission-critical environments, implement redundant RSyslog servers:

1
# Forward to multiple servers for redundancy
2
action(
3
    type="omfwd"
4
    Target="primary-log.example.com"
5
    Port="6514"
6
    Protocol="tcp"
7
    StreamDriver="gtls"
8
    StreamDriverMode="1"
9
    StreamDriverAuthMode="x509/name"
10
)
11

12
action(
13
    type="omfwd"
14
    Target="backup-log.example.com"
15
    Port="6514"
16
    Protocol="tcp"
17
    StreamDriver="gtls"
18
    StreamDriverMode="1"
19
    StreamDriverAuthMode="x509/name"
20
)

Conclusion#

A properly secured RSyslog implementation provides the foundation for robust security monitoring, compliance, and operational visibility. By implementing TLS encryption, certificate authentication, proper queue management, and following security best practices, you can ensure your logging infrastructure remains a security asset rather than a liability.

Remember that logging security is not a one-time setup but requires ongoing maintenance and monitoring. Regularly review your configuration, rotate certificates, and stay current with security updates to maintain the integrity of your logging system.