Systemd Service Setup for Podman and Docker Containers#

Managing containerized applications in production requires robust service management that ensures containers start automatically, restart on failure, and integrate seamlessly with the host system. This guide provides production-ready systemd service configurations for both Podman and Docker containers.

Understanding Systemd Integration#

Systemd provides powerful service management capabilities that complement container orchestration. By creating systemd unit files for containers, you gain:

Automatic container startup at boot
Dependency management between services
Automatic restart on failure
Resource control and limits
Centralized logging via journald
Integration with system monitoring tools

1
graph TD
2
    A[System Boot] --> B[systemd init]
3
    B --> C[Load Unit Files]
4
    C --> D[Resolve Dependencies]
5
    D --> E[Start Container Services]
6
    E --> F[Monitor Health]
7
    F --> G{Container Healthy?}
8
    G -->|Yes| H[Continue Running]
9
    G -->|No| I[Restart Container]
10
    I --> F
11
    H --> J[System Shutdown]
12
    J --> K[Stop Containers Gracefully]

Basic Systemd Service for Docker#

Simple Docker Container Service#

Create a basic systemd service file for a Docker container:

1
[Unit]
2
Description=MyApp Docker Container
3
Documentation=https://docs.myapp.com
4
Wants=network-online.target docker.service
5
After=network-online.target docker.service
6

7
[Service]
8
Type=simple
9
Restart=always
10
RestartSec=5
11
TimeoutStartSec=0
12
ExecStartPre=-/usr/bin/docker stop myapp
13
ExecStartPre=-/usr/bin/docker rm myapp
14
ExecStart=/usr/bin/docker run \
15
    --name myapp \
16
    --rm \
17
    -p 8080:8080 \
18
    -v /data/myapp:/data \
19
    -e APP_ENV=production \
20
    myapp:latest
21
ExecStop=/usr/bin/docker stop myapp
22

23
[Install]
24
WantedBy=multi-user.target

Docker with Health Checks#

For production environments, implement health checking:

1
[Unit]
2
Description=Web Application Docker Container
3
Documentation=https://docs.webapp.com
4
Wants=network-online.target docker.service
5
After=network-online.target docker.service
6
Before=nginx.service
7

8
[Service]
9
Type=notify
10
NotifyAccess=all
11
Restart=on-failure
12
RestartSec=10
13
StartLimitBurst=3
14
StartLimitIntervalSec=60
15

16
# Pull latest image before starting
17
ExecStartPre=/usr/bin/docker pull webapp:latest
18

19
# Remove any existing container
20
ExecStartPre=-/usr/bin/docker stop webapp
21
ExecStartPre=-/usr/bin/docker rm webapp
22

23
# Start container with health check
24
ExecStart=/usr/bin/docker run \
25
    --name webapp \
26
    --rm \
27
    --health-cmd="curl -f http://localhost:8080/health || exit 1" \
28
    --health-interval=30s \
29
    --health-timeout=10s \
30
    --health-retries=3 \
31
    --health-start-period=40s \
32
    -p 8080:8080 \
33
    -v /var/lib/webapp:/data \
34
    -e DATABASE_URL=postgresql://db:5432/webapp \
35
    --memory="1g" \
36
    --cpus="0.5" \
37
    webapp:latest
38

39
# Graceful shutdown
40
ExecStop=/usr/bin/docker stop -t 30 webapp
41

42
# Post-stop cleanup
43
ExecStopPost=-/usr/bin/docker rm -f webapp
44

45
[Install]
46
WantedBy=multi-user.target

Podman Systemd Services#

Basic Podman Service#

Podman integrates more naturally with systemd, supporting rootless containers:

1
# ~/.config/systemd/user/myapp-podman.service (for rootless)
2
# /etc/systemd/system/myapp-podman.service (for root)
3
[Unit]
4
Description=MyApp Podman Container
5
Documentation=https://docs.myapp.com
6
Wants=network-online.target
7
After=network-online.target
8

9
[Service]
10
Type=forking
11
Restart=always
12
RestartSec=5
13
TimeoutStartSec=30
14
Environment="PODMAN_SYSTEMD_UNIT=%n"
15
KillMode=mixed
16

17
# Create and start container
18
ExecStartPre=/usr/bin/podman create \
19
    --name myapp \
20
    --replace \
21
    -p 8080:8080 \
22
    -v myapp-data:/data:Z \
23
    -e APP_ENV=production \
24
    --health-cmd="curl -f http://localhost:8080/health || exit 1" \
25
    --health-interval=30s \
26
    --health-timeout=10s \
27
    --health-retries=3 \
28
    myapp:latest
29

30
ExecStart=/usr/bin/podman start myapp
31
ExecStop=/usr/bin/podman stop -t 30 myapp
32
ExecStopPost=/usr/bin/podman rm -f myapp
33

34
[Install]
35
WantedBy=default.target  # For user services
36
# WantedBy=multi-user.target  # For system services

Podman with Systemd Mode#

Podman can run containers with systemd as PID 1:

1
[Unit]
2
Description=Systemd Container with Podman
3
Documentation=https://docs.example.com
4
After=network-online.target
5

6
[Service]
7
Type=simple
8
Restart=always
9
RestartSec=10
10
TimeoutStartSec=0
11

12
# Run container with systemd mode
13
ExecStart=/usr/bin/podman run \
14
    --name systemd-container \
15
    --rm \
16
    --systemd=true \
17
    --cap-add=SYS_ADMIN \
18
    -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
19
    -v systemd-data:/data:Z \
20
    -p 8080:8080 \
21
    fedora-systemd:latest
22

23
ExecStop=/usr/bin/podman stop -t 60 systemd-container
24

25
[Install]
26
WantedBy=multi-user.target

Podman Generate Systemd#

Podman can automatically generate systemd unit files:

Generate from Running Container#

1
# Start a container
2
podman run -d \
3
    --name nginx \
4
    -p 80:80 \
5
    -v nginx-data:/usr/share/nginx/html:Z \
6
    nginx:alpine
7

8
# Generate systemd unit
9
podman generate systemd \
10
    --new \
11
    --name \
12
    --files \
13
    nginx
14

15
# Move generated file to systemd directory
16
sudo mv container-nginx.service /etc/systemd/system/

Generate for Pod#

1
# Create a pod
2
podman pod create \
3
    --name webapp \
4
    -p 8080:8080 \
5
    -p 5432:5432
6

7
# Add containers to pod
8
podman run -d \
9
    --pod webapp \
10
    --name webapp-app \
11
    myapp:latest
12

13
podman run -d \
14
    --pod webapp \
15
    --name webapp-db \
16
    postgres:13
17

18
# Generate systemd units for pod
19
podman generate systemd \
20
    --new \
21
    --files \
22
    --pod-prefix="" \
23
    --container-prefix="" \
24
    --separator="-" \
25
    webapp

Advanced Service Configurations#

Service with Dependencies#

1
[Unit]
2
Description=Application Stack Container
3
Documentation=https://docs.app.com
4
Wants=network-online.target postgresql.service redis.service
5
After=network-online.target postgresql.service redis.service
6
Before=nginx.service
7
Requires=postgresql.service redis.service
8

9
[Service]
10
Type=simple
11
Restart=on-failure
12
RestartSec=10
13
StartLimitBurst=5
14
StartLimitIntervalSec=300
15

16
# Environment variables
17
Environment="NODE_ENV=production"
18
EnvironmentFile=-/etc/sysconfig/app-stack
19

20
# Resource limits
21
LimitNOFILE=65536
22
LimitNPROC=4096
23
TasksMax=4096
24
MemoryLimit=2G
25
CPUQuota=200%
26

27
# Security settings
28
NoNewPrivileges=true
29
PrivateTmp=true
30

31
# Container execution
32
ExecStartPre=-/usr/bin/podman stop app-stack
33
ExecStartPre=-/usr/bin/podman rm app-stack
34
ExecStart=/usr/bin/podman run \
35
    --name app-stack \
36
    --rm \
37
    --network=host \
38
    --env-file=/etc/app-stack/env \
39
    --secret=app-secret,type=env,target=APP_SECRET \
40
    --memory=2g \
41
    --cpus=2 \
42
    --pids-limit=1000 \
43
    --ulimit nofile=65536:65536 \
44
    app-stack:latest
45

46
ExecStop=/usr/bin/podman stop -t 60 app-stack
47

48
# Cleanup on failure
49
ExecStopPost=-/usr/bin/sh -c 'podman rm -f app-stack || true'
50

51
[Install]
52
WantedBy=multi-user.target

Multi-Container Application#

1
# /etc/systemd/system/webapp@.service
2
[Unit]
3
Description=WebApp Container - %i
4
Documentation=https://docs.webapp.com
5
PartOf=webapp.target
6
After=network-online.target webapp-init.service
7
Requires=webapp-init.service
8

9
[Service]
10
Type=simple
11
Restart=always
12
RestartSec=5
13
Environment="INSTANCE=%i"
14

15
# Read instance-specific configuration
16
EnvironmentFile=/etc/webapp/%i.env
17

18
ExecStartPre=-/usr/bin/podman stop webapp-%i
19
ExecStartPre=-/usr/bin/podman rm webapp-%i
20
ExecStart=/usr/bin/podman run \
21
    --name webapp-%i \
22
    --rm \
23
    --network=webapp-net \
24
    --env-file=/etc/webapp/%i.env \
25
    -v webapp-%i-data:/data:Z \
26
    --label="com.example.webapp.instance=%i" \
27
    webapp:latest
28

29
ExecStop=/usr/bin/podman stop webapp-%i
30

31
[Install]
32
WantedBy=webapp.target

Podman Quadlets (Systemd Integration)#

Podman 4.4+ introduces Quadlets for native systemd integration:

Container Quadlet#

1
[Unit]
2
Description=Web Application Container
3
After=network-online.target
4

5
[Container]
6
Image=docker.io/myapp/webapp:latest
7
PublishPort=8080:8080
8
Volume=webapp-data:/data:Z
9
Environment=APP_ENV=production
10
Environment=LOG_LEVEL=info
11
Label=app=webapp
12
Label=version=1.0
13

14
# Health check
15
HealthCmd=curl -f http://localhost:8080/health
16
HealthInterval=30s
17
HealthTimeout=10s
18
HealthRetries=3
19
HealthStartPeriod=40s
20

21
# Resource limits
22
Memory=1g
23
CPUQuota=50%
24

25
# Security
26
NoNewPrivileges=true
27
ReadOnly=true
28
ReadOnlyTmpfs=true
29

30
[Service]
31
Restart=always
32
RestartSec=10
33
TimeoutStartSec=30
34

35
[Install]
36
WantedBy=multi-user.target

Pod Quadlet#

1
[Unit]
2
Description=Web Application Pod
3
After=network-online.target
4

5
[Pod]
6
PodName=webapp-pod
7
PublishPort=8080:8080
8
PublishPort=9090:9090
9
Network=webapp-net
10
Volume=shared-data:/shared:Z
11

12
[Service]
13
Restart=always
14

15
[Install]
16
WantedBy=multi-user.target

Service Management Commands#

Basic Operations#

1
# Reload systemd after creating/modifying unit files
2
sudo systemctl daemon-reload
3

4
# Start service
5
sudo systemctl start myapp.service
6

7
# Enable service to start at boot
8
sudo systemctl enable myapp.service
9

10
# Start and enable in one command
11
sudo systemctl enable --now myapp.service
12

13
# Check service status
14
sudo systemctl status myapp.service
15

16
# View service logs
17
sudo journalctl -u myapp.service -f
18

19
# Restart service
20
sudo systemctl restart myapp.service
21

22
# Stop service
23
sudo systemctl stop myapp.service

Troubleshooting#

1
# View detailed service information
2
systemctl show myapp.service
3

4
# Check why service failed
5
systemctl status myapp.service
6
journalctl -xe -u myapp.service
7

8
# List all failed services
9
systemctl list-units --failed
10

11
# Reset failed state
12
sudo systemctl reset-failed myapp.service
13

14
# View service dependencies
15
systemctl list-dependencies myapp.service
16

17
# Analyze boot time impact
18
systemd-analyze blame | grep myapp

Best Practices#

1. Container Lifecycle Management#

1
graph LR
2
    A[Pre-Start] --> B[Health Check]
3
    B --> C[Start Container]
4
    C --> D[Post-Start Validation]
5
    D --> E[Running State]
6
    E --> F[Health Monitoring]
7
    F --> G{Healthy?}
8
    G -->|Yes| E
9
    G -->|No| H[Restart Logic]
10
    H --> B
11
    E --> I[Graceful Shutdown]
12
    I --> J[Cleanup]

2. Restart Policies#

1
# Production-ready restart configuration
2
[Service]
3
# Restart on any non-zero exit code
4
Restart=on-failure
5

6
# Wait before restarting
7
RestartSec=10
8

9
# Limit restart attempts
10
StartLimitBurst=5
11
StartLimitIntervalSec=300
12

13
# What to do when start limit is hit
14
StartLimitAction=reboot  # or 'none', 'reboot-force'

3. Resource Management#

1
[Service]
2
# Memory limits
3
MemoryAccounting=true
4
MemoryMax=2G
5
MemoryHigh=1.5G
6

7
# CPU limits
8
CPUAccounting=true
9
CPUQuota=200%  # 2 cores
10
CPUWeight=100
11

12
# I/O limits
13
IOAccounting=true
14
IOWeight=100
15
IOReadBandwidthMax=/dev/sda 10M
16
IOWriteBandwidthMax=/dev/sda 10M
17

18
# Task limits
19
TasksAccounting=true
20
TasksMax=512

4. Security Hardening#

1
[Service]
2
# Run as non-root user
3
User=webapp
4
Group=webapp
5

6
# Filesystem protection
7
ReadOnlyPaths=/
8
ReadWritePaths=/var/lib/webapp
9
TemporaryFileSystem=/tmp:rw,nosuid,nodev,noexec,size=100M
10
PrivateTmp=true
11

12
# System call filtering
13
SystemCallFilter=@system-service
14
SystemCallErrorNumber=EPERM
15

16
# Capability restrictions
17
NoNewPrivileges=true
18
CapabilityBoundingSet=
19
AmbientCapabilities=
20

21
# Network isolation (if not needed)
22
PrivateNetwork=true
23

24
# Device access
25
PrivateDevices=true
26
DevicePolicy=closed

Monitoring and Alerting#

Systemd Service Monitoring#

1
#!/bin/bash
2
SERVICES=(
3
    "webapp-docker.service"
4
    "database-podman.service"
5
    "cache-redis.service"
6
)
7

8
for service in "${SERVICES[@]}"; do
9
    if ! systemctl is-active --quiet "$service"; then
10
        echo "CRITICAL: $service is not running"
11
        # Send alert (email, Slack, etc.)
12
        systemctl status "$service" | mail -s "Service $service failed" ops@example.com
13
    fi
14
done

Prometheus Integration#

1
# prometheus-systemd-exporter.service
2
[Unit]
3
Description=Prometheus Systemd Exporter
4
After=network-online.target
5

6
[Service]
7
Type=simple
8
ExecStart=/usr/bin/podman run \
9
    --name systemd-exporter \
10
    --rm \
11
    --pid=host \
12
    -v /sys/fs/cgroup:/host/sys/fs/cgroup:ro \
13
    -v /run/systemd:/run/systemd:ro \
14
    -p 9558:9558 \
15
    quay.io/prometheus/systemd-exporter:latest \
16
    --systemd.collector.unit-whitelist="(webapp|database|cache).*\.service"
17

18
[Install]
19
WantedBy=multi-user.target

Migration Strategies#

Docker Compose to Systemd#

1
#!/bin/bash
2
# Convert docker-compose.yml to systemd services
3

4
# Example: Convert a simple service
5
SERVICE_NAME="webapp"
6
IMAGE="myapp:latest"
7
PORT="8080:8080"
8
VOLUME="/data/webapp:/app/data"
9

10
cat > "/etc/systemd/system/${SERVICE_NAME}.service" << EOF
11
[Unit]
12
Description=${SERVICE_NAME} Container
13
After=docker.service
14
Requires=docker.service
15

16
[Service]
17
Type=simple
18
Restart=always
19
ExecStartPre=-/usr/bin/docker stop ${SERVICE_NAME}
20
ExecStartPre=-/usr/bin/docker rm ${SERVICE_NAME}
21
ExecStart=/usr/bin/docker run \\
22
    --name ${SERVICE_NAME} \\
23
    --rm \\
24
    -p ${PORT} \\
25
    -v ${VOLUME} \\
26
    ${IMAGE}
27

28
[Install]
29
WantedBy=multi-user.target
30
EOF
31

32
systemctl daemon-reload
33
systemctl enable --now ${SERVICE_NAME}.service

Conclusion#

Systemd provides a robust foundation for managing containerized applications in production. By leveraging systemd’s features with Docker and Podman, you can create reliable, self-healing container deployments that integrate seamlessly with your Linux infrastructure. Whether using traditional unit files or Podman’s Quadlets, the key is to implement proper health checking, resource management, and security controls to ensure your containers run reliably in production environments.