Managing Packages in Fedora CoreOS - A Comprehensive DNF and rpm-ostree Guide#

Fedora CoreOS (FCOS) represents a paradigm shift in operating system design, focusing on minimal, immutable infrastructure optimized for running containerized workloads. Unlike traditional Linux distributions, CoreOS uses a unique approach to package management that combines the atomic updating capabilities of rpm-ostree with the familiar DNF package manager. This guide explores how to effectively manage packages in this container-focused environment while maintaining the stability and security benefits of an immutable OS design.

Understanding the CoreOS Package Management Philosophy#

CoreOS adopts a fundamentally different approach to system updates and package management:

1
graph TD
2
    A[System Management in CoreOS] --> B[Immutable Base OS]
3
    A --> C[Atomic Updates]
4
    A --> D[Layered Packages]
5

6
    B --> E[Read-only /usr]
7
    B --> F[Predictable state]
8

9
    C --> G[rpm-ostree]
10
    G --> H[Complete System Images]
11
    G --> I[Bootable Snapshots]
12

13
    D --> J[DNF/rpm-ostree install]
14
    J --> K[User Installed Packages]
15
    J --> L[Container Integration]

The key concepts that differentiate CoreOS include:

Immutable Base System: The core OS is treated as a unit, updated atomically
Image-Based Updates: System updates are complete filesystem trees, not individual packages
Package Layering: Additional packages can be “layered” on top of the base image
Container Focus: Most applications should run as containers, not directly on the host

Initial CoreOS Setup#

Before diving into package management, let’s ensure your CoreOS system is properly set up:

Checking System Status#

1
# View current deployment status
2
rpm-ostree status
3

4
# Example output:
5
# State: idle
6
# AutomaticUpdates: disabled
7
# Deployments:
8
# ● fedora:fedora/x86_64/coreos/stable
9
#                Version: 35.20220116.3.0
10
#                 Commit: 9f12...
11
#                OSName: fedora

Enabling Container Tools#

CoreOS is designed primarily to run containers. Ensure container tools are enabled:

1
# Enable and start Podman
2
systemctl enable --now podman

Configuring Repository Sources#

While CoreOS updates primarily come through rpm-ostree, you can configure additional repositories for layered packages:

1
# Create a custom repository configuration
2
sudo mkdir -p /etc/yum.repos.d/
3
sudo tee /etc/yum.repos.d/custom.repo > /dev/null << EOF
4
[fedora]
5
name=Fedora \$releasever - \$basearch
6
baseurl=https://mirrors.fedoraproject.org/metalink?repo=fedora-\$releasever&arch=\$basearch
7
enabled=1
8
gpgcheck=1
9
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-\$releasever-\$basearch
10

11
[updates]
12
name=Fedora \$releasever - \$basearch - Updates
13
baseurl=https://mirrors.fedoraproject.org/metalink?repo=updates-released-f\$releasever&arch=\$basearch
14
enabled=1
15
gpgcheck=1
16
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-\$releasever-\$basearch
17
EOF

Basic System Update Operations#

Checking for Updates#

1
# Check for available updates without applying them
2
rpm-ostree upgrade --check

Applying System Updates#

1
# Update the system
2
rpm-ostree upgrade
3

4
# After updating, reboot to apply changes
5
systemctl reboot

Rebasing to a New Version#

When you want to move to a newer CoreOS release:

1
# Rebase to a specific Fedora version
2
rpm-ostree rebase fedora:35
3

4
# Or follow a specific stream
5
rpm-ostree rebase fedora/stable/x86_64/coreos

Managing Layered Packages#

While CoreOS discourages extensive host package installation, you can layer packages when needed:

Installing Packages#

1
# Install a single package
2
rpm-ostree install htop
3

4
# Install multiple packages
5
rpm-ostree install vim tmux git
6

7
# After installation, reboot to apply changes
8
systemctl reboot

Removing Packages#

1
# Remove a layered package
2
rpm-ostree uninstall htop
3

4
# Remove multiple packages
5
rpm-ostree uninstall vim tmux
6

7
# After removal, reboot to apply changes
8
systemctl reboot

Searching for Packages#

1
# Search for available packages
2
rpm-ostree search httpd

Listing Installed Packages#

1
# List all layered packages
2
rpm-ostree status --json | jq '.deployments[0].layered_packages'
3

4
# List all installed packages (base + layered)
5
rpm-ostree db list

Advanced Configuration and Maintenance#

DNF Configuration Options#

You can customize DNF behavior by creating or modifying /etc/dnf/dnf.conf:

1
# Create a custom DNF configuration
2
sudo tee /etc/dnf/dnf.conf > /dev/null << EOF
3
[main]
4
gpgcheck=1
5
installonly_limit=3
6
clean_requirements_on_remove=true
7
best=True
8
skip_if_unavailable=True
9
fastestmirror=True
10
max_parallel_downloads=10
11
deltarpm=True
12
EOF

Working with Package Groups#

CoreOS allows installing predefined groups of packages:

1
# List available groups
2
dnf group list
3

4
# Install a package group
5
rpm-ostree install @"Development Tools"
6

7
# After installation, reboot to apply changes
8
systemctl reboot

System Maintenance Operations#

Regular maintenance keeps your system running optimally:

1
# Remove old deployments (keeping only the current one)
2
rpm-ostree cleanup -p
3

4
# Remove all but the latest two deployments
5
rpm-ostree cleanup --rollback=2
6

7
# Clean DNF cache
8
dnf clean all
9

10
# Remove unused packages
11
rpm-ostree cleanup -m

System Rollback#

One of the key benefits of CoreOS is easy rollback to previous states:

1
# List available deployments
2
rpm-ostree status
3

4
# Rollback to previous deployment
5
rpm-ostree rollback
6

7
# Reboot to apply rollback
8
systemctl reboot
9

10
# Pin current deployment (prevent it from being garbage collected)
11
rpm-ostree pin

Integrating with Container Workflows#

CoreOS is designed primarily as a container host platform:

Container Base Management#

1
# Pull container images
2
podman pull registry.fedoraproject.org/fedora:latest
3

4
# Run a container with host integration
5
podman run -d --name web_server -p 80:80 nginx
6

7
# Build custom images
8
podman build -t custom_app:1.0 .

System Containers#

For services that need deeper system integration:

1
# Install system container packages
2
rpm-ostree install podman-compose
3

4
# Create a system container definition
5
mkdir -p ~/.config/containers/systemd/
6
cat > ~/.config/containers/systemd/web.container << EOF
7
[Unit]
8
Description=Web Server Container
9
After=network-online.target
10
Wants=network-online.target
11

12
[Container]
13
Image=docker.io/nginx:latest
14
PublishPort=8080:80
15
Volume=/var/www/html:/usr/share/nginx/html:Z
16

17
[Service]
18
Restart=always
19

20
[Install]
21
WantedBy=default.target
22
EOF
23

24
# Enable and start the container
25
systemctl --user daemon-reload
26
systemctl --user enable --now container-web.service

Troubleshooting Common Issues#

Failed Updates#

If updates fail to apply:

1
# Check system status
2
rpm-ostree status
3

4
# Clear cached data
5
rpm-ostree cleanup -b
6

7
# Force refresh metadata
8
rpm-ostree refresh-md -f

Package Conflicts#

When facing package installation conflicts:

1
# View transaction problems
2
rpm-ostree ex history
3

4
# Reset to last working state
5
rpm-ostree rollback
6

7
# Force package installation (use with caution)
8
rpm-ostree install --allow-inactive package_name

Logging and Debugging#

Viewing logs can help diagnose issues:

1
# View system logs
2
journalctl -u rpm-ostreed
3

4
# Enable verbose logging
5
rpm-ostree --verbose status
6

7
# Export transaction history
8
rpm-ostree ex history > history.txt

System Health Checks#

Regular health checks help maintain system integrity:

1
# Check system status
2
rpm-ostree status
3

4
# View service status
5
systemctl status rpm-ostreed
6

7
# Monitor system resources
8
top

Best Practices for CoreOS Package Management#

Performance Optimization#

Use delta RPMs: Enable deltarpm in DNF configuration to reduce download sizes
Configure appropriate mirror servers: Use geographically close mirrors
Implement caching strategies: Consider using a local proxy cache for multiple nodes

Security Considerations#

Always verify package signatures: Never disable gpgcheck
Keep system updated regularly: Schedule regular maintenance windows
Use trusted repositories only: Avoid third-party repos when possible

Maintenance Schedule#

Plan regular update windows: Schedule updates during low-usage periods
Test updates in non-production first: Validate changes before production deployment
Maintain backup deployments: Always keep at least one known-good deployment

Conclusion#

Fedora CoreOS provides a robust, secure platform for container workloads by fundamentally rethinking package management. The combination of rpm-ostree for atomic system updates and DNF for package layering offers both stability and flexibility. By following the practices outlined in this guide, you can effectively manage your CoreOS systems while maintaining the security and reliability benefits of an immutable infrastructure approach.

Remember that CoreOS is designed with containers in mind—the best practice is to run most applications as containers rather than installing packages directly on the host. This containerized approach enhances security, simplifies updates, and improves resource utilization across your infrastructure.