The Daemonless Developer

Mastering Podman for Secure, Efficient, and Kubernetes-Ready Applications

Apr 03, 2025

Podman provides a daemonless, OCI-compliant container engine deeply integrated with modern Linux systems. For developers, particularly those working with Java, Quarkus, and targeting Kubernetes environments, following Podman best practices is necessary when you aim at building and running efficient, secure, and manageable containerized applications. This little guide gives your an overview of key practices, focusing on technical substance and real-world applicability. If you miss something, make sure to comment!

Building Efficient OCI Images

The foundation of effective containerization lies in well-crafted images. This section covers best practices for the image build process, focusing on security, size, and efficiency.

1. Select and Vet Base Images

Why: Security, stability, and size start here. Official images (from OS vendors, language projects like Eclipse Temurin, or software vendors on hubs like Docker Hub/Quay.io) and verified images (like Red Hat UBI) are generally maintained and scanned. Smaller base images (Alpine, UBI Minimal, Distroless) reduce attack surface and improve transfer speed. For Quarkus native applications, a minimal base like ubi-minimal or scratch (with necessary libraries) is optimal.
How (Official/Verified): Choose deliberately. Always use fully qualified image names like tags (quay.io/simpson/homer:2.1) instead of latest to ensure reproducible builds. Consider multi-arch images if needed.

# Good: Specific tag for a minimal UBI image
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.3
# Best for Reproducibility: Use digest (find with 'podman pull ... && podman images --digests ...')
# FROM registry.access.redhat.com/ubi9/ubi-minimal@sha256:<digest>

How (Evaluating Other Images): If using non-official or community images, perform due diligence:
- Transparency: Can you find the source Containerfile and build repository (e.g., on GitHub)? Lack of source is a red flag.
- Containerfile Analysis: Does it use a recent, secure base? Does it install excessive build tools without using multi-stage builds? Does it follow basic security practices (e.g., non-root user)?
- Maintenance: Check the image's last push date on the registry. An image not updated in months may contain outdated libraries with vulnerabilities. Use podman inspect <image> for metadata.
- Build Process: Was the image built via an automated process (e.g., GitHub Actions)? This increases confidence in reproducibility compared to manual CLI pushes.
- Inspect Layers: Use tools like Podman Desktop's Image Layers Explorer extension (or podman history <image>) to examine the contents and commands of each layer for unexpected additions or large files.

2. Master Multi-Stage Builds

Why: Essential for minimizing final image size and attack surface, especially for compiled languages like Java. The core principle is separating the build environment from the runtime environment.
How: Use multiple FROM statements, naming build stages (AS builder). The "builder" stage contains SDKs (JDK, GraalVM,) and build tools (Maven, Gradle). Copy only the necessary runtime artifacts (JARs, native executables, static assets, essential configs) from the builder stage to a minimal final runtime stage using COPY --from=builder. Choosing the right final runtime base image is critical – it should contain only the minimal OS libraries and runtimes needed.
Example (Quarkus Native Build):

# Stage 1: Build Environment (JDK, Maven, Native Tools)
FROM quay.io/quarkus/ubi-quarkus-mandrel:23.1-java17 AS builder
# ... (Maven build steps as in previous example) ...
RUN ./mvnw package -Pnative -Dquarkus.native.container-build=false -DskipTests

# Stage 2: Minimal Runtime Environment
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
WORKDIR /app
# Copy only the executable and necessary runtime libs
COPY --from=builder /home/quarkus/target/*-runner /app/application
COPY --from=builder /home/quarkus/target/quarkus-app/lib/ /app/lib/
# ... (User setup, permissions, EXPOSE, CMD as before) ...

Make sure to check on the Quarkus guides for Native Image Builds and Tips for writing native applications.

3. Optimize Layers, Leverage Cache, and Consider Squashing

Why: Fewer layers can mean slightly smaller images. The build cache dramatically speeds up rebuilds by reusing unchanged layers. But keep in mind, that this heavily depends on the frequency of layer changes and sizes.
How (Layers & Cache): Combine related RUN commands using && and \. Clean up temporary files (e.g., dnf clean all, apt-get clean) within the same RUN instruction. Order Containerfile instructions from least frequently changing (base image, package installs) to most frequently changing (copying source code) to maximize cache hits.
How (Squashing): Forcing fewer layers is possible using podman build --squash (merges layers created during the build into one new layer) or podman build --squash-all (creates a single new layer on top of the base image).
- Caution: Squashing destroys intermediate layers, which can significantly hurt build cache performance for subsequent builds and prevent efficient layer sharing during pulls. It also obscures layer history. Use very cautiously, primarily for final "release" images if absolute minimum layer count is critical and caching trade-offs are acceptable.

# Build and squash layers created during the build
# podman build --squash -t my-squashed-app .

4. Define Container Startup (CMD vs. ENTRYPOINT)

Why: Clearly defines how the container executes its primary process.
How: Use the exec form (["executable", "param1"]) for both ENTRYPOINT (main command, harder to override) and CMD (default arguments for entrypoint, easily overridden). Essential for correct signal handling compared to the shell form.

# Quarkus Native
ENTRYPOINT ["/app/application"] # The executable is the entrypoint
CMD ["-Dquarkus.http.host=0.0.0.0"] # Default arguments easily overridden at runtime

5. Minimize Build Context (.containerignore)

Why: Prevents unnecessary files from being sent to Podman, speeding builds, avoiding accidental secret inclusion, and optimizing cache behavior.
How: Create a .containerignore (or .dockerignore) file in the build context root listing files/directories to exclude (e.g., .git, build output folders, IDE configs, local secrets).

6. Scan Images for Vulnerabilities

Why: Identify known vulnerabilities (CVEs) in image layers and dependencies. Essential security hygiene.
How: Integrate scanning tools (Trivy, Grype, Clair, Snyk) into CI/CD or run manually. Use registry-integrated scanning features.

trivy image my-app-image:latest # Example using Trivy

Running Secure Containers

Executing containers securely is critical. These practices leverage Podman's architecture and Linux security features to protect your applications and host system at runtime.

7. Run as Non-Root User

Why: Fundamental security practice. Limits the impact of container compromise. Podman's rootless operation provides host protection; running the process inside as non-root confines the application itself.
How: Use the USER instruction in the Containerfile to switch to a numeric UID or predefined user. Ensure application files have correct permissions for this user. Base images like UBI often provide a default non-root user (e.g., UID 1001).

8. Leverage User Namespaces

Why: Core Linux isolation mechanism, used automatically by rootless Podman. Maps container UIDs/GIDs to unprivileged host UIDs/GIDs, meaning "root" inside the container is not root on the host.
How: Primarily automatic with rootless Podman.

9. Utilize SELinux

Why: Podman deeply integrates with powerful Linux kernel security features like SELinux and Seccomp to provide robust, layered container confinement, significantly reducing the potential impact of a container compromise.
- SELinux (Security-Enhanced Linux): Provides Mandatory Access Control (MAC). It operates based on labels assigned to processes (containers) and objects (files, ports, devices). SELinux policies define which labels can interact, effectively confining container processes and controlling their access to host resources even if other mechanisms like user namespaces are bypassed. It's a cornerstone of security on distributions like RHEL, Fedora, and CentOS Stream.
- Seccomp (Secure Computing Mode): Provides system call (syscall) filtering. It defines which specific calls a process is allowed to make into the Linux kernel. By blocking unnecessary or potentially dangerous syscalls, Seccomp drastically reduces the kernel's attack surface accessible from within the container.
How:
- SELinux: On SELinux-enabled systems, Podman automatically manages contexts (usually assigning container_t to container processes). The crucial part for users is often volume mounting: Use the :z (private, unshared label) or :Z (shared label) suffix when mounting host volumes (-v host:container:Z) to allow Podman to relabel the host path so the container's SELinux context can access it. Disabling SELinux entirely (--security-opt label=disable) is strongly discouraged as it removes a major security layer.
- Seccomp: Podman applies a restrictive default seccomp profile by default (often located at /usr/share/containers/seccomp.json), which blocks ~40% of Linux syscalls, including many known dangerous ones, while allowing those typically needed by standard applications. You can enhance security further by creating a custom JSON profile that only allows the specific syscalls your application requires and loading it via --security-opt seccomp=<path/to/your_profile.json>. Conversely, disabling seccomp (--security-opt seccomp=unconfined) dramatically increases risk and should only be used in specific, well-understood scenarios (like needing unusual syscalls for tracing/debugging tools within the container).
Benefit: Using SELinux for broad access control and Seccomp for fine-grained syscall filtering creates a powerful defense-in-depth strategy, minimizing the container's potential capabilities if compromised.
Example:

# Run with default SELinux and default Seccomp profile (standard secure practice)
podman run -d --name myapp-secure -v /local/data:/app/data:Z myimage

# Run with SELinux and a stricter custom Seccomp profile blocking more syscalls
podman run -d --name myapp-stricter \
  --security-opt seccomp=/etc/podman/app-specific-profile.json \
  -v /local/data:/app/data:Z \
  myimage

# Example of potentially needing 'unconfined' (use with extreme caution & analysis):
# Running a low-level tracing tool inside the container might require this.
# podman run --rm -it --security-opt seccomp=unconfined my-tracing-tool-image

Effective Runtime Configuration & Management

Beyond the build, effective runtime management ensures containers operate reliably and efficiently. This section addresses configuration, resource control, state management, and operational health.

11. Set Resource Limits

Why: Prevent resource exhaustion (CPU, Memory, PIDs) and ensure fair usage. Critical for stability and multi-tenancy.
How: podman run --memory <limit> (e.g., 1g), podman run --cpus <limit> (e.g., "1.5").
Java/Quarkus: Set container memory (--memory) significantly higher than JVM heap (-Xmx) to accommodate JVM overhead (metaspace, threads, direct buffers). Monitor and tune both.

12. Externalize Configuration

Why: Avoid embedding environment-specific settings (database connection strings, API keys, passwords, certificates) directly into your container image. This practice is insecure and inflexible. Externalizing configuration allows you to use the same image across different environments (dev, staging, prod) and manage sensitive data appropriately, aligning with 12-Factor App principles and Kubernetes ConfigMaps/Secrets.
How: Podman offers several mechanisms:
- Environment Variables (-e or --env): Suitable for non-sensitive configuration parameters.
- Volume Mounted Files (-v): Good for larger configuration files (e.g., XML, YAML, properties files). Mount them read-only (:ro). Use with SELinux labeling (:Z or :z) if applicable. I’ve mentioned this for completeness. It is actually considered dangerous to do this. Especially for “secrets”!
- Podman Secrets (--secret): Recommended for sensitive data. Secrets are managed by Podman and securely mounted into the container as temporary files (typically in /run/secrets/) with restricted permissions, avoiding exposure through environment variables.
Example (Using Podman Secrets for Sensitive Data):
Bash

# 1. Create the secrets using 'podman secret create'
#    Secrets can be created from files, stdin, or directly as arguments.

# Create a secret for a database password from stdin
printf "p@sswOrd123!" | podman secret create db_pass -

# Create a secret for an API key from a file
echo "aVerySecretApiKeyGoesHere" > /tmp/api-key.txt
podman secret create service_api_key /tmp/api-key.txt
shred -u /tmp/api-key.txt # Securely delete the temporary file

# Verify secrets are created
podman secret ls
# NAME                DRIVER    CREATED AT                      UPDATED AT                      DESCRIPTION   LABELS    OPTIONS
# db_pass             file      2025-04-02 07:20:00 +0200 CEST  2025-04-02 07:20:00 +0200 CEST
# service_api_key     file      2025-04-02 07:20:05 +0200 CEST  2025-04-02 07:20:05 +0200 CEST

# 2. Run the container, injecting secrets via the --secret flag
#    Non-sensitive configuration can still use environment variables.
podman run -d --name my-secure-app \
  --secret db_pass \
  --secret service_api_key \
  -e DB_USER="app_user" \
  -e DB_NAME="production_db" \
  -e DB_HOST="database.internal.lan" \
  my-app-image:latest

# 3. Inside the running container:
#    - The database password will be available as the content of the file: /run/secrets/db_pass
#    - The API key will be available as the content of the file: /run/secrets/service_api_key
#    Your application needs to be configured to read these files to retrieve the secrets.

# 4. Clean up secrets when they are no longer needed
# podman secret rm db_pass service_api_key

Benefit: Using Podman Secrets minimizes the risk of exposing sensitive credentials compared to environment variables, as the secrets exist only as temporary, permission-controlled files within the running container's filesystem namespace.

13. Implement Health Checks

Why: Enable automated detection and recovery of unresponsive applications within containers. Essential for orchestrators.
Important Context
- The HEALTHCHECK instruction commonly seen in Dockerfile examples is a Docker-specific extension and is NOT part of the official OCI Image Specification.
- Podman does NOT execute health checks defined via the HEALTHCHECK instruction embedded in image metadata. It effectively ignores this instruction at runtime. (See containers/podman#25454).
- Kubernetes also ignores the image's HEALTHCHECK instruction. It requires health checks (livenessProbe, readinessProbe, startupProbe) to be explicitly defined within the Kubernetes resource YAML (e.g., Pod, Deployment).
How: Use the HEALTHCHECK instruction. For Java/Quarkus, leverage MicroProfile Health endpoints (/q/health/live, /q/health/ready) checked via curl.

HEALTHCHECK --interval=10s --timeout=2s --start-period=20s --retries=3 \
  CMD curl -f http://localhost:8080/q/health/live || exit 1

#Alternatively you can use the Podman CLI to run the HEALTHCHECK
podman healthcheck run container

14. Keep Containers Ephemeral (Stateless)

Why: Treat containers as immutable, disposable units. Persist state externally. Core principle for scalability and resilience, vital for Kubernetes.
How: Use Podman volumes (podman volume create, -v volume:/path:Z) or bind mounts (-v /host:/path:Z) for data needing persistence.

15. Use tmpfs for Sensitive Volatile Data

Why: In-memory storage for temporary data (e.g., session tokens) that should never hit disk and disappears on container stop.
How: podman run --tmpfs /path/in/container[:options].

16. Configure Logging Strategy

Why: Enable effective troubleshooting and monitoring.
How: Applications should log to standard output (stdout) and standard error (stderr). Podman captures these streams. Use podman logs <container>. When managed by systemd (see below), logs typically go to the journal, accessible via journalctl. Avoid logging to files inside the container unless using a dedicated logging sidecar pattern. Consider log drivers (--log-driver=journald) for direct integration.

Networking and Multi-Container Setups

Connecting containers and managing multi-container applications effectively is crucial. This section covers networking strategies and tools for local development setups.

17. Use Podman Pods for Co-located Containers

Why: Group tightly coupled containers sharing network (localhost communication) and other namespaces, mirroring Kubernetes Pods. Excellent for sidecar patterns (e.g., app + monitoring agent).
How: Use podman pod create to define the pod (mapping ports here) and podman run --pod <pod_name> to add containers to it.

podman pod create --name web-api-pod -p 8080:8080
podman run -d --pod web-api-pod --name frontend my-frontend 
podman run -d --pod web-api-pod --name backend my-backend-api

18. Use Docker Compose with Podman via the API Socket

Why: Leverage the widely adopted Docker Compose V2 tool and the familiar compose.yaml format to define and manage multi-container applications locally, while benefiting from Podman as the underlying container engine. This allows developers accustomed to the docker compose workflow to utilize Podman seamlessly.
How (Setup): To enable docker compose (V2, the standard command integrated with Docker or available standalone) to communicate with Podman, you need to activate the Podman API socket service and configure the DOCKER_HOST environment variable:
1. Enable Podman Socket: The Podman API service listens on a Unix socket. Activate it using systemd:
  - Rootless (Recommended): Run systemctl --user enable --now podman.socket. This starts the socket listener for your user session.
  - Rootful: Run sudo systemctl enable --now podman.socket to enable it system-wide.
2. Set DOCKER_HOST Variable: Inform the docker compose client where to find the Podman API socket.
  - Rootless: Set export DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock. It's recommended to add this line to your shell profile (e.g., ~/.bashrc, ~/.zshrc) for persistence.
  - Rootful: Set export DOCKER_HOST=unix:///run/podman/podman.sock.
3. Ensure docker-compose is Installed: You need the official docker-compose V2 binary (which might be installed as part of Docker Desktop or separately via package managers or direct download). Note: This refers to the standard docker compose command, not the unrelated, community-maintained podman-compose Python script.
How (Usage): With the socket active and DOCKER_HOST correctly set, you can use standard docker compose commands (e.g., docker compose up, docker compose down, docker compose ps, docker compose logs) directly with your existing compose.yaml or docker-compose.yaml files. These commands will interact with the Podman service via its API, and Podman will manage the containers, networks, and volumes accordingly.
Example (compose.yaml - Standard Syntax):

# compose.yaml
version: '3.8'
services:
  frontend:
    image: my-frontend-image:latest
    ports:
      - "8080:80"
    depends_on:
      - backend
    environment:
      BACKEND_URL: http://backend:9000
  backend:
    image: my-backend-image:latest
    expose:
      - "9000"
    depends_on:
      - database
    environment:
      DB_HOST: database
  database:
    image: docker.io/library/postgres:15-alpine
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: example_password
volumes:
  db_data: {}

# --- One-time Setup Steps (Rootless Example) ---
# systemctl --user enable --now podman.socket
# export DOCKER_HOST="unix:///run/user/$(id -u)/podman/podman.sock"
# # Add the export line to ~/.bashrc or similar for persistence

# --- Routine Usage (using standard 'docker compose') ---
# Navigate to the directory containing compose.yaml
docker compose up -d  # Start services in detached mode via Podman

docker compose ps     # List running services managed by Podman

docker compose down   # Stop and remove containers, networks, volumes

You can also use Podman Desktop and easily switch on Docker Compose support.
Benefit: Provides the familiar and powerful docker compose interface for defining and managing complex local application stacks, while harnessing Podman's daemonless architecture, security features (like rootless operation), and integration with the Linux ecosystem.

19. Create Custom Networks for Isolation

Why: Provide network segmentation and enable container-name DNS resolution.
How: podman network create <name>, podman run --network <name> ....

20. Use Host Networking Sparingly

Why: --network host breaks network isolation, always increasing security risks and causing port conflicts.
How: Only use when absolutely necessary for performance or specific network interface access.

Automation, Host Integration, and Kubernetes Alignment

To maximize efficiency and prepare for orchestration, integrate Podman into broader workflows. This section covers automation, systemd integration, and bridging local development with Kubernetes.

21. Integrate with Systemd for Service Management

Why: The standard Linux way to manage long-running services. Provides auto-start, dependency management, resource control, and robust logging integration via the journal. Ideal for running persistent containerized applications on Linux hosts.
How: Quadlet: Create simple .container, .volume, .network files in systemd directories (e.g., ~/.config/containers/systemd/ for rootless). Quadlet translates these into systemd units automatically on systemctl daemon-reload.

# After creating a Quadlet file (e.g., myapp.container) or generating a unit file:

systemctl --user daemon-reload
systemctl --user enable --now myapp.service # Or container-myapp.service
systemctl --user status myapp.service
journalctl --user -u myapp.service # View logs

22. Automate Builds in CI/CD

Why: Consistent, repeatable, tested image builds and deployments.
How: Use Podman commands (login, build, tag, push, scanning) in CI/CD pipeline scripts. Manage credentials securely.

podman pod create --name web-api-pod -p 8080:8080
podman run -d --pod web-api-pod --name frontend my-frontend # Can access backend via localhost:9000
podman run -d --pod web-api-pod --name backend -p 9000:9000 my-backend-api # Runs on port 9000 within pod

23. Leverage Pods and podman kube for Kubernetes Development

Why: Develop and test multi-container applications locally in Podman pods, mirroring Kubernetes structure, then easily generate Kubernetes manifests and test on a local K8s cluster (Kind, Minikube) before full deployment.
How:

Develop Locally in a Pod: Create a pod and run related containers (app, sidecar) inside it, testing interactions via localhost.

podman pod create --name k8s-dev-pod -p 8080:8080
podman run -d --pod k8s-dev-pod --name my-app my-quarkus-app:dev
podman run -d --pod k8s-dev-pod --name metrics-sidecar my-metrics-exporter
# Test interactions...

Generate Kubernetes YAML: Once satisfied, generate a K8s manifest directly from the running Podman pod.

podman kube generate k8s-dev-pod > k8s-dev-pod.yaml

Test on Local Kubernetes: Apply this manifest to your local K8s cluster (e.g., Kind, Minikube).

# Ensure kubectl context points to your local cluster (e.g., kind-kind)
kubectl apply -f k8s-dev-pod.yaml
kubectl get pods # Verify the pod is running in Kubernetes

Iterate: Modify code, rebuild image, update containers in the Podman pod (podman rm -f <container>; podman run ...), regenerate YAML, re-apply to K8s. This provides a rapid local dev loop aligned with K8s deployment artifacts. podman kube play <yaml> can also run K8s YAML directly with Podman.

24. Consider Auto-Updates (With Caution)

Why: Can automatically apply image updates.
How: Use io.containers.autoupdate label and the podman-auto-update.timer systemd service. Risk: High potential for breakage. Use specific tags, have robust testing/rollback. Not generally recommended for critical production workloads without significant safeguards.
In addition to these points, Podman auto-update supports two update policies:
- registry: Podman checks the container image registry for newer versions of the image.
- local: Podman checks for any newer versions of the image stored locally on the host.
Podman auto-update is a powerful tool that can help keep your containers up-to-date and secure. However, it is important to use it with caution and to thoroughly test any updates before deploying them to production.
Here are some additional tips for using Podman auto-update:
- Use specific version tags rather than latest to avoid unexpected changes.
- Monitor the auto-update process closely and be prepared to roll back if necessary.
- Keep your Podman installation up-to-date to benefit from the latest features and security improvements.
By following these best practices, you can leverage the power of Podman auto-updates to improve the security and reliability of your containerized applications while minimizing the risks of unintended consequences.

25. Be Aware of Storage Drivers

Why: Affect performance and features (e.g., overlayfs, btrfs).
How: Default (overlay) is usually fine. Check podman info. Advanced tuning via /etc/containers/storage.conf.

26. Use Podman Desktop for Local Container Management

Why: Podman Desktop offers a graphical user interface (GUI) for managing containers, images, pods, volumes, and registries on your local machine (Linux, macOS, Windows). It simplifies common tasks, provides a visual overview of your container environment, assists with Podman setup (especially on macOS/Windows via Podman machine), and can lower the barrier to entry for those less comfortable with the command line or migrating from other tools like Docker Desktop. Its extension model allows adding functionality like Kubernetes cluster interaction, advanced image inspection, and more.
How: Download and install from https://podman-desktop.io/downloads. Use the graphical interface to perform actions like listing/inspecting/starting/stopping resources, viewing logs, opening terminals into containers, managing registries, and interacting with extensions.
Benefit: Streamlines the local development inner loop by offering a visual alternative or complement to the Podman CLI for everyday tasks. It makes managing multiple resources, exploring image layers (with extensions), or interacting with local Kubernetes contexts more accessible.

Stay Secure, Flexible and Fast

Podman hands developers a secure, flexible, and Linux-native container engine. By applying these practices you can build and manage robust, cloud-ready applications efficiently and securely. If I did miss a tip, please let me know in the comments!