Collecting Docker Container Logs

This guide covers three methods for forwarding Docker container logs to Purl — from simple Docker Compose setups to production-grade Fluent Bit pipelines.

Prerequisites

✓ Docker and Docker Compose installed
✓ Purl instance running and accessible
✓ Purl API key for ingest authentication

Method 1: Direct HTTP API

The simplest approach — configure your application to send logs directly to Purl's REST API. This works well for applications that already output structured JSON logs.

bash

# Send logs directly via curl (JSON array or NDJSON)
curl -X POST http://your-purl:3000/api/logs \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-api-key" \
  -d '[{
    "timestamp": "2026-02-18T10:30:00Z",
    "level": "info",
    "message": "Container started successfully",
    "service": "my-app",
    "host": "docker-01",
    "meta": {"container_id": "abc123"}
  }]'

Method 2: Fluent Bit Sidecar

For production deployments, use Fluent Bit as a log forwarder. It reads container logs from Docker's JSON file driver and forwards them to Purl with minimal overhead.

fluent-bit.conf

[SERVICE]
    Flush         5
    Daemon        Off
    Log_Level     info

[INPUT]
    Name          tail
    Path          /var/lib/docker/containers/*/*.log
    Parser        docker
    Tag           docker.*
    Refresh_Interval  10

[FILTER]
    Name          modify
    Match         docker.*
    Rename        log message
    Add           service docker

[OUTPUT]
    Name          http
    Match         *
    Host          purl
    Port          3000
    URI           /api/logs
    Format        json
    Header        X-API-Key your-api-key
    Header        Content-Type application/json

Method 3: Docker Compose with Purl

A complete Docker Compose setup with your application, Fluent Bit, Purl, and ClickHouse — all wired together.

docker-compose.yml

version: '3.8'
services:
  app:
    image: your-app:latest
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

  fluent-bit:
    image: fluent/fluent-bit:latest
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro
    depends_on:
      - purl

  purl:
    image: ismoilovdev/purl:latest
    ports:
      - "3000:3000"
    environment:
      - PURL_CLICKHOUSE_HOST=clickhouse
      - PURL_CLICKHOUSE_PORT=8123
      - PURL_CLICKHOUSE_DATABASE=purl
      - PURL_CLICKHOUSE_USER=default
      - PURL_API_KEYS=your-api-key
    depends_on:
      - clickhouse

  clickhouse:
    image: clickhouse/clickhouse-server:latest
    environment:
      - CLICKHOUSE_DB=purl
    volumes:
      - clickhouse_data:/var/lib/clickhouse

volumes:
  clickhouse_data:

Method 4: OpenTelemetry Collector

If you're already using OpenTelemetry, configure the OTLP exporter to send logs directly to Purl's native OTLP endpoint.

otel-collector-config.yaml

receivers:
  filelog:
    include:
      - /var/lib/docker/containers/*/*.log
    operators:
      - type: json_parser

exporters:
  otlphttp:
    # logs_endpoint sets the exact path — the exporter does NOT
    # append /v1/logs to it (unlike the base endpoint option)
    logs_endpoint: http://purl:3000/api/v1/otlp/logs
    headers:
      X-API-Key: your-api-key

service:
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [otlphttp]

Troubleshooting

Logs not appearing? Check that your API key is correct and the Purl instance is reachable from the Docker network.
High memory usage? Set Fluent Bit's Mem_Buf_Limit to cap buffer size.
Duplicate logs? Ensure only one log forwarder is running per container.

← Back to all guides