docker-orchestration

First-party

Reference: full SKILL.md

Below is the complete skill definition this hub loads when the skill is triggered — what the agent sees as its instructions, verbatim and unabridged.

# Docker Orchestration

## Role
You are an elite container orchestration architect. You design production-grade Docker
configurations with security hardening, resource optimization, multi-stage builds, and
GPU passthrough for AI workloads.

---

## Part 1: Production Multi-Stage Dockerfiles

### Python FastAPI (Production)
```dockerfile
# Stage 1: Build dependencies
FROM python:3.12-slim AS builder

WORKDIR /app
RUN pip install --no-cache-dir uv

COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-editable

# Stage 2: Production image
FROM python:3.12-slim AS production

# Security: non-root user
RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /sbin/nologin appuser

WORKDIR /app

# Copy only virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
COPY src/ ./src/

# Set environment
ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

USER appuser
EXPOSE 8000

CMD ["uvicorn", "src.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
```

### Next.js (Production)
```dockerfile
FROM node:20-alpine AS base

FROM base AS deps
WORKDIR /app
COPY package.json pnpm-lock.yaml ./
RUN corepack enable pnpm && pnpm install --frozen-lockfile

FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN corepack enable pnpm && pnpm build

FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs
EXPOSE 3000
ENV PORT=3000 HOSTNAME="0.0.0.0"

CMD ["node", "server.js"]
```

---

## Part 2: Docker Compose Production Stack

### Full Stack with FastAPI + Next.js + PostgreSQL + Redis
```yaml
# docker-compose.yml
version: "3.9"

x-common-env: &common-env
  TZ: UTC
  LOG_LEVEL: info

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
      target: production
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      <<: *common-env
      DATABASE_URL: postgresql+asyncpg://app:${DB_PASSWORD}@db:5432/appdb
      REDIS_URL: redis://redis:6379/0
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "2.0"
          memory: 1G
        reservations:
          cpus: "0.5"
          memory: 256M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 15s
    networks:
      - app-network

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      <<: *common-env
      NEXT_PUBLIC_API_URL: http://backend:8000
    depends_on:
      - backend
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    networks:
      - app-network

  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: app
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      resources:
        limits:
          memory: 1G
    networks:
      - app-network

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - redis-data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3
    deploy:
      resources:
        limits:
          memory: 512M
    networks:
      - app-network

volumes:
  postgres-data:
  redis-data:

networks:
  app-network:
    driver: bridge
```

---

## Part 3: GPU Passthrough for Ollama

### Docker Compose with Ollama GPU
```yaml
services:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama-data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
        limits:
          memory: 32G
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
      interval: 30s
      timeout: 10s
      retries: 3
    environment:
      OLLAMA_NUM_PARALLEL: 2
      OLLAMA_MAX_LOADED_MODELS: 2

  # Pre-pull models on startup
  ollama-init:
    image: curlimages/curl:latest
    depends_on:
      ollama:
        condition: service_healthy
    entrypoint: |
      sh -c '
        curl -s http://ollama:11434/api/pull -d "{\"name\": \"qwen2.5:7b\"}"
        curl -s http://ollama:11434/api/pull -d "{\"name\": \"nomic-embed-text\"}"
      '
    restart: "no"

volumes:
  ollama-data:
```

### macOS (Apple Silicon — No Docker GPU)
```bash
# On macOS, run Ollama natively for Metal GPU access
# Docker on Mac does NOT support GPU passthrough
brew install ollama
ollama serve &
ollama pull qwen2.5:7b

# Connect Docker containers to host Ollama
# In docker-compose.yml:
# environment:
#   OLLAMA_HOST: host.docker.internal:11434
```

---

## Part 4: Docker Networking Patterns

### Internal Service Communication
```yaml
services:
  api:
    networks:
      - frontend-net
      - backend-net

  db:
    networks:
      - backend-net  # Only accessible from backend network

  nginx:
    networks:
      - frontend-net
    ports:
      - "80:80"  # Only nginx is publicly exposed

networks:
  frontend-net:
  backend-net:
    internal: true  # No external access
```

### Custom Bridge with Fixed IPs
```yaml
networks:
  app-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16

services:
  api:
    networks:
      app-net:
        ipv4_address: 172.28.0.10
```

---

## Part 5: Secrets Management

### Docker Secrets (Swarm Mode)
```yaml
services:
  api:
    secrets:
      - db_password
      - api_key
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    external: true
```

### Environment File Pattern (Compose)
```yaml
services:
  api:
    env_file:
      - .env            # Base config
      - .env.production  # Production overrides
```

```bash
# .env
DATABASE_URL=postgresql://user:pass@db:5432/app
REDIS_URL=redis://redis:6379/0

# .env.production (gitignored)
DATABASE_URL=postgresql://prod:${DB_PASSWORD}@db:5432/app
```

---

## Part 6: Compose Profiles

```yaml
services:
  api:
    build: ./api
    profiles: ["app", "full"]

  db:
    image: postgres:16
    profiles: ["app", "full"]

  redis:
    image: redis:7
    profiles: ["app", "full"]

  # Dev-only services
  adminer:
    image: adminer
    profiles: ["dev"]
    ports:
      - "8080:8080"

  mailhog:
    image: mailhog/mailhog
    profiles: ["dev"]
    ports:
      - "1025:1025"
      - "8025:8025"

  # Monitoring (optional)
  prometheus:
    image: prom/prometheus
    profiles: ["monitoring", "full"]

  grafana:
    image: grafana/grafana
    profiles: ["monitoring", "full"]
```

```bash
# Start just the app
docker compose --profile app up -d

# Start app + dev tools
docker compose --profile app --profile dev up -d

# Start everything
docker compose --profile full up -d
```

---

## Part 7: Docker Best Practices Checklist

### Image Optimization
```dockerfile
# 1. Use specific base image tags (never :latest in production)
FROM python:3.12.3-slim-bookworm

# 2. Combine RUN commands to reduce layers
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

# 3. Use .dockerignore
# .dockerignore:
# .git
# __pycache__
# *.pyc
# .env
# node_modules
# .next

# 4. Order layers by change frequency (least → most)
COPY requirements.txt .        # Changes rarely
RUN pip install -r requirements.txt
COPY src/ ./src/               # Changes often

# 5. Use COPY not ADD (unless extracting tar)
COPY . .

# 6. Set proper stop signal
STOPSIGNAL SIGTERM
```

### Security Hardening
```dockerfile
# Non-root user
RUN useradd -r -s /bin/false appuser
USER appuser

# Read-only filesystem
# In compose: read_only: true, tmpfs: [/tmp]

# No new privileges
# In compose: security_opt: [no-new-privileges:true]

# Scan for vulnerabilities
# docker scout cves myimage:latest
```

---

## Part 8: Useful Docker Commands

```bash
# Cleanup
docker system prune -af --volumes  # Nuclear cleanup
docker builder prune -af           # Clear build cache

# Debugging
docker compose logs -f --tail=100 api  # Follow logs
docker compose exec api bash           # Shell into running container
docker compose run --rm api pytest     # One-off command

# Build
docker compose build --no-cache api    # Force rebuild
docker compose build --parallel        # Build all in parallel

# Resource monitoring
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"

# Copy files from container
docker cp container_id:/app/data.json ./data.json
```

## AXE MCP Server Integration

Every skill in the AXE Skills Hub runs with access to the **AXE MCP Server** — giving it the full fleet intelligence toolkit automatically. No setup required; tools are available in any AXE-powered session.

### Core Tools Available

| Category | Tools | Use Case |
|----------|-------|----------|
| **Memory** | `read_memory`, `write_memory`, `list_memory` | Persist context across sessions |
| **Web** | `web_search`, `web_fetch` | Live data, docs, research |
| **File Ops** | `read_file`, `write_file` | Read/write any local file |
| **Fleet** | `fleet_ssh`, `axe_push` | Run commands on JL2/JL3/JL4, send notifications |
| **AI Models** | `query_team_channel`, `get_partner_state` | Cross-agent coordination |
| **Data** | `qdrant_search`, `qdrant_store` | Semantic memory & vector search |
| **Pipeline** | `hydra_add` | Add high-quality outputs to Edge training |
| **Skills** | `hub_list_skills`, `hub_get_skill`, `hub_search_skills`, `hub_get_registry`, `hub_skill_metadata` | Chain skills together |
| **Secrets** | `get_secret` | Retrieve API keys securely |

### Quick Start

```python
# In any AXE session, tools are pre-loaded. Example chaining:

# 1. Search for context
results = qdrant_search("user query here", collection="axe_persistent_memory")

# 2. Fetch live data if needed
content = web_fetch("https://docs.example.com/api")

# 3. Write result to memory for next session
write_memory("shared/last_result.md", output)

# 4. Log quality output to Edge training pipeline
hydra_add(prompt=user_query, response=output, score=0.9, source="skill-name")
```

### Edge Training Integration

High-quality skill outputs are automatically eligible for Edge model training via `hydra_add`. When a response scores ≥0.85 in evals, pipe it to the Hydra pipeline to compound Edge's knowledge. This is how skills make Edge smarter over time.

```python
# After generating a high-quality response:
hydra_add(
    prompt=user_input,
    response=final_output,
    score=0.9,          # eval score
    source="skill-name" # tracks provenance
)
```

Docker Orchestration

Role

You are an elite container orchestration architect. You design production-grade Docker

configurations with security hardening, resource optimization, multi-stage builds, and

GPU passthrough for AI workloads.

Part 1: Production Multi-Stage Dockerfiles

Python FastAPI (Production)

# Stage 1: Build dependencies
FROM python:3.12-slim AS builder

WORKDIR /app
RUN pip install --no-cache-dir uv

COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-dev --no-editable

# Stage 2: Production image
FROM python:3.12-slim AS production

# Security: non-root user
RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /sbin/nologin appuser

WORKDIR /app

# Copy only virtual environment from builder
COPY --from=builder /app/.venv /app/.venv
COPY src/ ./src/

# Set environment
ENV PATH="/app/.venv/bin:$PATH" \
    PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1

# Health check
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

USER appuser
EXPOSE 8000

CMD ["uvicorn", "src.api.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

Next.js (Production)

FROM node:20-alpine AS base

FROM base AS deps
WORKDIR /app
COPY package.json pnpm-lock.yaml ./
RUN corepack enable pnpm && pnpm install --frozen-lockfile

FROM base AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN corepack enable pnpm && pnpm build

FROM base AS runner
WORKDIR /app
ENV NODE_ENV=production

RUN addgroup --system --gid 1001 nodejs && \
    adduser --system --uid 1001 nextjs

COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs
EXPOSE 3000
ENV PORT=3000 HOSTNAME="0.0.0.0"

CMD ["node", "server.js"]

Part 2: Docker Compose Production Stack

Full Stack with FastAPI + Next.js + PostgreSQL + Redis

# docker-compose.yml
version: "3.9"

x-common-env: &common-env
  TZ: UTC
  LOG_LEVEL: info

services:
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
      target: production
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      <<: *common-env
      DATABASE_URL: postgresql+asyncpg://app:${DB_PASSWORD}@db:5432/appdb
      REDIS_URL: redis://redis:6379/0
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "2.0"
          memory: 1G
        reservations:
          cpus: "0.5"
          memory: 256M
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 15s
    networks:
      - app-network

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      <<: *common-env
      NEXT_PUBLIC_API_URL: http://backend:8000
    depends_on:
      - backend
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M
    networks:
      - app-network

  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: appdb
      POSTGRES_USER: app
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    volumes:
      - postgres-data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 10s
      timeout: 5s
      retries: 5
    deploy:
      resources:
        limits:
          memory: 1G
    networks:
      - app-network

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    volumes:
      - redis-data:/data
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 3s
      retries: 3
    deploy:
      resources:
        limits:
          memory: 512M
    networks:
      - app-network

volumes:
  postgres-data:
  redis-data:

networks:
  app-network:
    driver: bridge

Part 3: GPU Passthrough for Ollama

Docker Compose with Ollama GPU

services:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama-data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
        limits:
          memory: 32G
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:11434/api/tags"]
      interval: 30s
      timeout: 10s
      retries: 3
    environment:
      OLLAMA_NUM_PARALLEL: 2
      OLLAMA_MAX_LOADED_MODELS: 2

  # Pre-pull models on startup
  ollama-init:
    image: curlimages/curl:latest
    depends_on:
      ollama:
        condition: service_healthy
    entrypoint: |
      sh -c '
        curl -s http://ollama:11434/api/pull -d "{\"name\": \"qwen2.5:7b\"}"
        curl -s http://ollama:11434/api/pull -d "{\"name\": \"nomic-embed-text\"}"
      '
    restart: "no"

volumes:
  ollama-data:

macOS (Apple Silicon — No Docker GPU)

# On macOS, run Ollama natively for Metal GPU access
# Docker on Mac does NOT support GPU passthrough
brew install ollama
ollama serve &
ollama pull qwen2.5:7b

# Connect Docker containers to host Ollama
# In docker-compose.yml:
# environment:
#   OLLAMA_HOST: host.docker.internal:11434

Part 4: Docker Networking Patterns

Internal Service Communication

services:
  api:
    networks:
      - frontend-net
      - backend-net

  db:
    networks:
      - backend-net  # Only accessible from backend network

  nginx:
    networks:
      - frontend-net
    ports:
      - "80:80"  # Only nginx is publicly exposed

networks:
  frontend-net:
  backend-net:
    internal: true  # No external access

Custom Bridge with Fixed IPs

networks:
  app-net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16

services:
  api:
    networks:
      app-net:
        ipv4_address: 172.28.0.10

Part 5: Secrets Management

Docker Secrets (Swarm Mode)

services:
  api:
    secrets:
      - db_password
      - api_key
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    external: true

Environment File Pattern (Compose)

services:
  api:
    env_file:
      - .env            # Base config
      - .env.production  # Production overrides

# .env
DATABASE_URL=postgresql://user:pass@db:5432/app
REDIS_URL=redis://redis:6379/0

# .env.production (gitignored)
DATABASE_URL=postgresql://prod:${DB_PASSWORD}@db:5432/app

Part 6: Compose Profiles

services:
  api:
    build: ./api
    profiles: ["app", "full"]

  db:
    image: postgres:16
    profiles: ["app", "full"]

  redis:
    image: redis:7
    profiles: ["app", "full"]

  # Dev-only services
  adminer:
    image: adminer
    profiles: ["dev"]
    ports:
      - "8080:8080"

  mailhog:
    image: mailhog/mailhog
    profiles: ["dev"]
    ports:
      - "1025:1025"
      - "8025:8025"

  # Monitoring (optional)
  prometheus:
    image: prom/prometheus
    profiles: ["monitoring", "full"]

  grafana:
    image: grafana/grafana
    profiles: ["monitoring", "full"]

# Start just the app
docker compose --profile app up -d

# Start app + dev tools
docker compose --profile app --profile dev up -d

# Start everything
docker compose --profile full up -d

Part 7: Docker Best Practices Checklist

Image Optimization

# 1. Use specific base image tags (never :latest in production)
FROM python:3.12.3-slim-bookworm

# 2. Combine RUN commands to reduce layers
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

# 3. Use .dockerignore
# .dockerignore:
# .git
# __pycache__
# *.pyc
# .env
# node_modules
# .next

# 4. Order layers by change frequency (least → most)
COPY requirements.txt .        # Changes rarely
RUN pip install -r requirements.txt
COPY src/ ./src/               # Changes often

# 5. Use COPY not ADD (unless extracting tar)
COPY . .

# 6. Set proper stop signal
STOPSIGNAL SIGTERM

Security Hardening

# Non-root user
RUN useradd -r -s /bin/false appuser
USER appuser

# Read-only filesystem
# In compose: read_only: true, tmpfs: [/tmp]

# No new privileges
# In compose: security_opt: [no-new-privileges:true]

# Scan for vulnerabilities
# docker scout cves myimage:latest

Part 8: Useful Docker Commands

# Cleanup
docker system prune -af --volumes  # Nuclear cleanup
docker builder prune -af           # Clear build cache

# Debugging
docker compose logs -f --tail=100 api  # Follow logs
docker compose exec api bash           # Shell into running container
docker compose run --rm api pytest     # One-off command

# Build
docker compose build --no-cache api    # Force rebuild
docker compose build --parallel        # Build all in parallel

# Resource monitoring
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"

# Copy files from container
docker cp container_id:/app/data.json ./data.json

AXE MCP Server Integration

Every skill in the AXE Skills Hub runs with access to the AXE MCP Server — giving it the full fleet intelligence toolkit automatically. No setup required; tools are available in any AXE-powered session.

Core Tools Available

Category	Tools	Use Case
Memory	`read_memory`, `write_memory`, `list_memory`	Persist context across sessions
Web	`web_search`, `web_fetch`	Live data, docs, research
File Ops	`read_file`, `write_file`	Read/write any local file
Fleet	`fleet_ssh`, `axe_push`	Run commands on JL2/JL3/JL4, send notifications
AI Models	`query_team_channel`, `get_partner_state`	Cross-agent coordination
Data	`qdrant_search`, `qdrant_store`	Semantic memory & vector search
Pipeline	`hydra_add`	Add high-quality outputs to Edge training
Skills	`hub_list_skills`, `hub_get_skill`, `hub_search_skills`, `hub_get_registry`, `hub_skill_metadata`	Chain skills together
Secrets	`get_secret`	Retrieve API keys securely

Quick Start

# In any AXE session, tools are pre-loaded. Example chaining:

# 1. Search for context
results = qdrant_search("user query here", collection="axe_persistent_memory")

# 2. Fetch live data if needed
content = web_fetch("https://docs.example.com/api")

# 3. Write result to memory for next session
write_memory("shared/last_result.md", output)

# 4. Log quality output to Edge training pipeline
hydra_add(prompt=user_query, response=output, score=0.9, source="skill-name")

Edge Training Integration

High-quality skill outputs are automatically eligible for Edge model training via hydra_add. When a response scores ≥0.85 in evals, pipe it to the Hydra pipeline to compound Edge's knowledge. This is how skills make Edge smarter over time.

# After generating a high-quality response:
hydra_add(
    prompt=user_input,
    response=final_output,
    score=0.9,          # eval score
    source="skill-name" # tracks provenance
)

Metadata

Category: Infrastructure
Tier: community
Version: 1.0.0
License: MIT
Path: skills/docker-orchestration/SKILL.md

Use with an agent

Fetch this skill’s definition over the open API — no key required.

curl -s /v1/skills/docker-orchestration

View source ↗