In-Memory HTML Page Caching with Spedo

Since version v0.67.0

Spedo provides an ultra-fast, zero-allocation HTTP page cache layer designed to accelerate public web pages, product showcases, and documentation with sub-millisecond response latencies (X-Cache: HIT-SPEDO, < 0.2ms), while strictly bypassing and protecting private administration and authentication surfaces.

Why In-Memory Edge Caching?

Public documentation guides and dynamic portal pages often require file system reads, template engine rendering, and regex-based Markdown-to-HTML conversion on every incoming HTTP request. Under sudden traffic spikes (such as Hacker News, Reddit, or Product Hunt launches), CPU and disk I/O bottlenecks can degrade responsiveness.

By placing Spedo's memory runtime directly behind the web gateway:

• Sub-millisecond Latency: Responses are served directly from Spedo's zero-copy memory store in under 200 microseconds.
• CPU & Disk Offloading: The application framework (FastAPI/Uvicorn) avoids repetitive template and markdown parsing.
• Proven Dogfooding: Demonstrates Spedo's speed and reliability directly on its own documentation portal.

sequenceDiagram
    autonumber
    actor Client as 🌐 Web Client
    participant App as ⚡ Application Server
    participant Spedo as 🧠 Spedo In-Memory RAM (Port 6380)
    participant Disk as 📄 Markdown & Storage

    Client->>App: GET /docs/guides/ai-vectors-and-search
    App->>Spedo: GET portal:cache:page:/docs/guides/ai-vectors-and-search
    alt Cache HIT
        Spedo-->>App: Raw HTML bytes (< 0.2ms)
        App-->>Client: 200 OK (X-Cache: HIT-SPEDO, Lookup: 140µs)
    else Cache MISS
        App->>Disk: Read Markdown & render HTML (6-12ms)
        Disk-->>App: Generated HTML
        App->>Spedo: SETEX portal:cache:page:... 300 HTML
        App-->>Client: 200 OK (X-Cache: MISS-SPEDO)

Python FastAPI / Starlette Integration

Integrating Spedo in-memory page caching into any standard FastAPI, Starlette, or ASGI Python web gateway takes fewer than 35 lines of code:

import time
from spedo import Spedo
from fastapi import FastAPI, Request, Response

app = FastAPI()

# Connect using native Spedo Python SDK (or any RESP client on port 6380)
spedo = Spedo(host="127.0.0.1", port=6380, socket_timeout=0.05)

PAGE_CACHE_TTL_SECS = 300
ADMIN_PREFIXES = ("/admin", "/control", "/api/admin", "/stats", "/metrics")

@app.middleware("http")
async def spedo_page_cache_middleware(request: Request, call_next):
    # 1. Strictly bypass non-GET requests and administrative routes
    if request.method != "GET" or request.url.path.startswith(ADMIN_PREFIXES):
        response = await call_next(request)
        if request.url.path.startswith(("/admin", "/control", "/api/admin")):
            response.headers["X-Cache"] = "BYPASS-ADMIN"
        return response

    # 2. Check Spedo In-Memory Cache (sub-millisecond < 0.2ms lookup)
    cache_key = f"portal:cache:page:{request.url.path}"
    start_t = time.perf_counter()

    try:
        cached_html = spedo.get(cache_key)
        if cached_html and len(cached_html) > 0:
            lookup_us = max(1, int((time.perf_counter() - start_t) * 1_000_000))
            return Response(
                content=cached_html,
                media_type="text/html; charset=utf-8",
                headers={
                    "X-Cache": "HIT-SPEDO",
                    "X-Cache-Lookup-Us": str(lookup_us),
                    "X-Cache-Engine": "Spedo-In-Memory-Runtime",
                    "Cache-Control": f"public, max-age={PAGE_CACHE_TTL_SECS}",
                }
            )
    except Exception:
        pass  # Transparent offline fallback (zero downtime)

    # 3. Cache MISS: evaluate route dynamically and store pre-rendered HTML in RAM
    response = await call_next(request)
    if response.status_code == 200 and "text/html" in response.headers.get("content-type", ""):
        body = b"".join([chunk async for chunk in response.body_iterator])
        if len(body) > 0:
            try:
                spedo.set(cache_key, body, ex=PAGE_CACHE_TTL_SECS)
            except Exception:
                pass

        headers = dict(response.headers)
        headers["X-Cache"] = "MISS-SPEDO"
        headers["X-Cache-Lookup-Us"] = str(max(1, int((time.perf_counter() - start_t) * 1_000_000)))
        headers["X-Cache-Engine"] = "Spedo-In-Memory-Runtime"
        return Response(content=body, status_code=200, headers=headers, media_type=response.media_type)

    return response

# =========================================================================
# 4. WHERE IT'S USED: Your actual application routes!
# =========================================================================
@app.get("/docs/{slug}", response_class=HTMLResponse)
async def view_documentation(slug: str):
    # ⚡ Cache HIT: This route function is COMPLETELY BYPASSED!
    #   Spedo serves pre-rendered HTML straight from RAM in < 0.2ms.
    # ⚡ Cache MISS: This executes once, renders Markdown (12ms), and is cached in Spedo RAM.
    return render_markdown_page(slug)

@app.get("/admin/analytics")
async def admin_analytics():
    # 🔒 Admin Route: The middleware detects "/admin" and strictly BYPASSES caching.
    #   Always executes live with fresh database data and X-Cache: BYPASS-ADMIN.
    return {"live_active_users": 128}

Key Architectural Characteristics

• Native Spedo SDK & Universal RESP: Uses the official from spedo import Spedo Python client (pip install spedo), or standard redis-py interchangeably over Spedo's RESP engine on port 6380.
• Fail-Safe Offline Protection: If Spedo is restarting or temporarily unreachable, the try/except block ensures requests seamlessly fall back to dynamic generation with zero request drops (X-Cache: BYPASS-OFFLINE).
• Zero Leaks: Administrative surfaces (/admin, /api/admin/), authenticated Bearer requests, and dynamic telemetry are never cached.

Security & Exclusion Boundary

To guarantee zero data leakage and avoid caching sensitive user or operator states, the cache middleware strictly enforces the following bypass rules:

Response Headers

When caching is active, the following diagnostic headers are attached to responses:

• X-Cache: HIT-SPEDO, MISS-SPEDO, BYPASS-ADMIN, BYPASS-AUTH, or BYPASS-OFFLINE.
• X-Cache-Lookup-Us: Measured in-memory lookup duration in microseconds (e.g. 142).
• X-Cache-Engine: Spedo-In-Memory-Runtime.
• Cache-Control: public, max-age=300 (on Cache HITs).

Invalidation & Administrative Purge

Cached pages automatically expire after the configured TTL (default 300 seconds / 5 minutes).

Administrators can also trigger an immediate invalidation at any time:

1. Admin Dashboard UI: In Observability, click the 🧹 Purge In-Memory Cache button.

2. REST API: Send an authenticated POST /api/admin/cache/purge:

```bash

curl -X POST https://spedo.dev/api/admin/cache/purge \

-H "Authorization: Bearer <ADMIN_TOKEN>"

```

Response:

```json

{

"status": "success",

"purged_count": 14,

"message": "Successfully purged 14 page(s) from Spedo in-memory cache."

}

```

Configuration

The cache behavior is controlled via environment variables: