Spedo Benchmark & Observability

Name: Spedo In-Memory Performance Benchmark Dataset
Creator: Spedo

v0.69.0

Performance Dashboard & Real-World Architectural Benchmark vs Redis 7.4

YCSB-inspired access profiles Exploratory

Exploratory memtier profiles named after familiar access mixes. This is not the Yahoo YCSB harness and must not be cited as an official YCSB result: Workload A (Update Heavy 50/50), Workload B (Read Heavy 95/5), Workload C (Read Only 100%), Workload D (Read Latest 95/5), and Workload F (Read-Modify-Write / Atomic CAS). Measured with 40 concurrent connections, 50,000 keys dataset, 128B payload, and 16-deep pipeline.

YCSB-inspired access-profile matrix

Exploratory Redis 7.4 and Spedo memtier profiles; not the official YCSB client or semantics

Workload / Scenario	Access Profile & Real-World Use Case	Distribution	Redis 7.4 (ops/s)	Spedo (ops/s)	Speedup vs Redis	Spedo Latency P50 / P99	Status

Exploratory throughput by access profile (ops/sec)

Visual comparative throughput between Redis 7.4 and Spedo

Zipfian Skewed Distribution (Power-Law Hotspots θ=0.99) 80/1 Hotspot & L1 Hot Proxy

Models real-world production cache behavior where 1% of hot keys capture ~80% of total read traffic (Zipfian skew factor $ heta = 0.99$). Compares traditional Redis TCP network lookups against Spedo Multi-Thread TCP and Spedo's in-process L1 Hot Proxy with reactive bind_var and server push invalidations.

3-Tier Access Hierarchy: Network vs In-Process Memory

Throughput and sub-millisecond to nanosecond latency across access tiers

Access Tier	Architecture & Mechanism	Redis 7.4 (ops/s)	Spedo (ops/s)	Speedup	Latency P50 / P99	Advantage / WOW Factor

Access Tier Throughput & Latency Scaling

Comparing TCP socket network roundtrip vs in-process reactive memory cache

RepoFlow-inspired memtier profiles Exploratory / Not externally comparable

Local memtier profiles inspired by scenarios discussed in the RepoFlow article « Redis vs Valkey vs DragonflyDB vs KeyDB Benchmarks » (source article). They do not reproduce its hardware, binaries or full methodology, so external engine figures are context only and are not ranked against this host. Using redislabs/memtier_benchmark (native C++ epoll/event socket engine), this suite measures true raw multi-core server throughput and sub-millisecond latencies without Python client CPU bottlenecks.

Multi-Engine Comparative Throughput & Latency Matrix

Local Redis 7.4 and Spedo profiles only; external engine references are not comparable measurements

Workload / Scenario	Configuration	Redis 7.4 (ops/s)	Valkey 7.2 (ref)	KeyDB v6 (ref)	DragonflyDB (ref)	Spedo (ops/s)	Speedup vs Redis	Spedo Latency P50 / P99	Status

Throughput Comparison by Workload (Ops/sec)

Visual throughput distribution across tested engines

Benchmark scope and product truth

Core RESP figures compare standalone Redis and standalone Spedo only when both receive the same confirmed network replies. Local L1 hits, write-behind enqueue paths, preview features, snapshots, and experimental control-plane APIs have different semantics and are never evidence of server throughput, durability, high availability, or Redis Cluster equivalence. See PLAN_FIABILITE_PRODUCTION.md for the current product contract and validation gates.

Real-World Comparative Matrix: Redis 7.4 Application vs Spedo Full-Stack Application

Real-world production gains measured on identical hardware and container environments

Use Case & Feature	Redis 7.4 Approach	Spedo Full-Stack Approach	Redis Speed	Spedo Speed	Real Speedup	Key Architectural Advantage
Hot Key Read (Single GET)	Round-trip TCP socket (network RTT)	L1 In-Process Cache + Push Invalidation	4,029 GET/s	398,747 GET/s	99.0× (+9,798%)	Zero network RTT latency, automatic real-time push invalidation
Reactive Variable / Shared Counter	Repeated TCP polling queries	Direct Memory Pointer (`LiveKey`)	4,188 reads/s	3,389,319 reads/s	809× (+80,834%)	Direct RAM access (0 ms) synchronized by CDC push
AI Vector Search (2000 vecs 64D)	Network MGET of 2000 vectors + NumPy calculation	In-Engine Rust/C SIMD AVX2 (`VECTOR.SEARCH`)	40.5 ms / req	0.8 ms / req	51.2× (+5,018%)	Zero network transfer of vector embeddings, native SIMD SQ8 math
Single-Threaded Write (Single SET)	Synchronous blocking on every socket ACK	Local Enqueue + Async Write-Behind	3,796 SET/s	34,051 SET/s	9.0× (+797%)	Application thread never blocks on write network latency
Partial JSON Document Update	Download 50 KB + modify + re-upload 50 KB	Targeted RFC 7396 Merge Patch (`JSON.PATCH`)	50 KB payload	32 bytes patch	50× Bandwidth Savings	In-memory atomic mutation without full serialization round-trips
Business Logic Scripting & Transactions	Interpreted single-threaded Lua engine	Compiled Rhai Rust Engine (`EVALSHA`)	3,785 tx/s	5,888 tx/s	1.6× (+56%)	Pre-compiled AST, multi-threaded parallel execution
RAM Saturation (70 MB / 64 MB Limit)	Outright eviction (keys deleted)	Transparent NVMe Tiered Storage	18.7% keys dropped	0% keys dropped	Zero Data Loss	Transparent LZ4 compressed spillover to SSD/NVMe (~50µs)
High Availability & Failover	Complex external Sentinel / Cluster setup	Instant In-Process Failover / Active-Passive	External Ops	Native 0s downtime	Zero Ops	Immediate master recovery with Kubernetes PVC / block volume
Multi-Connection Throughput (16x16)	Single-threaded Redis (1 core CPU bottleneck)	Native Multi-Core Sharded `SO_REUSEPORT`	171,798 ops/s	220,358 ops/s	1.28× (+28.3%)	Full hardware multi-core CPU saturation in a single container

Spedo Pipeline (P50)

—

Redis 7.4 Pipeline (P50)

—

Core RESP Direct Gap

—

Historical Versions Tracked

—

💡 Methodological Note: The Core RESP direct benchmark isolates key creation and overwrite, using dedicated keys per worker without database reset after warm-up.

Historical Pipeline Evolution (SET ops/s)

Spedo Core Redis 7.4

Multi-Scenario Direct RESP Matrix (`make matrix`)

Scenario:

Version	Date	Redis 7.4 (ops/s)	Spedo (ops/s)	Speedup	Status

Complete Pipeline History (SET ops/s)

Version	Date	Redis SET/s	Spedo SET/s	Delta	Context & Release Notes

SDK Capabilities & In-Process Local Cache (`make bench-spedo-capabilities`)

Application-level benchmarks (each row explicitly states its ACK delivery semantics)

Version	Application Capability	Throughput (Ops/s)	ACK Semantics	Comparison Scope

Spedo Exclusive Killer Features — WOW Effect (`make wow`)

SIMD Vector Search, Fair Priority Queues, Cold Archive RAM Retention, Live Reactive Variables

Version	Feature	Redis / Python Equivalent	Spedo Native	Speedup	Scope & WOW Factor

Sustained Throughput (64 clients)

24,592 req/s (160,000 ops)

Sustained Throughput (128 clients)

23,268 req/s (256,000 ops)

Overall Error Rate

0.00% (0 errors)

Memory Leak (RAM Delta)

+0.35 MB (Zero Leak)

High-Concurrency Stress Test (`make stress`) & Live Demo (`make demo`)

Benchmarks under concurrent traffic (80% GET, 15% SET, 5% JSON/Vectors) & crash recovery validation (SIGKILL + Fsync snapshot)

Version	Test Scenario	Concurrency / Volume	Throughput	Latency P50 / P99	Error Rate	RAM Behavior / Integrity