Spedo — Technical Lexicon & Architecture Reference Guide

Reference v0.64.0 — Product terms, protocol, persistence, and operations.

Support and documentation assistant

• Support Bundle / spedo-support-bundle: archive .tar.gz generated for an incident. It collects redacted, read-only diagnostic state (INFO, PERSIST.STATUS, observability and optional supplied logs), never keyspace values or credentials.
• Redaction: replacement of common secret shapes (passwords, tokens, Bearer headers, licenses and AUTH values) with [REDACTED]. It reduces exposure but does not guarantee recognition of every organisation-specific secret format.
• Spedo Guide: local, citation-first documentation assistant. V1 retrieves only reviewed public docs, has no cloud model provider, keyspace, admin, shell, Docker or outbound-network access.
• Citation-first: answer policy that attaches source documents and returns an explicit insufficient-evidence response instead of inventing a guarantee.
• Conversation journal: redacted SQLite record of question, answer, citations and UTC timestamp, retained for 30 days by default; visible read-only to the authenticated Control Center.

Target audience: Systems architects, C/Rust/Python engineers, SREs, and data engineering teams.

This lexicon describes the currently published technical contract. In case of discrepancies with earlier examples, the product maturity register and the persistence guide take precedence. Spedo remains a single-node engine: local recovery mechanisms are not high-availability (HA) cluster features.

v0.51 correctness and measurement concepts

• Hot path: the code executed for every common request, such as option-free GET or SET. A mutex lookup, key hash, allocation, or subscriber fan-out placed here is paid even when the associated feature is unused.
• Cold/control path: infrequent setup or administration work, such as enabling tracking, opening a WAL, or changing an observability flag. Moving optional setup here keeps disabled features out of the hot path.
• Canonical mutation / set_plain: one authoritative storage implementation for option-free SET. set_plain avoids generic command construction but delegates admission, max-memory, TTL removal, OCC versioning, CDC, metrics, and eviction to the same canonical rules as ordinary SET.
• Linearization point: the instant at which a concurrent operation logically takes effect. An invalidation must be published after a successful mutation, and a tracking receiver must exist before its enable acknowledgement, so a later mutation cannot disappear through a subscription race.
• Lazy invalidation subscription: ordinary connections own no broadcast receiver. A receiver is created only by CLIENT TRACKING ON or SPEDO.WATCH, and is dropped on TRACKING OFF or disconnect.
• Invalidation hub: connection-independent broadcaster carrying either a typed key invalidation or a full-flush event plus the source client ID. The typed event avoids confusing FLUSHDB with an empty key.
• NOLOOP: client-tracking option that suppresses invalidations caused by the same connection while preserving changes made by other clients.
• Lag-safe full flush: when a bounded tracking receiver falls behind, the server sends an empty RESP3 invalidation-key array, instructing the client to discard its complete local cache. Silent loss could leave stale values indefinitely.
• WAL-disabled fast path: SPEDO_WAL_FSYNC=disabled is resolved once at startup; mutations then return before touching the WAL mutex. By contrast, none, no, and off keep the WAL enabled but omit explicit fsync.
• Benchmark artifact v3: JSON contract containing immutable provenance, resolved workload, every raw trial, recomputable summaries, acknowledgement semantics, comparability, and resource-measurement quality. Its detached .sha256 hashes the exact bytes on disk.
• Provenance: commit, dirty-state diff digest, producer/configuration hashes, image identities, version, and redacted command arguments needed to identify what actually produced a result.
• Publication eligibility: explicit boolean gate. A complete run from a clean, identified source may be eligible; a dirty tree remains useful for exploration but cannot be presented as a release reference.
• Raw trial: one measured target/case/repetition with actual elapsed nanoseconds and completed operation count. A median without its raw trials is not a v3 result.
• Comparison group: identifier pairing equivalent Redis and Spedo trials executed under the same case and repetition conditions.
• Strict comparability: ratio eligibility requiring paired Redis/Spedo trials with identical confirmed_resp acknowledgement semantics. Local enqueue and L1-cache hits are capabilities, not server comparisons.
• Coefficient of variation (CV): sample standard deviation divided by sample mean. High CV flags an unstable campaign; it must not be disguised by a rounded median.
• Bootstrap 95% confidence interval (CI95): interval estimated by resampling paired trial ratios. A performance win is not claimed when the lower bound fails to exceed 1, even if the median ratio is above 1.
• Measurement digest: semantic SHA-256 over raw measurements and workload, excluding release labels and timestamps. It detects a copied benchmark renamed as a new version.
• Unknown resource value: represented by null with quality unavailable; zero means a counter was measured and its value really was zero.

📑 Table of Contents

1. Concurrency Model, Networking & Threading

2. C Client Accelerator & Zero-Copy Serialization

3. Reactive Variables & Real-Time Push Invalidation

4. State Machines & Distributed Workflow Orchestration

5. AI Vector Engine & SIMD Computing

6. Priority Queues & Fair Scheduling

7. Memory Density, Retention & Persistence

8. Persistence, Recovery & HA Boundaries

9. Data Structures & Memory Allocation

10. Python SDK, Synchronization & Data Models

11. Abbreviations, Variables and Conventions

12. Executive Summary for Technical Audits & Engineering Due Diligence

1. Concurrency Model, Networking & Threading

• CPU Affinity Attempt and SO_REUSEPORT (v0.50.0+)

• Implemented behavior: on Linux, Spedo makes a best-effort affinity call around listener startup and uses SO_REUSEPORT for multiple listeners. Tokio tasks may migrate after an await; therefore this is not evidence of a strict one-connection/one-core or one-worker/one-core architecture.
• Operational rule: treat affinity as an experiment until sched_getcpu() tests under a restricted cpuset prove ownership across await points. The portable fallback remains ordinary TCP/IP and Tokio scheduling in Docker, Kubernetes, and cloud VPCs.

• Adaptive Lease Observe Mode

• Implemented behavior (v0.51.0): SPEDO_ADAPTIVE_LEASE_MODE=observe constructs a connection-local classifier, samples key hashes, and records temporary lease experiments. The default is disabled, so no classifier is constructed and no key is hashed for this feature.
• Boundary: observe mode does not route a request to a core owner and does not change RESP semantics. It is telemetry for a future design, not a production affinity guarantee or a demonstrated cache-coherence speedup.

• SO_REUSEPORT Multi-Workers vs Single-Thread Event Loop (Redis ae.c)

• Spedo Implementation : Spedo opens N independent listener sockets (socket2::Socket) on the same TCP port (6380) using the kernel SO_REUSEPORT socket option. The Linux/POSIX kernel distributes incoming client connections directly across 16 Tokio worker threads via hardware 4-tuple IP/Port hashing. No central router thread or lock bottleneck is required.
• Redis Implementation : Redis historically relies on a strictly single-threaded event loop (ae.c based on epoll/kqueue). Even with Threaded I/O introduced in Redis 6 (networking.c), auxiliary threads only read/write socket bytes, while 100% of command executions remain serialized on a single CPU core.
• Spedo Advantage : Complete elimination of the single-core bottleneck. Spedo saturates all server cores to achieve over 385,000 multi-worker read ops/s.

• Sharded Memory & Lock Striping vs Global Table dict.c (Redis)

• Spedo Implementation : Primary memory is partitioned into 256 independent Shards aligned to 64-byte hardware cache lines (#[repr(align(64))]). Each shard is an isolated structure Shard { entries: RwLock<HashMap<Vec<u8>, Value>> }. Key access locks only its specific shard via a shared reader (read()) or exclusive writer (write()). Requests targeting keys in different shards execute concurrently with 0% lock contention.
• Redis Implementation : Redis uses a single global hash table dict.c. Resizing the table requires incremental rehashing across requests, which must traverse table buckets while blocking the main thread.
• Spedo Advantage : Massively parallel write throughput with zero "stop-the-world" latency spikes during scale-up.

• Lockless Atomic Clock vs server.lruclock & robj Bitfields (Redis)

• Spedo Implementation : LRU access metadata and value read counters are stored in hardware atomic CPU fields (AtomicU64). A read operation (GET) increments the access counter without ever acquiring an exclusive write lock on the shard (shared read() lock only).
• Redis Implementation : Redis packs the LRU timestamp into 24 bits of the robj object header (unsigned lru:LRU_BITS). Updating it requires mutating the object header within the single main thread loop.
• Spedo Advantage : Ultra-fast read hot-path with zero CPU cache-line invalidation across worker threads.

• TCP_NODELAY & Nagle's Algorithm vs Default Network Buffering

• Principle : Nagle's algorithm merges consecutive small TCP packets by buffering data until previous packet acknowledgments (ACKs) are received. While reducing network packet counts, it introduces an artificial latency penalty of 10 to 40 ms on interactive request-reply workloads.
• Spedo Implementation : All client and server TCP sockets immediately enforce set_nodelay(true). Every RESP command or response is pushed instantly onto the network interface card without waiting for intermediate ACKs.
• Spedo Advantage : Ultra-low P50/P99 network latency (< 0.2 ms in local/container VPC networks).

• Epoll, Kqueue & IO_Uring (Asynchronous I/O Multiplexing)

• Principle : Operating system syscall mechanisms allowing a single thread to monitor tens of thousands of concurrent network connections in $O(1)$ complexity without blocking polling loops:

- epoll : Scalable Linux I/O event notification facility.

- kqueue : High-performance event filter mechanism on macOS / FreeBSD.

- io_uring : Next-gen Linux interface using dual kernel/user-space shared ring buffers for zero-copy non-blocking asynchronous I/O.

• Spedo Implementation : Built upon the Rust Tokio runtime compiled natively for each OS target (epoll on Linux, kqueue on macOS), maximizing host kernel throughput.

• CAS (Compare-And-Swap) & CPU Atomic Primitives

• Principle : Hardware CPU instructions (CMPXCHG on x86, LDREX/STREX on ARM) that atomically update a memory address only if it matches an expected value, executing in a 100% indivisible step.
• Hardware Impact : Enables lock-free data structures. No OS thread is put to sleep by the kernel scheduler, eliminating thread context-switch penalties (~1.5 µs each).
• Spedo Implementation : Extensively leveraged in internal monotonic clocks, real-time metric counters, transaction sequence counters, and work dispatchers.

2. C Client Accelerator & Zero-Copy Serialization

• spedo_core.c vs hiredis & redis-py

• Spedo Implementation : Pure C compiled extension (spedo_core.c with -O3 -fPIC) bound directly to CPython internal object structures:

1. encode_command : Precomputes exact byte length and allocates a single contiguous raw memory buffer (malloc), serializing arguments via pointer arithmetic without allocating intermediate Python string objects.

2. decode_resp : Parses RESP protocol frames (+, -, :, $, *) in a single linear pass over the receive buffer with zero intermediate heap allocations.

3. execute_pipeline : Batches thousands of commands into a single sendall() syscall and decodes all responses in a tight C loop.

• Redis Implementation : Pure Python redis-py performs extensive bytes, list, and dict allocations, triggering heavy Python Garbage Collector pressure. Even with hiredis, Python wrapping layers retain significant interpreter overhead.
• Spedo Advantage : 185,239 ops/s pipelined throughput within a single Python client process.

• GIL Release (Py_BEGIN_ALLOW_THREADS) vs GIL Contention

• Spedo Implementation : Inside spedo_core.c, all blocking network syscalls (getaddrinfo, connect, send, recv) are systematically wrapped in Py_BEGIN_ALLOW_THREADS / Py_END_ALLOW_THREADS. The Python Global Interpreter Lock (GIL) is released during socket I/O.
• Standard Implementation : Standard Python drivers frequently hold the GIL during I/O or serialization, preventing concurrent Python application threads from executing.
• Spedo Advantage : True multi-threaded Python parallelism without CPU lock starvation.

• Dynamic Buffer Shrinkage (maybe_shrink_buffer)

• Spedo Implementation : When large payloads (e.g., 5 MB) flow through the C accelerator, the receive buffer recv_buf grows dynamically. Once the payload is consumed and cleared, maybe_shrink_buffer automatically reallocates the buffer back to its default baseline (64 KB) to immediately return memory to the OS.
• Redis Implementation : Client drivers often retain peak allocated buffer capacities for the lifetime of the connection, causing heap fragmentation and lingering memory consumption.

• RESP2 vs RESP3 (REdis Serialization Protocol v2 & v3)

• Principle : RESP is the text/binary wire protocol used to exchange commands and replies:

- RESP2 : Historical standard limited to primitive types: Simple Strings (+), Errors (-), Integers (:), Bulk Strings ($), and Arrays (*).

- RESP3 : Modern extension introducing rich semantic types: Dictionaries / Maps (%), Sets (~), Booleans (#), Nulls (_), Doubles (,), Verbatim strings (=), and out-of-band asynchronous Push frames (>).

• Spedo Implementation : Streaming zero-allocation parser in Rust and C supporting both RESP2 and RESP3, with native extensions for tensor matrices, AI vectors, and workflow leases.
• Spedo Advantage : Unambiguous client-side typing with zero intermediate JSON re-encoding overhead.

• Apache Arrow & Columnar Zero-Copy Format

• Principle : Standardized in-memory contiguous columnar format for tabular and AI vector data. Instead of row-oriented storage, records are aligned column-by-column with null bitmasks.
• Hardware Impact : Allows CPU SIMD instructions to scan, filter, and aggregate millions of numerical values per second without deserialization. Processes can share the same memory region via zero-copy IPC.
• Spedo Implementation : Target format for dense matrix exchange, LLM KV-cache tensor sharing, and distributed DataFrames.

• JSON RFC 7396 (JSON Merge Patch) vs RFC 6901 (JSON Pointer)

• Principle :

- RFC 7396 (JSON.MERGE) : Applies a recursive diff patch to an existing JSON document (overwrites supplied fields, deletes null fields, preserves untouched keys).

- RFC 6901 (JSON.GET /path) : Directly targets and extracts a specific nested path (/users/42/address/city).

• Spedo Implementation : Executed in-situ inside server RAM via optimized Rust (serde_json), eliminating the need to transfer full documents across the network for client-side patching.
• Spedo Advantage : Saves up to 95% network bandwidth on large JSON document updates.

3. Reactive Variables & Real-Time Push Invalidation

• SpedoLiveVar & SPEDO.WATCH vs Redis Client Side Caching (CLIENT TRACKING)

• Spedo Implementation :

- SPEDO.WATCH registers the client socket in the server invalidation broadcast registry (broadcast::Sender).

- SpedoLiveVar caches the decoded value directly in the local RAM of the application process.

- Read : $0\text{ ms}$ network latency, instantaneous local RAM read ($< 0.0001\text{ ms}$, +2.56 million reads/s).

- Mutation : Whenever a key mutates on the server, a compact push invalidation frame is broadcast in real time. The client updates its local cache and immediately wakes waiting coroutines.

• Redis Implementation : Redis introduced Client Side Caching in RESP3 (CLIENT TRACKING on). However, it requires complex management of slot tables, out-of-band messages, and provides no native application bindings (developers must manually build synchronization, locks, and listeners).
• Spedo Advantage : Turnkey API (var = client.bind_var("config")), native await var.wait_until("DONE"), zero network polling loops.

4. State Machines & Distributed Workflow Orchestration

• Workflow Engine ($O(1)$ Atomic Claim) vs Redis Streams & Lua Redlock

• Spedo Implementation : Native Rust state engine on the current node (src/features/workflow.rs):

- WF.SUBMIT : Enqueues document into the step's FIFO queue ($O(1)$).

- WF.CLAIM : Atomically dequeues the next document and assigns an execution lease with TTL to a worker_id ($O(1)$).

- WF.TRANSITION : Validates completion of the current step and atomically moves the document to the next stage ($O(1)$).

- WF.RECOVER : Scans expired leases and requeues them only when a worker or scheduler explicitly invokes the command.

• Redis Implementation : Requires composing Streams (XADD, XREADGROUP, XACK), Pending Entries Lists (PEL), Sorted Sets for delayed retries (ZSET), and complex Lua scripts to maintain atomicity. Worker crash recovery requires dedicated pollers executing XPENDING and XCLAIM.
• Boundary : Claims and transitions are checked atomically by this single-node engine. It is not a durable event log, HA/failover mechanism, or automatic recovery service; WorkflowRunner is the optional helper that periodically calls WF.RECOVER while it is alive.

5. AI Vector Engine & SIMD Computing

• SIMD Cosine Similarity vs RediSearch Vector Index

• Spedo Implementation :

- Built-in native vector search engine (src/storage/vectors.rs).

- On VECTOR.ADD, vectors are L_2-normalized upon insertion and stored as CPU-aligned contiguous memory blocks (repr(C) / Vec<f32>).

- VECTOR.SEARCH computes dot products via hardware vector instructions: AVX2 / AVX-512 / ARM NEON (executing 8 to 32 floating-point FMA operations per clock cycle).

- Maintains Top-K via a fixed-size BinaryHeap (Min-Heap) of size K, avoiding sorting the full dataset ($O(N \log K)$).

• Redis Implementation : Requires installing the proprietary RediSearch module (redisearch.so). On Community Edition Redis, vectors must be pulled to the client (massive MGET + NumPy in Python), saturating network bandwidth.
• Spedo Advantage : Top-5 search in 0.90 ms (vs 43.1 ms external NumPy round-trip, 47.9× faster).

• Vector Quantization (SQ8 / Int8)

• Principle : Transforms float32 (4 bytes, 32 bits) vector dimensions into signed int8 (1 byte, 8 bits) integers via linear scaling:

x_int8 = round(127 × (x - min) / (max - min))

• Hardware Impact :

- 4x RAM reduction (6.14 GB → 1.53 GB for 1M vectors).

- VNNI / AVX2 CPU acceleration : Computes 32 dimensions in a single CPU instruction instead of 8, accelerating search throughput by 2x to 4x while preserving 99.5% semantic fidelity.

• In-Situ Computing (In-DB) vs Ex-Situ Client Processing (NumPy/Pandas)

• Principle :

- Ex-Situ (Traditional) : Database returns raw payload over the wire (MGET, JSON.GET), client deserializes and executes computations (NumPy matrix multiply, attention KV-cache, logit filtering). Network and serialization create a massive bottleneck.

- In-Situ (Spedo) : Mathematical and vector computations execute directly in Spedo server RAM in Rust with SIMD hardware acceleration. Only the final scalar result or Top-K array is returned.

• Spedo Advantage : Throughput increased by 5x to 80x; network bandwidth consumption reduced by 98%.

• GEMM (General Matrix Multiply) & FMA (Fused Multiply-Add)

• Principle :

- GEMM : Fundamental linear algebra operation C = α(A · B) + βC, at the core of Transformer attention layers and LLM inference.

- FMA : x86/ARM processor instruction computing a × b + c in a single clock cycle with a single numerical rounding step (high precision).

• Spedo Implementation : Native matrix engine (MMUL, MATRIX.DOT) partitioned into contiguous memory blocks with 8-way unrolled FMA loops.

• Contiguous Matrix Engine (0.3 ns) vs Key-Value Hash Table (50-150 ns)

• Principle :

- Key-Value O(1) : Requires computing a hash (SipHash), bucket modulo division, traversing scattered DRAM pointers (Pointer Chasing), and string equality checks → 150 to 350 CPU cycles (50 to 150 ns).

- Contiguous Matrix O(1) : Pure arithmetic RAM offset offset = (row cols + col) 4 bytes → 1 CPU cycle (0.3 ns) with continuous hardware L1 cache-line prefetching (64 bytes).

• Spedo Advantage : Direct scalar element access in 0.24 µs, 100% vectorized matrix multiplication without heap allocations.

6. Priority Queues & Fair Scheduling

• Spedo FairShare 7:2:1 — ordonnanceur batch pondéré et work-conserving

• Spedo Implementation :

- QPUSH queue item [PRIORITY 0|1|2] [DELAY secs] [FAMILY tenant]

- Three priority tiers: P0 = Critical, P1 = Standard, P2 = Low-Priority/Batch.

- A canonical QPOP ... 10 batch targets 7 tasks from P0, 2 from P1 and 1 from P2 when all tiers have enough ready work.

- The ratio is per batch, not a persistent credit window across calls; count=1 is strict priority.

- If a tier is empty, its share is reassigned dynamically (surplus borrowing) so ready capacity is not wasted.

- Inside each tier, FAMILY buckets are interleaved round-robin for multi-tenant fairness.

• Redis Implementation : BRPOP queue_p0 queue_p1 queue_p2 strictly checks queues in sequential order. Continuous message arrivals in queue_p0 cause queue_p1 and queue_p2 to NEVER be processed (Total Task Starvation).
• Spedo Advantage : Deterministic 7:2:1 allocation for ten-item dequeue batches, surplus borrowing between tiers, and round-robin FAMILY rotation inside each tier. Single-item pops intentionally remain strict-priority.

7. Memory Density, Retention & Persistence

• COL, ColdArchive and Tiered Storage vs Eviction

• COL (Cost Of Life) : Memory pressure supervisor. When threshold limits are reached, it shortens long TTLs; it is a memory management policy, not durable storage.
• ColdArchive / Tiered Storage : In-memory compression and memory overflow mechanics. Retains inactive keys in compressed RAM partitions to prevent eviction drop loss.
• LZ4 : High-speed lossless compression codec (> 4 GB/s). During KV/JSON checkpointing, compressed records are materialized into canonical byte sequences.
• Redis / Spedo Eviction : Eviction policies handle RAM pressure; they never replace a durability or backup contract.

• Snapshot SPDO2 vs Generic Memory Snapshot

• Snapshot / Checkpoint : Point-in-time durable state of KV/JSON data at a given WAL sequence. SAVE triggers async background writing; SAVE WAIT blocks until disk commit is confirmed.
• SPDO2 : Versioned, bounded snapshot format containing KV records, absolute Unix TTL timestamps, record count counters, and CRC-32C checksums. The reader validates file integrity before atomically replacing active keyspace memory.

• NVMe (Non-Volatile Memory Express) vs SATA SSD vs HDD

• Principle :

- NVMe : PCIe-attached flash storage protocol supporting up to 64,000 command queues of 64k entries each, with ultra-low latency of 10 to 30 µs.

- SATA SSD (AHCI) : Legacy protocol designed for spinning disks, limited to 1 command queue of 32 entries (latencies 100-500 µs).

- HDD (Mechanical Disks) : Physical rotating platters, 5 to 15 ms latency (500x slower than NVMe), limited to 100-200 IOPS.

• Spedo Impact : WAL and checkpoint engines exploit NVMe multi-queue parallelism to eliminate storage commit bottlenecks.

• IOPS (Input/Output Operations Per Second) & Throughput (MB/s)

• Principle :

- IOPS : Number of distinct read/write storage operations per second (critical for small random WAL appends and fsync operations).

- Throughput (MB/s) : Total byte volume transferred per second (critical for bulk snapshot creation and recovery).

• Saturation Formula : Throughput (MB/s) = (IOPS × Block Size (KB)) / 1024. Writing 10,000 WAL entries/s with fsync=always consumes 10,000 disk IOPS.

• Linux Kernel Page Cache & Direct I/O (O_DIRECT) vs Buffered I/O

• Principle :

- Buffered I/O (Default) : File writes enter kernel RAM buffers (Page Cache). Syscall write() returns in ~1 µs, but data is not yet committed to physical flash media.

- Direct I/O (O_DIRECT) : Bypasses kernel Page Cache and writes directly from user-space buffers to hardware controllers.

- fsync() : Forces kernel to flush dirty Page Cache blocks to physical storage media before returning success.

• Spedo Implementation : Config SPEDO_WAL_FSYNC=always guarantees fsync before issuing client ACK for strict local durability.

8. Persistence, Recovery & HA Boundaries

• Durability, ACK and Local Volume

• ACK (Acknowledgment) : Success response returned to client. A response OK represents durable write confirmation only when SPEDO_PERSIST_PATH is set and SPEDO_WAL_FSYNC=always.
• Local Durability : Resilience against process or container restarts via host-attached volume. It depends on underlying volume reliability and does not protect against physical disk destruction or regional outages.
• Backup : Independent, verified copy stored outside the primary working volume. A local snapshot file alone is not an offsite backup.

• WAL, AOF, Sequence and Watermark

• WAL (Write-Ahead Log) : Binary append-only log written prior to issuing durable ACKs. Each KV/JSON record carries a monotonic sequence number, record type (SET, DEL, FLUSH), bounded payload lengths, absolute expiration timestamp, and CRC-32C checksum.
• SPWL1 : Magic byte header identifying Spedo binary WAL format.
• WAL Record / Frame : Encoded journal unit (header, payload, CRC). Corrupt lengths or CRC mismatches fail immediately rather than silently replaying garbage.
• Watermark : Highest WAL sequence number committed into a validated checkpoint. Startup replay processes only entries newer than the watermark.
• Compaction : Safe WAL rewrite following checkpoint completion to purge acknowledged transactions.

• fsync, WAL Modes and Atomic Commit

• fsync / sync_all() : Instructs OS filesystem to flush pending blocks to physical storage.
• SPEDO_WAL_FSYNC=disabled : Default mode; no WAL active; RPO equals the last valid checkpoint.
• always : Append WAL and invoke fsync before client ACK. Single-node synchronous local durability.
• everysec : Append WAL and flush approximately once per second. Data loss bounded to ~1 second window.
• none : Append WAL without explicit fsync.
• Atomic Snapshot Commit : Unique temporary file on same filesystem → fsync file → rotate .previous → atomic rename to current → fsync parent directory. Prevents corrupt files from replacing valid checkpoints.

• SAVE, SAVE WAIT and PERSIST.STATUS

• SAVE : Initiates async background checkpoint and returns immediately.
• SAVE WAIT <timeout_ms> : Blocks until checkpoint is fully committed to disk (fsync, rename, dir fsync); bounded between 1..600000 ms.
• PERSIST.STATUS : Administrative operational inspection command returning file paths, active status, last success/error, byte size, record count, recovery provenance, watermark, and WAL metrics.

• Recovery, Integrity and Failure Testing

• Recovery : On boot, Spedo verifies current, falls back to .previous if corrupted, and replays WAL entries past watermark.
• SPEDO_RECOVERY_POLICY=fail : Halts startup if persistence files are corrupted. Default with always WAL.
• empty_on_invalid : Boots empty cache profile while preserving error diagnostics in PERSIST.STATUS.
• CRC-32C : Hardware-accelerated Castagnoli CRC checksum detecting bit flips and incomplete writes.
• Absolute TTL : Unix epoch expiration timestamps stored on disk so offline downtime is accurately deducted upon recovery.

• HA, RPO/RTO and Non-Existent Features

• HA (High Availability) : Automated multi-node failover and replication. Spedo v0.36 is a single-node engine and is not an HA cluster.
• RPO (Recovery Point Objective) : Maximum acceptable data loss window. always targets zero loss for ACKed writes on local storage.
• RTO (Recovery Time Objective) : Target service recovery time upon restart.
• Distributed Quorum & Fencing : Raft/Paxos consensus, split-brain protection, and distributed fencing are not implemented in single-node Spedo.

• P0 to P3 Milestones of the Snapshot Plan

In PLAN_SNAPSHOT_FIABILITE.md, P0 designates observable persistence status, P1 verified SPDO2 snapshots, P2 crash tests and operations, and P3 single-node WAL logging. These represent single-node durability milestones, not an HA roadmap.

9. Data Structures & Memory Allocation

• CPU Cache Hierarchy (L1, L2, L3) & Cache Lines (64 bytes)

• Principle :

- CPU Registers : ~0.3 ns (immediate ALU access).

- L1 Cache (Data/Instruction) : ~0.9 ns (32 to 64 KB per core).

- L2 Cache : ~2.8 ns (512 KB to 1 MB per core).

- L3 Cache (Shared) : ~11 to 15 ns (16 to 128 MB shared across all cores).

- Main DRAM : ~60 to 100 ns (100x slower than L1 cache).

• Cache Line : CPU cores never fetch a single byte from RAM; they load contiguous blocks of 64 bytes.
• Spedo Impact : Tensors, SIMD vectors, and shard metadata structures are aligned to 64-byte boundaries to maximize cache-line utilization.

• Pointer Chasing & Cache Miss vs Contiguous Memory

• Principle :

- Pointer Chasing : Traversing fragmented pointer structures (linked lists, sparse trees). Jumping to unpredictable memory addresses causes L1/L2 Cache Misses, stalling the CPU pipeline for ~200 clock cycles while awaiting DRAM.

- Contiguous Memory (Flat Arrays / Slices) : Storing data in flat vectors (Vec<f32>, Bytes). The CPU Hardware Prefetcher streams subsequent elements into L1 cache before they are requested.

• Spedo Advantage : Matrix and vector scans execute up to 30x faster than traditional pointer-chasing data structures.

• Latency Percentiles (P50, P90, P99, P99.9) & Jitter

• Principle :

- P50 (Median) : 50% of requests complete faster than this threshold.

- P90 / P95 : Reflects performance experienced by the majority of requests.

- P99 / P99.9 (Tail Latency) : Maximum latency experienced by the slowest 1% or 0.1% of requests.

- Jitter : Latency variability caused by Garbage Collection pauses or lock contention.

• Spedo Advantage : Written in Rust with no Garbage Collector and 64-partition lock striping, delivering stable P99 tail latency under 0.2 ms.

• Backpressure & Flow Control

• Principle : Protective flow control mechanism triggered when producers submit requests faster than consumers, network, or storage can ingest.
• Spedo Implementation : Strict bounding on priority queues and streaming channels to rate-limit incoming traffic before encountering Out-Of-Memory (OOM) conditions.

10. Python SDK, Synchronization & Data Models

• SpedoClient, SpedoDict, SpedoList and @spedo_model

• SDK : Python client communicating via RESP to Spedo. Local optimizations (pipeline, L1 proxy, write-behind) provide client-side acceleration.
• SpedoDict / SpedoList : Python dict and list abstractions mapped directly to Spedo for seamless distributed state sharing.
• @spedo_model : Class decorator synchronizing attributes of dataclasses, Pydantic models, or Python classes via Spedo.

• auto_sync(globals()), Globals and Claim-Check Pattern

• auto_sync(globals()) : Inspects functions already defined in a module, pulls matching global: keys before each wrapped call, then pushes detected changes afterward. It is intended for small, low-contention convenience state.
• Consistency Boundary : There is no watcher, lock, compare-and-set, transaction, or immediate cross-instance propagation. Concurrent read-modify-write is non-atomic and resolves last-writer-wins; use an explicit atomic server command for correctness-sensitive shared state.
• Python Global : Variable stored in a process module dictionary. Functions added after auto_sync() are not automatically wrapped, and in-place mutations can evade its reference-based change detection.
• Claim-Check / Thin Envelope Pattern : Messaging pattern where event payloads store a compact reference ID, keeping large binary payloads in Spedo RAM.

11. Abbreviations, Variables and Conventions

12. Executive Summary for Technical Audits & Engineering Due Diligence

When communicating technical architecture with engineering leads, principal architects, or auditors:

1. Concurrency : « Spedo leverages SO_REUSEPORT and a 64-partition sharded memory architecture; throughput figures vary by workload and are documented in PERFORMANCE.md. »

2. Client & Networking : « spedo_core accelerates the Python SDK and releases the GIL during network I/O; local L1 hits or write-behind queues have different operational semantics than confirmed server ACKs. »

3. Durability : « KV/JSON mutations achieve synchronous local durability only when SPEDO_PERSIST_PATH is configured with SPEDO_WAL_FSYNC=always. Single-node SPDO2 snapshots and WAL logs do not constitute remote offsite backups or distributed HA. »

4. Memory & Reactivity : « COL, in-RAM compression, and SPEDO.WATCH are high-performance memory and synchronization primitives; they do not replace distributed multi-datacenter consensus or durable streaming brokers. »

Spedo — Technical & Architectural Reference Lexicon v0.64.0