Living Reactive Dataflow DAG & Materialized Keys

Since version v0.55.0

Spedo introduces Living Reactive Dataflow DAG & Materialized Keys, bringing instant in-memory reactive computing directly inside the cache engine.

1. Overview & Architecture

In standard caching architectures, updating a derived value (e.g. shopping cart total = items sum + tax - discount) requires the application layer to:

1. Pull all raw values over the network.

2. Recompute the derived value in Python/Node/Go.

3. Write the computed result back over the network.

4. Deal with concurrency anomalies or stale intermediate states.

With Spedo Living Reactive Dataflow, you define a DAG computation rule once on the server. Whenever any input key in the dependency graph changes, Spedo automatically re-evaluates the output node in sub-microsecond Rust time and updates the materialized key instantly!

                   +------------------------+
                   |  SET order:subtotal    |
                   |      (Value: 100)      |
                   +------------------------+
                                \
                                 \  (Auto-propagated)
                                  v
+-----------------------+     +-----------------------+     +-----------------------+
|   SET order:tax       | --> |  COMPUTED.DEFINE      | --> |  order:total          |
|    (Value: 20)        |     |  Op: SUM              |     |  (Materialized: 120)  |
+-----------------------+     +-----------------------+     +-----------------------+
                                  ^
                                 /  (Auto-propagated on mutation)
                   +------------------------+
                   |  SET order:discount    |
                   |      (Value: 15)       |
                   +------------------------+

2. Built-In Computation Operators

Operator	Description	Example Inputs → Output
`SUM`	Numerical addition across all valid numeric input keys	`[100, 20]` → `120`
`AVG`	Arithmetic average across input keys	`[10, 20, 30]` → `20.0000`
`MIN`	Minimum numeric value	`[15, 8, 42]` → `8`
`MAX`	Maximum numeric value	`[15, 8, 42]` → `42`
`CONCAT`	Comma-separated concatenation of string payloads	`["Alice", "Online"]` → `Alice,Online`
`MERGE_JSON`	Merges JSON objects from inputs into a single combined JSON	`{"a": 1}` + `{"b": 2}` → `{"a": 1, "b": 2}`

3. RESP Protocol Commands

`COMPUTED.DEFINE`

Defines a reactive computed node:

COMPUTED.DEFINE <output_key> <input_key1,input_key2,...> <operator>

`COMPUTED.GET`

Fetches current evaluated result of a computed node:

COMPUTED.GET <output_key>

`COMPUTED.LIST`

Lists all active dataflow nodes, their input dependencies, operators, and execution versions:

COMPUTED.LIST

`COMPUTED.DROP`

Deletes a computed definition and purges its materialized key:

COMPUTED.DROP <output_key>

4. Python SDK Usage

from spedo import SpedoClient

client = SpedoClient(host="127.0.0.1", port=6380)

# 1. Define Reactive Computed Node: order:total = SUM(order:price, order:tax)
client.computed_define("order:total", ["order:price", "order:tax"], "SUM")

# 2. Write initial inputs
client.set("order:price", "100")
client.set("order:tax", "20")

# 3. Read materialized output (instantly computed!)
total = client.get("order:total")
print(f"Computed Total: {total}")  # b'120'

# 4. Mutate input key -> reactive propagation occurs in Rust 0-RTT
client.set("order:price", "250")

# 5. Output key is already up to date!
new_total = client.get("order:total")
print(f"Updated Total: {new_total}")  # b'270'

# 6. List active dataflow nodes
nodes = client.computed_list()
for n in nodes:
    print(f"Node: {n['output']} <= {n['inputs']} (Op: {n['op']}, Version: {n['version']})")