On-Demand Scanning

On-demand scanning allows AI models to trigger an immediate device read or write outside the normal scan cycle. The device side controls when cached data is fresh enough to reuse and how long to wait before executing (to batch concurrent requests).

Settings Reference

Found on the device's Configuration tab under Advanced Configuration.

On-Demand Freshness

When an on-demand read is requested, Koios checks how old the cached data is. If newer than the freshness threshold, the cached data is returned immediately — no device read needed.

Freshness = 5s

    Cache age: 3s  →  Return cached (fresh enough)
    Cache age: 8s  →  Read device (stale)

Setting to 0 means every request triggers a fresh read. This guarantees the freshest data but increases device I/O.

On-Demand Batch Window

When multiple models request reads from the same device at nearly the same time, the batch window groups them into a single device read.

Batch window = 0.5s

  t=0.0s  Model A requests read  ─┐
  t=0.1s  Model B requests read   ├─→ Single read at t=0.5s
  t=0.3s  Model C requests read  ─┘

Increasing the batch window improves efficiency but adds latency to every on-demand cycle. The model must wait for the window to close before the device is polled.

Setting to 0 means every request is executed immediately with no batching.

How It Fits Together

Model requests read → Freshness check
                         ├─ Fresh → Return cached data
                         └─ Stale → Batch window → Read device → Return fresh data
                                                                       ↓
                                                                 Model infers
                                                                       ↓
                                                                 Model writes

Total on-demand latency: (batch window if not cached) + device read time. This must be less than the model's on-demand timeout.

Configuration Guidelines