--- title: "Data Retention" description: "Configure storage monitoring, history compression, and automatic cleanup policies" source_url: https://ai-ops.com/docs/system/retention --- # Data Retention Navigate to **System > Retention** to configure how long Koios keeps historical data and how aggressively it compresses stored values. The page is organized into three sections: storage monitoring, history compression, and retention policies. --- ## Storage Overview The storage overview card shows the current state of disk usage on the Koios server. - **Ring indicator** — percentage of disk used, color-coded: teal (healthy), orange (above 90%), red (above the alarm limit) - **Free space** — remaining capacity in GB - **Daily change rate** — average storage consumed or freed per day - **Projected time remaining** — estimated days until the disk is full, based on the daily rate - **Daily storage chart** — a bar chart showing daily storage deltas for the last 7 days. Green bars indicate growth; blue bars indicate shrinkage (from retention cleanup or manual deletion). ### Storage Alarm Limit Below the chart, a **Storage Alarm Limit** field sets the disk usage percentage that triggers a storage alarm. This setting saves automatically when changed. --- ## History Compression History compression controls how tag values are recorded in the time-series database. Rather than storing every single sample, Koios uses **Swinging Door Trending (SDT)** to discard values that don't meaningfully change the trend. ### How SDT Works SDT maintains a tolerance band — a "door" — around the last recorded point. As new values arrive, the algorithm checks whether they fall inside or outside this band. Values inside the band are redundant (the trend line already represents them) and are discarded. When a value falls outside the band, it is recorded and the door resets. The width of the door is controlled by the **Compression Deviation** setting, which is calculated as a **percentage of the tag's configured range** (Range Min / Range Max). For example, a tag with a range of 0–100 and a 2% deviation has a tolerance band of 2.0 units. A tag with a range of 0–1000 and the same 2% deviation has a tolerance band of 20.0 units. ### Settings | Setting | Description | Range | |---------|-------------|-------| | **Compression Enabled** | Master switch. When turned off, all tag values are stored without compression. Individual tags can override this with their own setting. | On / Off | | **Compression Deviation** | Maximum allowed deviation from the trend line, as a percentage of the tag's range. Lower values store more points (higher fidelity). | 0.01–100% | | **Maximum Time Between Samples** | Forces a sample after this many seconds even if the value hasn't deviated. Guarantees a minimum recording rate for stable signals. | 1–86,400s | When compression is disabled, the deviation and maximum time settings are grayed out — they have no effect until compression is re-enabled. > [!TIP] Balancing fidelity and storage > A compression deviation of 1–2% works well for most process data. Critical control signals may benefit from per-tag overrides with tighter deviation. Slow-moving values like ambient temperature can tolerate 5–10%. ### Per-Tag Compression Overrides The settings above apply globally to every tag. If a specific tag needs different compression behavior — tighter tolerance for a critical control signal, or compression disabled entirely for a diagnostic value — you can override the global settings on that tag. On a tag's **Configuration** tab, the **History Compression** section has an **Override global compression settings** toggle. When enabled, three tag-specific settings appear: | Setting | Description | |---------|-------------| | **Compression Enabled** | Whether to compress this tag's history (overrides the global switch) | | **Compression Deviation** | Tag-specific deviation percentage (overrides the global deviation) | | **Maximum Time Between Samples** | Tag-specific forced sample interval (overrides the global interval) | When the override is off, the tag uses whatever is configured here on the Retention page. The tag's configuration shows a link back to this page so you can see what the current global settings are. > [!NOTE] Override scope > Per-tag overrides are fully independent of the global settings. A tag with override enabled uses only its own compression deviation and maximum time — changes to the global settings have no effect on that tag. This is useful for tags that must always record at high fidelity regardless of how the global settings are tuned. ### Tag Range and Compression Because SDT calculates the tolerance band from the tag's range, **an incorrect range can silently prevent historization**. The default tag range is **0–100**. If a tag's actual values only span a much narrower band — for example 0–1, 4–20 mA, or 0.0–5.0 — the tolerance band is far too wide relative to the signal. A 1% deviation on a 0–100 range is 1.0 unit, which is larger than the entire 0–1 signal. The result: almost no data points pass the filter, and Trends show flat lines or missing data. **To avoid this, always set each tag's Range Min and Range Max to match the actual expected value range.** A tag that reads 0–1 should have its range set to 0–1, not the default 0–100. > [!WARNING] Common gotcha with default tag range > If a tag's live value is updating but Trends show flat lines or far fewer recorded points than expected, the most likely cause is a mismatch between the tag's range and the compression deviation. See [Tag is not historizing properly](https://ai-ops.com/docs/tags/troubleshooting.md#tag-is-not-historizing-properly) for a step-by-step fix. --- ## Retention Policies Retention policies control automatic cleanup of old data. Three independent retention periods are configurable: | Setting | Description | Range | |---------|-------------|-------| | **History Database Retention** | How long to keep time-series tag history. Specified in weeks. Leave empty for infinite retention. | 1–520 weeks | | **Event Retention** | Automatically delete events older than this many days. | 1–365 days | | **Log File Retention** | Automatically delete log files older than this many days. | 1–90 days | > [!WARNING] Reducing retention deletes data > When you reduce a retention period, existing data older than the new limit is permanently deleted. A confirmation dialog appears before saving any reduction. --- ## What's Next - [Tag Introduction](https://ai-ops.com/docs/tags/introduction.md) — tag configuration including per-tag compression overrides - [Troubleshooting a Tag](https://ai-ops.com/docs/tags/troubleshooting.md) — diagnosing tag errors including historization issues - [System Overview](https://ai-ops.com/docs/system/information.md) — disk and hardware details - [Logs](https://ai-ops.com/docs/system/logs.md) — view and stream system logs