Continued training: reinforcement‑learning control that keeps improving.
A study showing that periodic re‑validation and incremental retraining let a reinforcement‑learning controller keep gaining after it is deployed, the way a human operator keeps getting better with experience. Proven on a simple but unforgiving tank‑level loop balancing multiple feeds where downstream flow stability was paramount: each retraining iteration raised performance, and the second model reached 2.8× a well‑tuned PID, still trending upward.
A deployed control model does not have to be frozen in time. This study set out to prove a simple idea: that incremental, continued training lets a reinforcement‑learning (RL) controller go on improving through successive validated retraining builds, mirroring how a human operator keeps building skill as new situations and operating experience accumulate.
The proving ground was a simple level‑control loop feeding a tank from multiple sources, where holding a steady downstream flow mattered most. A well‑tuned PID set the baseline. The first RL model reached 2.6× that PID's capability; after a second incremental training iteration it reached 2.8×, an ~8% gain, and was still trending upward when the run ended, evidence that the improvement had not plateaued and that the next iteration would likely gain again.
Ai-OPs
·Continued Training · Incremental RL01
Why continued training
A model trained once and frozen captures the plant as it was on the day it was built. A plant, and the people who run it, never stop changing, so the question was whether an RL controller could keep learning the way an operator does.
Operators do not stop improving after their first shift; they accumulate experience and get steadily better at holding the process where it should be. We start the same way: a solid foundational model, trained, validated and deployed as a fixed starting point, but reinforcement learning lets us keep improving it from there rather than freezing it at day one.
The hypothesis
- Incremental beats one‑shot: retraining on accumulated operating experience should raise performance beyond the first model, not just maintain it.
- Each iteration builds on the last: a new model starts from the prior one and the experience it gathered, so gains compound rather than reset.
- The curve keeps climbing: if performance is still trending up at the end of an iteration, there is more to be had from the next, just as an experienced operator keeps refining.
Continued training is the model's version of an operator's career: deploy, run, accumulate experience, then come back sharper. The point of this study was to show that loop produces measurable, repeatable gains.
FIG. 1: FOUNDATION, THEN IMPROVEMENT. We deploy a trained, static foundational model as the starting point, then use reinforcement learning to incrementally improve it over time, rather than leaving it frozen at day one.
Ai-OPs
·Continued Training · Incremental RL02
The proving ground
The test loop was deliberately simple: hold the level of a tank fed from several sources while keeping the downstream flow it supplies as steady as possible. Simple to state, hard to do well, and easy to measure.
Several feeds arrive at the tank at once, each with its own variability. The loop holds tank level, but the real prize sits downstream: whatever the tank supplies needs a smooth, stable flow, and every feed disturbance threatens it.
Why it is a good proving ground
- Unforgiving objective: downstream‑flow stability punishes overcorrection.
- Strong baseline: a well‑tuned PID is a fair, real benchmark, not a strawman.
- Clear scoring: performance is a single number, so progress is unambiguous.
FIG. 2: TEST LOOP (P&ID). Several feed flows (FIC-11–14, valves FV-11–14) charge the tank. The level controller (LIC-1) and the Ai‑OPs agent (AIOP-1) drive the outlet valve (LV-1) to hold tank level while keeping the downstream flow steady, the objective the controller is scored on.
The PID was genuinely well tuned, not a handicap. Starting from a strong baseline is what makes a 2.6×, then 2.8×, result meaningful.
Ai-OPs
·Continued Training · Incremental RL03
How continued training works
Continued training is a sequence of discrete, validated model builds, not online self‑learning. A fixed pre‑trained model runs the loop; after a period its running data is exported and the model is retrained offline, starting from the last best version; once validated, the new fixed model is deployed, one clean change at a time.
FIG. 3: THE TRAINING TIMELINE. An initial RL model is trained on current process data and deployed; over time its running data is exported and the model is retrained offline from the foundational version, validated, saved, and re‑deployed on Koios in place of the prior model. The first two steps happen once; the rest repeat over time, each a discrete, validated change that fits a standard Management‑of‑Change process.
Built for Management of Change
Because retraining happens offline and each new model is validated before deployment, every improvement reaches the plant as a single, deliberate change, exactly the clear engineering stage gate an MOC process expects, never an opaque model editing itself in production.
Why incremental, not from scratch
A fresh model would throw away everything the deployed one learned. Building incrementally on the prior model means each iteration inherits its skill and adds to it, the same way experience compounds for an operator rather than restarting every shift.
Ai-OPs
·Continued Training · Incremental RL04
Results
Periodic re‑validation and incremental training improved performance over time. Each model build climbed above the well‑tuned PID, and the second iteration extended the lead, while still trending upward.
FIG. 4: PERFORMANCE OVER TIME. Recreated from the study. A higher reward means the loop fulfills its goal better. The 1st model rose to 2.6× the PID baseline; the 2nd, incrementally retrained, reached 2.8× (~8% higher) and was still climbing at the end of the run, the improvement curve had not flattened.
Reward is simply the performance score the agent is graded on each step, higher reward means the loop met its goal better. It is how well the controller "plays the game"; the function itself is project‑specific and not disclosed here.
The gain from the 2nd build, and the fact that it was still trending up, is the proof: continued training keeps paying off, iteration after iteration.
Ai-OPs
·Continued Training · Incremental RL05
The optimization ceiling
Continued training does not improve forever. Each build adds a little less than the one before, until reward levels off at the most the system can physically deliver, the point where the model has reached its potential.
FIG. 5: THE OPTIMIZATION CEILING. The first two builds are this study's measured results; beyond them, theory says each incremental build adds less than the last as reward approaches the ceiling set by the system's fixed physical constraints. Without reshaping the reward or changing the equipment, that ceiling is the model's true potential.
Why the gains shrink
A control loop can only do so much within its valves, vessels and flows. As the model nears that hard limit there is less headroom left to capture, so each retraining build returns a smaller increment, classic diminishing returns.
Why the plateau is the goal
Reaching the ceiling is the win, not a failure: it means the controller is extracting the full capability of the system as built. Going higher would require reshaping the objective or changing the physical process, not more training.
Ai-OPs
·Continued Training · Incremental RL06
Where this applies
Continued training is not tied to one process, it is a property of how Koios models are deployed and maintained. Any closed‑loop RL application can be enrolled in the same deploy‑observe‑revalidate‑retrain cycle:
About Koios
Koios is Ai‑OPs' self‑hosted machine‑learning inferencing platform for deploying scalable AI models on industrial control systems. It is hardware‑agnostic, runs fully on‑premises and air‑gapped, and writes optimized control back to PLCs, SCADA and smart instruments in real time. Your models and data stay inside your plant; you own every model Koios trains, and every model it retrains.
It installs as a single Docker image from a Docker Verified Publisher, or as an offline ISO / virtual‑appliance image, and is licensed as an annual per‑instance subscription.
| Approach | Continued / incremental RL |
| Protocols | EtherNet/IP · OPC‑UA · Modbus TCP & more |
| Deployment | On‑prem · air‑gapped |
| Install | Single Docker image / ISO |
| Per license | Unlimited devices, tags & models |