Reinforcement‑learning steam optimization for distillation column profitability.
A deep‑RL agent that runs the reboiler steam balance of an ethylbenzene / styrene column as a profit problem, not a setpoint problem. It optimizes the column's economics in dollars per hour, balancing steam cost against product value, and it was trained safely against a multi‑model ensemble of digital twins. It runs closed‑loop on the existing Honeywell DCS.
A binary distillation column separates ethylbenzene from styrene, and reboiler steam is its dominant energy cost. Operators ran the column by hand and conservatively, well away from the economic optimum: steam was burned to over‑purify, styrene carried overhead had to be reprocessed, and ethylbenzene was under‑recovered in the bottoms, all while the quality lab numbers that should have guided the trade‑off arrived slowly and infrequently.
Ai‑OPs deployed a Koios deep‑reinforcement‑learning agent that optimizes the column's economics in dollars per hour, weighing product value recovered against steam and reprocessing cost. Rather than risk the plant, the agent learned its policy inside a virtual column built from an ensemble of seven digital‑twin models. Every validation run improved profit, for an estimated annual benefit of $83K to $169K, with the largest gains still ahead once reflux is automated alongside steam.
Ai-OPs
·EB / Styrene Steam Optimization01
Background & objective
The objective was an economic one: hold both product specifications while spending the least possible reboiler steam, so the column runs at its profit optimum instead of a conservative, hand‑tuned margin away from it.
The column makes a clean split between two close‑boiling products. Reboiler steam drives the separation; more steam buys more purity but costs energy. Two quality limits bound the operation: overhead styrene must stay at or below spec, and bottoms ethylbenzene must stay at or below its limit. Between those limits lies a continuous trade‑off between energy spent and product value recovered, and that trade‑off moves with feed and column conditions.
Why manual operation left money on the table
- Steam inefficiency: operators ran conservatively, away from the optimum, burning steam to over‑purify when the economics did not justify it.
- Reprocessing losses: styrene carried overhead has to be re‑separated downstream, an energy and yield penalty every time.
- Under‑recovery: ethylbenzene lost into the bottoms is product value left behind.
- Slow feedback: the lab numbers that govern the trade‑off were measured infrequently and lagged the process, so corrections came late.
Holding spec is easy if you simply burn more steam. The win is the opposite: blend purity and energy continuously so the column sits exactly on the economic optimum, which is a profit calculation made every scan, not a fixed setpoint.
FIG. 1: THE ECONOMIC LEVERS. Reboiler steam is the single manipulated lever between two quality constraints. Spend too little and bottoms purity slips; spend too much and energy cost outruns the value of the extra separation. Profit lives in the narrow band between.
Ai-OPs
·EB / Styrene Steam Optimization02
A multi-model environment to train the agent
Reinforcement learning needs to explore, and you cannot let it explore on a live column. So the agent learned inside a virtual column: an ensemble of seven data‑driven digital twins that reproduce the column's multivariable dynamics, with no plant risk.
FIG. 2: CLOSED TRAINING LOOP. The agent acts, the ensemble predicts the column's response, the economic score rates that state in dollars per hour, and the policy updates. Training runs in simulation until convergence, then shadow‑mode validation precedes any live control.
Why an ensemble, not one model
Robustness and generalization. Each twin is trained under slightly different data distributions and process phases, so the agent sees a range of plausible columns and learns a policy that holds up across operating states and disturbances, not one memorized response.
Safe, efficient learning. The policy is learned offline against the twins, so there are no exploratory control moves on the real process. Once it converges it is proven in shadow mode and only then promoted to closed‑loop.
FIG. 3: ENSEMBLE FIDELITY. Mean absolute error of each twin against held‑out history. Flows predict to within 1 to 3 percent; the slower, infrequently sampled lab compositions sit near 5 percent. Average accuracy across the ensemble is about 2.7% MAE.
Ai-OPs
·EB / Styrene Steam Optimization03
Closed-loop control architecture
In production the trained policy reads the live column, computes the economically optimal reboiler steam target every 15 minutes, and writes it as a cascade setpoint, with the native steam controller and PID fallback always underneath.
FIG. 4: CONTROL & CONNECTIVITY ARCHITECTURE. A steam target cascades to the existing controller with PID fallback underneath. Koios reaches the field through one of two paths: the Honeywell DCS over OPC, or a native EtherNet/IP Object.
What the agent watches
Reflux and distillate flow, overhead styrene and bottoms ethylbenzene composition, and bottoms product flow. Koios manages the historian data, data quality and data‑range tolerances, then the policy inferences on a recent runtime window so it acts on a trend, not a single noisy sample.
What it controls
One value: the reboiler steam target, written as a cascade setpoint to the existing steam controller. The policy acts only inside trained valid ranges; outside them, or on operator command, the loop reverts to PID with a bumpless handoff.
Ai-OPs
·EB / Styrene Steam Optimization04
Optimized for the plant's economics
The agent is not tuned to a setpoint, it is optimized for profit. Every scan it weighs the net economic effect of its action in dollars per hour, mirroring the site's advanced‑process‑control financial objective, so the policy learns the same profit‑seeking behavior a perfectly tuned optimizer would target.
The behavior this produces
Because over‑purifying the overhead costs more in steam than the reprocessing it avoids, the rational policy is clear and the agent learns it: hold bottoms ethylbenzene at or just under its limit, run steam as low as that allows, and accept small overhead‑styrene increases rather than burn disproportionate energy chasing deeper separation. When bottoms purity is threatened it spends steam without hesitation, because protecting product value dominates the energy cost. The result is a calm, economically rational operating policy that adapts continuously to feed and column conditions.
A fixed setpoint is right at one operating point and wrong everywhere else. Optimizing for economics re‑solves the trade‑off every scan, so the column tracks its true profit optimum as conditions move.
Ai-OPs
·EB / Styrene Steam Optimization05
Validation & annual return
Across four held‑out validation scenarios the policy behaved exactly as its economics predict: energy‑lean when quality had margin, decisive with steam when a penalty threatened.
| Set | What the agent did | Δ Profit | Quality |
|---|---|---|---|
| 01 | Held an energy‑lean profile with stable constraints. | +$1.38K | EB rose slightly but stayed under limit; styrene on‑spec. |
| 02 | Raised steam briefly to counter bottoms‑EB pressure, then stabilized. | +$0.04K | EB pulled down toward safe margin; no reprocess penalty. |
| 03 | Constrained steam mildly while keeping EB safely under limit. | +$0.36K | EB held with good margin; styrene controlled. |
| 04 | Added steam early to suppress instability, then held steady. | +$0.48K | EB brought down and controlled; styrene in range. |
FIG. 5: VALIDATION SUMMARY. Four recent shadow‑mode scenarios in a tight, low‑margin region with no large penalties active, so gains are modest but consistent; earlier full‑scale excursions cost tens to hundreds of thousands per year when penalties were avoided.
Annualized return
Scaling the observed per‑scan improvement across a normal operating year, then discounting to a conservative band because these validations were low‑penalty periods, gives the estimate below. These figures reflect steam‑side optimization only.
These gains come from steam alone, with reflux left in manual. Automating reflux jointly with steam is what captures the full value of the policy: steadier separation, fewer overhead excursions, and tighter bottoms control.
Ai-OPs
·EB / Styrene Steam Optimization06
Where this applies
The same pattern, a reinforcement‑learning agent optimized on plant economics and trained safely against a multi‑model ensemble of digital twins, closed‑loop on the existing control system, generalizes across refining and petrochemical operations. Applications available today on Koios:
About Koios
Koios is Ai‑OPs' self‑hosted machine‑learning inferencing platform for deploying scalable AI models on industrial control systems. It is hardware‑agnostic, runs fully on‑premises and air‑gapped, and writes optimized control back to DCS, PLCs and SCADA in real time. Your models and data stay inside your plant; you own every model trained for you.
It installs as a single Docker image from a Docker Verified Publisher, or as an offline ISO / virtual‑appliance image, and is licensed as an annual per‑instance subscription.
| Protocols | OPC‑UA · EtherNet/IP · Modbus TCP · BACnet & more |
| Deployment | On‑prem · air‑gapped |
| Install | Single Docker image / ISO |
| Min. hardware | 4‑core 64‑bit · 8 GB RAM · 128 GB SSD · 1 GbE |
| Per license | Unlimited devices, tags & models · historian & diagnostics · support |