Reinforcement‑learning fill control for dry‑powder packing.
Two Deep Q‑Network models work together: one sets a floating box‑weight target that rides just above the legal minimum, the other sets the flask head height to hit it, anticipating the downstream checkweigher and compensating for the density segregation that comes off the day bins. Less giveaway, fewer under‑ and overshoots, full legal‑weight compliance, running closed‑loop on the existing Allen‑Bradley PLC over native EtherNet/IP.
A dry‑powder consumer‑goods line fills boxes on a rotary filler with multiple flasks fed from a day bin. A servo sets each flask's head height, which sets the dose, but powder density drifts as the bins segregate, and the only weight feedback comes from a checkweigher several boxes downstream. The legacy regulation is always correcting late: it overshoots, then overfills to stay safely above the legal minimum, paying for that safety margin in product giveaway.
Ai‑OPs deployed two Koios Deep Q‑Network (DQN) models. One learns a floating weight target that sits just above the legal limit; the other sets the flask head height to hit that target, using a density signal and anticipating the checkweigher lag instead of reacting to it. In validation it cut box‑weight error by up to 34% and reduced overpack giveaway, running closed‑loop on the existing Allen‑Bradley PLC.
Ai-OPs
·Dry-Powder Packing DRL Control01
Background & objective
The objective was to reduce the under‑ and overshoot on fill weight, holding boxes to legal‑weight and over‑pack requirements while cutting the product giveaway the old control padded on to stay safe.
On the line, product moves from feed bins into a day bin, then meters into boxes through a rotary filler with several flask filling stations. A servo sets the vertical head height of the flasks, which sets the dose per box. A downstream checkweigher measures every filled box and feeds back a running average of the last several boxes, the only weight signal the loop has.
The control problem
- Density segregation: powder density drifts as the day bins segregate, so the same flask height yields a different box weight from one moment to the next.
- Downstream lag: the checkweigher sits several boxes downstream of the filler, so every correction is applied to a condition that has already moved on.
- Overshoot & hunting: the legacy regulation chases that lagged average, overshooting and oscillating around the target instead of settling on it.
- Giveaway: to stay safely above the legal minimum, the line overfills, consistently exceeding the declared weight and burning product margin every box.
Under the average‑quantity rules (76/211/EEC), the mean fill must clear the nominal weight and almost no pack may fall below the legal minimum. The win is to satisfy that with the thinnest safe margin, not a generous one.
FIG. 1: FILL LINE & LAG. The servo sets flask head height at the filler, but the only weight feedback arrives from a checkweigher several boxes downstream. A reactive loop is always correcting a fill that has already passed.
Ai-OPs
·Dry-Powder Packing DRL Control02
Closed-loop control architecture
Two Deep Q‑Network models run in tandem. The first sets a floating weight target that rides just above the legal floor; the second sets the flask head height to hit that target, anticipating the checkweigher rather than chasing it.
FIG. 2: CONTROL & CONNECTIVITY ARCHITECTURE. Model A writes a floating weight target; Model B drives the flask head‑position target to meet it, with bumpless fallback to the existing regulation. Koios reaches the loop directly over native EtherNet/IP, across both controller‑ and program‑scoped tags.
What the models watch
The checkweigher's running average, the weight error from the target, day‑bin weight, head height, and a new density signal that is the key to handling segregation. The models inference once per second on the prior 10 samples of runtime.
What they control
Model A outputs the floating weight target; Model B outputs the head‑position target that the servo drives through the existing rung logic. Because Model B already knows density and the current dose, it pre‑empts the next box instead of reacting after the scale.
Ai-OPs
·Dry-Powder Packing DRL Control2.3
Digital twin & control model
Anticipating the scale
A recurrent‑neural‑network digital twin forecasts the weight error in grams from head height, the floating target, day‑bin weight and the new density feature. Predicting the error signal, rather than the raw weight, sidestepped normalization issues and let the controller act on where weight is heading before the checkweigher confirms it.
The density feature is what makes the twin robust to segregation: the same head height no longer implies the same weight, so the model learns the dose‑to‑weight relationship as density moves.
Control objective
The control model is trained to hit the floating target precisely, favoring on‑target box weight while never crossing the legal floor, so it trims giveaway without ever risking an underfill. Precise, stable weight tracking is the priority the policy optimizes toward.
| Output | Head‑position target |
| Output type | Positional |
| Max step | Bounded per move |
| Scan rate | 1 s · train & execute |
| History depth | 10 samples |
| Head height | positional |
| Floating target | grams |
| Weight error | ± grams |
| Density | derived signal |
| Filter / vibrator state | 0 / 1 |
TABLE 1: MODEL CONFIGURATION. The twin predicts weight error; the control model outputs a head‑position target, normalized and clamped to its valid range.
03
Implementation & integration
Minimal PLC edits
Integration adds a small set of tags, a new density calculation and a weight‑error signal as inputs, plus the model's target, heartbeat and status as outputs. All other signals are existing tags, and the servo drives through the existing rung logic, no new hardware.
Train, validate, deploy
Models are trained and validated offline against historical production, then run in monitoring before promotion to closed‑loop control, with a clean handoff back to the line's native regulation at any time.
A heartbeat watchdog blocks control if the model signal goes stale, a model‑status enumeration gates whether control is permitted, and bumpless transfer hands the head position back to the existing regulation at its current state on any fault or operator command.
Ai-OPs
·Dry-Powder Packing DRL Control04
Results
Across validation the controller held box weight in a tight band hugging a floating target that rides just above the legal floor, cutting both the swing and the giveaway while staying fully compliant.
FIG. 3: CONTROL VALIDATION. Recreated from the validation sets. The old box weight (purple) swings wide, overshooting high and dipping toward the limit (circled), and averages well above nominal, paid in giveaway. The RL weight (orange) holds a tight band on the floating target just above the floor, an estimated 34% reduction in box‑weight error with less overpack.
"By using DRL for head‑position targeting, the range of on‑target box weights improved by as much as 34%, with reduced overpack while holding the legal‑weight requirements."
Ai‑OPs Control Strategy & Model Validation, dry‑powder packing line
Ai-OPs
·Dry-Powder Packing DRL Control05
Where this applies
The same pattern, reinforcement‑learning agents that set a compliant target and pre‑empt a downstream measurement, closed‑loop on the existing control system, generalizes across packaging and consumer‑goods manufacturing. Applications available today on Koios:
About Koios
Koios is Ai‑OPs' self‑hosted machine‑learning inferencing platform for deploying scalable AI models on industrial control systems. It is hardware‑agnostic, runs fully on‑premises and air‑gapped, and writes optimized control back to PLCs, SCADA and smart instruments in real time. Your models and data stay inside your plant; you own every model Koios trains.
It installs as a single Docker image from a Docker Verified Publisher, or as an offline ISO / virtual‑appliance image, and is licensed as an annual per‑instance subscription.
| Protocols | EtherNet/IP · OPC‑UA · Modbus TCP · BACnet & more |
| Deployment | On‑prem · air‑gapped |
| Install | Single Docker image / ISO |
| Min. hardware | 4‑core 64‑bit · 8 GB RAM · 128 GB SSD |
| Per license | Unlimited devices, tags & models |