A predictive model can perform well on historical data and still be difficult to use in day-to-day work. Its inputs may arrive too late. The outcome it predicts may differ from the one the business cares about. Staff may receive its warning too late to do anything about it.
Taking the model into production means checking those conditions and building a system that handles them. Late-delivery prediction is a useful example: the system needs to warn someone early enough to act, then wait to find out what happened. The examples below are hypothetical, rather than results from a particular deployment.
Define what counts as late
An order can have several promised dates: the original customer promise, the supplier’s commitment, a revised delivery date and an internal service deadline. Choose which promise the model should predict a failure to meet. Each choice answers a different question.
If a planner moves the promised date after a delay becomes apparent, an order may count as on time against the revised date while still missing the original promise. Training against the revised date can therefore hide the delays you wanted to predict. It does not necessarily mean the model learns whether someone revised a date; that depends on the inputs and how you build the target.
Also decide how much delay counts and when the system will make its prediction. Predicting any missed deadline is different from predicting a delay beyond an agreed tolerance. Predicting at order entry is different from predicting shortly before delivery, when more information is available.
Write these choices down with the people responsible for the delivery process. Keep the date used to judge the outcome separate from later changes to the order, unless predicting performance against a revised promise is explicitly the goal.
Use inputs that were available at the time
The inputs a model uses are called features. For a delivery model, these might include the carrier, the current shipment status, an estimated arrival date and the supplier’s previous delivery record.
A historical extract can contain information that was added after the point when the model would have made its prediction. An order table might hold the final delivery status or an arrival estimate corrected after a problem became obvious. A supplier’s on-time rate for an entire month can include deliveries that had not happened when an earlier order was scored.
Using that information is data leakage. The model gets clues in its test that it will not have when someone actually needs a warning. Holding out rows from training does not fix this if those rows contain the same future information.
Build each historical example from the values available at its prediction time. This is point-in-time correctness. History tables, recorded changes and event logs can help reconstruct those values. A feature store can also help, but only if its timestamps and retrieval rules reflect when the values became available.
For example, suppose a carrier scans a parcel in the morning, but the scan reaches your system after the model makes its prediction. The historical example must not include it, even though the scan had already happened. The time of the event and the time your system received it are different.
Check how the tool you use handles late arrivals and corrections. Feast’s documentation, for example, explains that its default point-in-time join constrains the event timestamp but can still return values corrected later. It offers an additional created-timestamp filter for supported stores, provided that timestamp records when the value became available.
If you cannot reconstruct an input reliably, leave it out of the historical test or start recording it prospectively. Compare the features used in live predictions with those produced by the training pipeline. For a system expected to predict future orders, also test on later orders rather than relying only on a random split.
Wait until the outcome is knowable
A label is the recorded outcome used to train or evaluate the model. In this example, it says whether the order missed the chosen deadline.
An open order without a late flag is not automatically an on-time delivery. Before its deadline, the result may still be unknown. After the deadline passes without delivery, it may already count as late, even though the shipment remains open. The point when the label becomes knowable depends on the target definition and how quickly records arrive.
Keep unresolved cases separate from confirmed outcomes. Define how you handle cancellations, missing delivery records and orders still open after the reporting window. Otherwise, the newest data can look unusually successful simply because many failures have not yet been recorded.
For a fixed-window target, group predictions by the date they were made and evaluate groups whose outcome window and reporting allowance have elapsed. These groups are often called cohorts. This prevents older completed orders from being mixed with recent unresolved ones.
If the target is time until delivery, an undelivered order at the end of observation is a censored record: you know it has not arrived yet, but not how long delivery will ultimately take. Survival methods can represent that incomplete observation. Treating it as a completed on-time case cannot.
The same delay limits experiments. You need enough resolved outcomes to judge a change. Input checks and other early measures can help you find problems sooner, but they do not establish whether the new model prevents late deliveries.
Record what people do after a warning
Suppose the model flags an order and a planner expedites it. The order arrives on time. From the final delivery record alone, you cannot tell whether the warning was wrong or the planner prevented the delay.
This is a feedback effect: the prediction prompts an action that changes the outcome. Retraining on those outcomes without recording the actions can change what the next model learns. It may learn that orders like the successfully expedited ones usually arrive on time and give them lower scores. That is a risk, not an inevitable result of every retraining run.
A related problem, selective labels, arises when a decision determines which outcomes you get to observe at all. For example, a rejected application may never produce the repayment record that an approved application would. Changing an observed delivery outcome and hiding an outcome are different problems, even though both complicate evaluation.
Record the score, whether staff saw it, the action they took and the rule or threshold in use. If actions are randomized, record their probabilities at the time too. These records help distinguish model performance from the effects of the intervention policy.
Where it is safe and acceptable, a randomized comparison can estimate whether model-assisted work improves delivery outcomes. A control group might continue with the existing process while the other group receives model warnings. The comparison needs to account for other interventions and effects that spill between groups. Withholding a warning does not automatically create a group receiving no help.
Comparing periods with the same intervention policy can also make results easier to interpret. It does not, by itself, reveal what would have happened without intervention. Logging action probabilities can support methods that estimate outcomes under a different policy, but those methods still need assumptions and adequate coverage of the actions being compared.
Fit the prediction to the work
Start with the person who will receive the warning and the action they can take. If expediting requires three days’ notice, a warning one day before delivery is too late for that action. Another action, such as notifying the customer, may still be possible.
Choose the point when the system scores an order around that workflow. Then determine which features are available at that point. A model that relies on information from dispatch cannot support a decision made at order entry.
The team also needs a manageable queue. In a hypothetical operation that can review fifty orders a day, five hundred warnings would force staff to choose which ones to ignore. Assess the model at the volume the team can handle, along with the costs of unnecessary action and missed delays.
Precision measures how many flagged orders meet the target. Recall measures how many of all the late orders the model flags. Both matter, but neither alone tells you whether staff can act effectively. AUC, a measure of how well the model ranks positive cases above negative ones across thresholds, also does not tell you which cut-off fits the team’s workload.
Check calibration if you use the scores as probabilities. A calibrated score should match the observed frequency of the outcome among comparable predictions. Ranking orders well does not automatically make the probabilities reliable. Interpret calibration in light of the actions taken after scoring.
Define what receives a score: an order line, a shipment or a customer. If several lines travel in one shipment, decide how their scores become one decision. Put the warning in the queue or planning screen where someone can use it, rather than leaving it in a separate dashboard.
Staff may need relevant facts alongside the score, such as a missed dispatch milestone. A model explanation can show which inputs influenced the prediction. It does not prove what caused the delay or establish which intervention will work.
Monitor inputs now and outcomes later
Some checks can run as soon as a prediction is made. Watch for missing or stale inputs, unexpected changes in shipment mix, shifts in scores, unusual volumes and slow or failed requests. These checks can reveal a broken feed or a changed operating environment before delivery outcomes arrive.
Once outcomes are ready, evaluate precision, recall and calibration for the relevant prediction cohorts. Break results down where useful, such as by carrier or route, while checking that each group has enough observations. Compare equivalent seasonal periods so that a peak-season change is not automatically blamed on the model.
Save the prediction time, input values, model version, threshold and whether the warning reached staff. Use a stable identifier to connect that record to the eventual outcome and any action taken.
Do not make confirmed outcome deterioration a prerequisite for every response. If an input feed breaks, fix it or use a fallback immediately. If the business changes its delivery process, review whether the target and scoring policy still fit. Retraining is one possible response to a diagnosed problem, alongside data fixes, policy changes and rollback.
Make failures recoverable
Before release, the team should be able to:
- Reproduce a model from its training data, code, settings and feature definitions.
- Trace a prediction to the inputs and model version that produced it.
- Meet the workflow’s response-time requirement and handle missing or stale data explicitly.
- Fall back to an agreed rule, manual process or previous model when scoring fails.
- Validate a proposed replacement on appropriate resolved data before deploying it.
- Name who investigates alerts, approves changes and restores service.
Test those paths as well as the prediction itself. A useful release test follows an order through scoring, the staff workflow, any intervention and the eventual outcome record. It should show that the team can use the warning and later evaluate what happened.
Sources
- Google’s Rules of Machine Learning: testing infrastructure, logging live features and measuring training-serving differences
- Feast documentation: point-in-time joins and filtering values by availability time
- Scikit-learn: common pitfalls, including data leakage
- Scikit-learn: probability calibration
- Performative Prediction: how decisions based on predictions can change the outcomes being predicted