Blog › Machine Learning Explained

A package is about to leave a warehouse. The carrier promised delivery by 5 p.m. tomorrow, and you want a model to flag the package now if it is likely to arrive late. In a spreadsheet of past shipments, there is a column that seems perfect for the job: the actual delivery time. But what is data leakage in machine learning? It is what happens when information a model would not legitimately have for a real prediction slips into training or testing. Here, the spreadsheet gives the model a fact it cannot know at dispatch.
Start by separating the target, which is what you want to predict, from the features, which are the inputs the model gets. For the package task, the target is either late or on time. A package counts as late if its actual delivery time is after the promised deadline. If it arrives exactly at the deadline, it counts as on time.
You need the actual delivery time to label each completed shipment. That is fine. The mistake is keeping that timestamp in the features you give the model. This is an example of target leakage: an input reveals the outcome, or gives away information about it, that would not be available when the prediction is needed. The same column can be valid for creating the target and invalid as an input.
The deciding factor is timing. At dispatch, you can use the promised deadline if it has already been recorded. You can also use the destination or the shipping service selected for the package. An estimated arrival time generated and stored before dispatch may be usable too. The actual arrival time is different: it records an event that has not happened yet.
Imagine a test set with four completed shipments. Each had a deadline of 5 p.m. on its promised delivery day. All the actual delivery times below are on that day, in the same local time as the deadline.
| Package | Promised deadline | Actual delivery time | Correct label |
|---|---|---|---|
| A | 5:00 p.m. | 4:42 p.m. | On time |
| B | 5:00 p.m. | 5:18 p.m. | Late |
| C | 5:00 p.m. | 5:00 p.m. | On time |
| D | 5:00 p.m. | 6:07 p.m. | Late |
A rule that compares the two timestamps gets four out of four right. A model that learns that comparison does not need to notice which routes are slow or whether a shipping service leaves enough time. It is answering the same question the spreadsheet used to create the labels. A decision tree can pick up a cutoff like this when the deadline is fixed. For a closer look at how that kind of split works, see our walkthrough of a decision tree prediction.
None of these four packages has to appear in the training set for the test to be misleading. Keeping test rows separate prevents the model from training on those rows, but it does not prevent every test row from carrying its own answer. At dispatch, the timestamp column would be empty or absent. The comparison cannot run, and filling the blank with a default value does not restore the missing information. The perfect score tells you the rule works after delivery, not how well the model can warn you before it.
Dropping the obvious timestamp is a good start, but a column called minutes past deadline could give away just as much. Suppose the warehouse sets it to zero for on-time shipments and to the number of minutes late for late ones. In our example, packages A and C would have zero, B would have 18, and D would have 67. A model could use those numbers to recover every label without ever seeing a column named actual delivery time.
Other columns can leak information without revealing every answer. A customer complaint filed after a late delivery might be a strong clue, even though many late packages never lead to complaints. And a field called estimated arrival might look safe until you learn that the spreadsheet stores the last estimate, which the carrier may have updated while the package was in transit. For a prediction at dispatch, you need the version that existed at dispatch. The column name alone cannot tell you which version you have.
Write the prediction time down before selecting features: Make one prediction when the package is dispatched. For each input, check when its value first becomes available and whether the source overwrites it later. Keep a record of the dispatch-time values if possible. After delivery, attach the outcome as a label, but keep it out of the inputs. If you cannot recover what a field said at dispatch, leave it out of this version of the model rather than substituting a later value.
Build the test the same way. A useful setup for this task is to train on earlier shipments whose outcomes were already known at the training cutoff, then test on later shipments using only the inputs available at their dispatch times. Compare those predictions with the outcomes once the packages have arrived. A time-based split helps imitate the order of real use, but it will not fix a feature that was calculated from the future. You also need to keep the test set out of any preprocessing that learns from the data, such as filling missing weights with a median. The scikit-learn guide to common pitfalls explains how to fit those steps on training data only.
Try the dispatch check with a paper exercise: give a partner the promised deadline, shipping service, and destination for each shipment, but fold the page to hide the delivery time and label. Have them write down a prediction before you unfold it. Then hand them a field called minutes past deadline and ask them to trace where the number came from. For more exercises in reasoning about model inputs and results, see the IMLC training and practice problems.
Can I use actual delivery times to make training labels?
Yes. Compare them with the promised deadlines to label completed shipments, but do not give the timestamps or information derived from them to a model predicting at dispatch.
Does a separate test set stop data leakage?
Not by itself. Even a shipment the model has never seen can give away its answer if its test inputs include information recorded after dispatch.
Is an estimated delivery time safe to use?
Yes, if you use the estimate that was actually available at prediction time. A later update cannot stand in for the original estimate.
About the International Machine Learning Competition: an online machine learning competition for high school students from around the world. Qualification Round 2026: submissions are open until Sunday, 13 December 2026. How the rounds work