Machine learning is empirical: you try many combinations of data, features, model architectures, and hyperparameters to find what works. Without a system, those experiments live in notebooks and memory, and within a week you cannot say which run produced your best result or how to reproduce it. Experiment tracking records every run so results are comparable and reproducible — it is the lab notebook the empirical process demands.
Log everything that determines the result
A run's outcome is a function of several inputs, and reproducing it means capturing all of them:
- Parameters — hyperparameters and configuration for the run.
- Metrics — accuracy, loss, and whatever you optimise, logged over training.
- Code version — the exact commit that produced the run.
- Data version — which dataset and preprocessing, because the same code on different data is a different experiment.
- Artifacts — the trained model, plots, and outputs.
The one people forget is the data version. A model's result depends on its training data as much as its code, so a run logged without knowing exactly which data it saw is not truly reproducible — you can rerun the code and get a different answer.
Compare runs systematically
The payoff of logging is comparison. A tracking tool lets you line up runs side by side — this learning rate versus that one, these features versus those — and see which changes actually moved your metrics. This turns the empirical search from a vague memory of what you tried into a queryable record you can reason over, which is the difference between learning from your experiments and just running them.
Use a tool, not a spreadsheet
Tools like MLflow, Weights & Biases, or similar exist precisely for this, and they integrate into your training code with a few lines to log parameters, metrics, and artifacts automatically. A spreadsheet you update by hand is better than nothing but decays fast, because manual logging is the first thing dropped under deadline pressure. Automatic tracking in the training loop is what makes it reliable enough to trust.
The test of experiment tracking is whether you can take a result from three months ago and reproduce it exactly. That requires the code version, the data version, the parameters, and the environment all captured together. If any one is missing, the result is a number you cannot stand behind — and in ML, a result you cannot reproduce is barely a result at all.
Experiment tracking records the parameters, metrics, code, data version, and artifacts of every ML run so results are reproducible and comparable rather than lost. Log automatically from your training code with a proper tool, never forget the data version, and use the comparison view to actually learn from the empirical search. It is the discipline that turns a pile of experiments into cumulative, reproducible knowledge.
Independent software engineer in Nairobi specialising in Acumatica customisations, Laravel backends, and tax fiscalisation integrations across East and Southern Africa.