In Machine Learning, Good Data Beats a Smart Model

To put it simply, a machine learning model is a student that learns from examples, and the data is its textbook. If the textbook is full of mistakes, missing pages, and wrong answers, even the brightest student will learn the wrong things — while an average student with a clean, honest textbook often does surprisingly well.

I used to think the model was the whole game. Bigger model, more layers, more parameters, better results. That is the part everyone talks about. It took a few painful experiments to learn that the model is often the least interesting part of the problem. The data decides what the model believes about the world, and the model just repeats it back at speed.

Think of it like teaching someone to cook from a recipe book

Imagine you hand a keen young cook a recipe book and tell him to learn from it. He is smart. He reads fast. He remembers everything.

Now imagine the book has problems. The salt and sugar amounts are swapped in half the recipes. A few pages are stuck together, so whole dishes are missing. One chapter keeps repeating the same soup, over and over, so he thinks soup is most of what cooking is. And some recipes were copied down wrong by whoever wrote the notes — "bake for 3 hours" when it should have been 30 minutes.

Your cook will not become bad because he is stupid. He will become bad in very specific ways that match the book exactly. He will over-salt. He will burn things. He will serve soup for dessert and not understand why you are upset.

That is training data. Every flaw in it becomes a habit in the model.

Wrong labels teach wrong answers, confidently

A label is the answer you attach to an example. This photo is a cat. This email is spam. This customer did leave, this one stayed.

Models trust labels completely. They have no way to say "are you sure?" If you feed a system ten thousand photos where some dogs are labelled as cats — because a tired person rushed through the labelling, or two people disagreed about blurry pictures — the model learns a strange, fuzzy idea of what a cat is. Then it gets your test photos wrong, and it gets them wrong with total confidence.

This is why careful labelling is slow, boring, and worth every minute. For the tricky examples, the blurry photo, the email that could be spam or could be a real receipt, you want a human to stop and think. One hour spent fixing labels at the start can save you a week of tuning a model that was never the problem.

Here is a small test I like. Take a hundred random examples from your data and check them by hand, yourself. Not a sample the computer picked nicely — just grab them and look. If you find mistakes in that hundred, they are in the thousands too. They are always in the thousands too.

Balance matters more than size

People love to say "we have a million rows of data" like the number settles everything. It does not. A million rows can still teach a lopsided lesson.

Say you are building something to spot a rare problem — faulty products on a factory line, let's say. Faulty items might be one in every thousand. If your data mirrors that, a model can score 99.9% by simply saying "fine" every single time. Perfect score. Completely useless. It never learned what a fault looks like because, in its world, faults basically do not exist.

So you balance things. You make sure the rare case shows up often enough during training for the model to study it properly.

• Show the rare examples enough times that the model can actually learn their shape
• Keep enough ordinary examples that it still knows what normal looks like
• Test it separately on the rare cases, not just on the overall score
• Watch what it misses, not only what it gets right

Balance is not cheating. It is making sure the student studies every chapter, not just the thick ones.

Cleaning is unglamorous and it is most of the job

Real data is dirty. That is not an insult to anyone, it is just what happens when information comes from real people, real forms, and real systems that were never designed to agree with each other.

You get the same person twice, spelled two ways. You get a date written as 03/04/2026 and nobody knows if that is March 4 or April 3. You get "N/A", "-", "unknown", and an empty box all meaning the same thing in the same column. You get ages of 0 and ages of 240.

A model does not know any of this is strange. It will happily learn that some humans are 240 years old.

Cleaning is where you fix that, patiently:

• Remove exact duplicates, so one example does not shout louder than the rest
• Decide once how missing values are handled, and do it the same way everywhere
• Standardise the simple things — dates, capital letters, units like kg and lb
• Throw out the impossible values, or go back and check them against the source
• Write down what you cleaned and why, so the next person is not guessing

None of this makes a good demo. Nobody claps for a clean spreadsheet. But this is the work that decides whether the finished system is trustworthy or just impressive for the first five minutes.

A smarter model cannot rescue bad lessons

Here is the part that surprises people. Take your messy dataset and train a much bigger, more expensive model on it. What do you get?

Usually, a model that learns the mess faster and repeats it more confidently. It memorises the wrong labels better. It finds cleverer ways to lean on the lopsided parts. You have not fixed the teaching, you have just hired a faster student for the same bad book.

The other direction works far better than people expect. Take a modest, well-understood model and feed it clean, balanced, honestly labelled data. It is cheaper to run, easier to explain, and it keeps doing the right thing on new examples because its lessons were true in the first place.

If your results are bad, your first move should not be "find a better model." Your first move should be "go and look at fifty of my examples with my own eyes." I mean literally look at them. The problem is usually sitting there, visible, a bit embarrassing, and fixable.

Data work is slow and nobody posts about it. Do it anyway. It is where the quality is hiding.

References and further reading

• Data quality — Wikipedia
• Data cleansing — Wikipedia
• Garbage in, garbage out — Wikipedia
• Machine learning — Wikipedia

See you in the next post.

— Min SiThu Software Engineer & Tech Content Creator https://minsithu.org