9 checks before you fine-tune an AI model: don't spend weeks training one you can't use

The most expensive part of fine-tuning an AI model isn't the hardware. It's trial and error in the wrong direction. At a GTC Taipei 2026 workshop, NVIDIA laid out the pitfalls it hit itself: eight rounds of experiments wasted on an unsuitable technique, a model too small, an architecture that didn't support the method, and scores cut by more than half after merging. This post turns that into 9 checks owners and technical leads can run before work starts.

Johnny Yang9 min read
9 checks before you fine-tune an AI model: don't spend weeks training one you can't use

"We spent a month training the model. The training numbers kept getting better. Then we shipped it, and it was worse than before."

This happens a lot in AI projects. During training, loss goes down and scores go up, and everyone thinks they're making progress. Then you test it against real conditions and find out it was all wasted. The worst part isn't a few days of GPU costs. It's weeks of the whole team's time.

I'm Johnny Yang, founder of Brightstream Technology in Taiwan. I've led my team through systems implementation for seven years, across more than 252 projects in over 32 industries. On June 2 this year, I attended the full-day hands-on workshop "Adding New Knowledge to Large Language Models" at NVIDIA GTC Taipei 2026. What I found most valuable all day wasn't the steps that worked. It was the pitfalls they hit building this case themselves. They even wrote down how many rounds of experiments they wasted.

My answer: most fine-tuning failures aren't about badly tuned parameters. They come from choosing the wrong method at the start. Before you begin, confirm three things: whether you're measuring the right thing, whether this is a problem training can solve, and whether the model's size and architecture can support the method. Each takes a few dozen minutes. Together they can save you weeks.

This post extends the case study How a small model beat top large models at its own job, turned into a checklist for before you start.

First, learn the order: start with the cheapest

The workshop ranked the ways to make a model better, from cheapest to most expensive:

OrderMethodWhat it fixes
1Write the rules into the promptRules the model can understand and just follow
2Small-scale fine-tuning (LoRA)Answer format, wording, consistency
3Full-parameter fine-tuningWhen small-scale fine-tuning truly lacks capacity
4Preference training (DPO)Which of two reasonable answers is better
5Reinforcement learning (GRPO, etc.)Tasks with clear right and wrong answers that can be scored automatically, and the model must be large
6Continued pretrainingDomain knowledge the model simply doesn't have

There is only one principle: if you have no evidence this layer can't do it, don't move to the next one. Each layer down needs more data, more compute, and carries more risk of failure.

The workshop also stressed this: a knowledge base (RAG) lets the model look things up. It doesn't teach the model rules. If writing the rules into the prompt doesn't teach it, retrieving the rules from a knowledge base and stuffing them into the prompt won't teach it either. For how knowledge bases and fine-tuning split the work, see Should you fine-tune an AI model.

Six layers of methods from prompting to continued pretraining, ordered from lowest to highest cost

Check 1: Does your test measure what you actually need?

The workshop's first test set was domain knowledge Q&A. After fine-tuning, performance on the training data improved, but the test score didn't move at all. The reason: the test measured things the model already knew, while what they actually needed was answer format and judgment. It took three versions to get the test right.

Another example: at one point they set the goal as "all four judgments correct" and pushed it from 16% to 40%. But then they asked: would the organization actually use a system that's fully correct 40% of the time? They realized each of the four judgments was useful on its own. Switching to average accuracy (50% to 77%) showed the real value.

How to check: ask yourself, "If this score improves by 30 points, will the company actually use it?" If the answer is "No, because...", the test needs to be rewritten.

Check 2: Have you tried writing the rules into the prompt?

Before generating any training data, write all the rules into the prompt and measure the score once. In the case, this step alone took it from 50% to 74%. Only "which team to assign it to" barely moved. That told them fine-tuning only needed to fix that one piece.

How to check: if writing the rules into the prompt hits the target, don't train.

Check 3: Are you teaching format, or reasoning?

The workshop ran two experiments trying to teach the model reasoning with small-scale fine-tuning. They tried two different ways of writing the data. Scores on reasoning questions improved by zero both times.

The reason: small-scale LoRA fine-tuning teaches "what a correct answer looks like": wording, format, structure. Research also shows that about 1,000 good examples are enough to teach format (LIMA). To teach reasoning, the industry uses hundreds of thousands of reasoning traces and full-parameter training. That's a completely different scale.

How to check: if the base model sometimes gets it right but is inconsistent, fine-tuning helps. If the base model never gets it right, small-scale fine-tuning probably can't teach it either.

Check 4: Are you using preference training in the right place?

The workshop spent eight rounds of experiments trying to raise classification accuracy with preference training (DPO). All of them failed.

The reason: in classification, the "good answer" and "bad answer" differ by only a few fields. The rest of the text is nearly identical. Research shows that in this situation, DPO pushes down the probability of both answers together (Smaug / DPOP). They later used DPO to adjust answer style instead, where good and bad answers differ by a whole passage of text. That worked.

How to check: look at your comparison data. How many words separate the good and bad answers? If they differ only in a few field values, don't use DPO.

Check 5: Is the model big enough?

Another eight rounds of experiments ran DPO on a 4-billion-parameter model. Every time, it learned the training data well. On unseen questions, zero improvement. They switched to an 8-billion-parameter model and it worked immediately.

The research the workshop cited shows DPO is stable at 7 billion parameters and above, unstable at 2 to 4 billion, and of limited effect below 2 billion (Apple research, NeurIPS 2024).

How to check: if your model is under 7 billion parameters, find evidence of success on a similar task at the same size before you commit. For choosing a model, see How to choose an on-premise AI model.

Check 6: Does the model architecture support this method?

The workshop ran LoRA fine-tuning on NVIDIA's Nemotron Nano 9B. Training loss dropped to 0.01. It looked perfect. Actual scores were 2 to 5 percentage points worse than the original. Four attempts, same result.

The reason: about half of this model uses the Mamba architecture. LoRA adjusts ordinary linear layers and can't reach the parts of the Mamba layers that actually do the reasoning. The workshop explained that as of early 2026, this is still an open research problem.

How to check: confirm the model architecture before you start. For hybrid Mamba models, train a few steps on a tiny amount of data first, then test on unseen questions. If the score doesn't improve, change the method or change the model.

Check 7: Is the teacher model teaching the right things?

Many companies use a large model to generate training data, then use it to teach a small model. The problem: when the teacher is wrong, the student is wrong too, and wrong with confidence.

The workshop recommends manually spot-checking at least 5%. If the error rate is over 5%, fix the teacher's prompt first. Another good practice is "set the answer before writing the question": decide the correct answer first, then have the large model write content to match it. That way the answer is guaranteed correct.

One more detail: if you only give positive examples, the model learns the wrong shortcut. For example, if every mention of "road" is labeled "traffic blocked", the model concludes that any mention of a road gets that label. You have to deliberately add examples like "mentions a road, but the road is fine" before the model learns the real criteria.

Check 8: After merging and converting, are the results still the same?

This pitfall is very technical, but it can kill a project. After the workshop merged the fine-tuned weights into the base model, the score dropped from 40% to 16%. The reason: when saving, the format of one positional parameter in the config file changed. The inference engine couldn't read it and fell back to a wrong default value.

How to check: after any merge or conversion, compare the answers to the same question before and after. If they differ, compare the config files and overwrite with the base model's config file.

Check 9: Run a small version of every expensive step first

Each full training run at the workshop took 23 minutes. A 20-step smoke test took 30 seconds, and it caught three errors that would each have wasted more than an hour. Apply the same principle to every step: generate 10 data samples first and look at them, score 3 to 5 questions first, sample 5 comparison pairs first.

When you use a large model for automatic scoring, always have a person review at least 10 results. If it disagrees with your judgment on more than one or two, fix the scoring criteria first.

A 9-item checklist to run before starting AI fine-tuning

How should an owner use this checklist?

Owners don't need to know what LoRA or DPO is. At each stage, ask your team or vendor three questions:

StageWhat the owner should ask
Before startingWhere's the test set? If this score improves, will we actually use it?
Before trainingHave you tried writing the rules into the prompt? Which piece is falling short?
During trainingDoes this method fit this task and this model? Did you run a small test first?

If they can answer, the money is being spent on a basis. If they can't, you're gambling. For how to look at AI implementation costs, see How much does AI implementation cost.

FAQ

Fine-tuning keeps failing. Do we need better GPUs?

Usually not. The experiments that wasted the most at the NVIDIA workshop all came from choosing the wrong method: preference training for classification, preference training on a model that was too small, LoRA on an architecture that didn't support it. Better GPUs wouldn't have made any of these work.

The training numbers keep improving. Why is it worse in production?

Because the model just memorized the training data and didn't learn to judge. You have to test on questions it hasn't seen, and the test set has to be written before the training data is generated, so you're not only testing what it already practiced.

Is AI automatic scoring reliable?

You can use it, but you have to spot-check it by hand. The workshop saw a scoring model mark correct answers as wrong because the scoring criteria weren't written clearly enough. Look at at least 10. If too many disagree with your own judgment, fix the criteria first.

Do small and mid-sized businesses need to be this rigorous?

Yes, even more so. Big companies can afford trial and error. Small and mid-sized businesses don't have weeks to waste. Most of these checks take only a few dozen minutes. The most important ones are writing the test first and trying the prompt first. A lot of the time the problem is solved there and you don't need to train at all.

Conclusion: the cheapest lesson is one someone else already paid for

The most honest thing about NVIDIA's material is that it writes down its own wasted experiments: eight rounds with the wrong method, eight rounds with a model too small, four rounds on an unsupported architecture. We don't have to pay that money and time again.

If you're evaluating whether to fine-tune a model, or you've been at it for a while without progress, take a look at how we do AI system development, or bring your test set and your current scores to me and we'll figure out which layer you're stuck on.

Found this useful? Pass it on
30 minutes · free

Does this sound like your company?

Spend 30 minutes telling Johnny Yang how you do it now, and you get told where hours are leaking, where the risk is buried, and whether this is worth paying to fix. No pitch, no course to sell — if it is not a fit, that is fine too.

Pick a slot online · with Johnny Yang himself · 252+ projects behind it

Johnny Yang (熊董)
Johnny Yang (熊董)
Founder, Brightstream Technology

Serial founder leading a small team that builds systems into an owner's everyday work.

As a founder
7 yrs
Projects delivered
252+
Industries served
32+
About Johnny