AI models make mistakes.
The important question is not whether you can write a better prompt.
The important question is what happens after the model answers.
Four projects with public code took the same basic approach. Instead of trusting the model’s output directly, they changed the job so ordinary software could check part of the answer before anything important happened.
The projects came from hackathons, but the pattern is useful far beyond hackathons.
The common idea is simple:
Ask the model for something that normal code can verify.
1. Turn “left or right?” into a number
Gilbeot is a walking assistant for older adults in Korea.
A user can photograph a confusing corner, and a small vision model, Gemma 4 E2B, tries to tell the user which way to go.
That sounds straightforward, but the developers left an important note in the code: small vision-language models can reverse left and right.
In a walking assistant, that is not a harmless wording mistake. A wrong direction can send someone the wrong way.
So Gilbeot does not rely only on the model saying “left” or “right.”
The model also returns two numbers: the horizontal position of the arrow’s tip and the horizontal position of its tail.
Then ordinary code compares those numbers.
If the tip is to the left of the tail, the arrow points left.
If the tip is to the right of the tail, the arrow points right.
If the two numbers are too close together, the code treats the result as uncertain.
Roughly, the logic looks like this:
if (tipX == null || tailX == null) return sentence;
if (Math.abs(tipX - tailX) < 0.05) return sentence;
const dir = tipX < tailX ? "left" : "right";
return replaceWord(sentence, opposite(dir), dir);This does not prove that the model found the correct arrow in the image.
But it does move one part of the problem out of the model.
The model may be unreliable at saying “left” or “right.”
Normal code is very reliable at answering “which number is smaller?”
That is the pattern.
If part of the answer can be turned into something deterministic, let code handle that part.
2. Let the model choose from safe options
Sentinel scans MCP servers for security problems.
It uses GPT to review possible findings.
The model returns structured JSON rather than free-form text.
That makes several checks possible.
The software can reject a review if the model cites lines it was never shown.
It can reject finding IDs that do not belong to the current batch.
It can reject a proposed probe if that probe does not match the tool’s allowed input format.
The probe design is especially useful.
The model is allowed to choose which predefined probe to run first and which input field to target.
But the model does not invent the payload itself.
The actual values come from a small fixed set of placeholder tokens.
So the model still makes a useful decision, but the dangerous part is constrained to options the software already understands.
If the model produces an invalid review, Sentinel can retry.
If nothing valid comes back, the finding stays marked as needing review, or the scan fails, depending on the setting.
The model is not trusted to produce anything it wants.
It is asked to choose among things the host program already knows how to check.
3. Check the action itself, not whether the model “meant well”
AirBridge streams Windows audio to AirPlay speakers.
It also includes an assistant that can call tools.
The safety problem is obvious.
Once an AI system can call tools, a bad tool call can do more than produce a bad sentence.
AirBridge handles this with a local catalog of allowed actions.
Every tool is labeled with rules such as:
- read-only
- reversible
- confirmation required
- forbidden
One example is arbitrary_shell, which is explicitly forbidden.
If a tool is missing from the catalog, the request is refused.
Arguments are also checked.
For example, volume has to stay between 0 and 100.
And when the user has to confirm an action, the confirmation is tied to the exact tool name and exact arguments.
That matters because the system is not trying to decide whether the model’s intention was safe.
It checks the actual action the model is asking the computer to perform.
If the action is not allowed, it does not run.
The refusal is then sent back to the model as a tool result, so the assistant can explain what happened or ask the user for something else.
The useful shift is this:
Do not ask, “Did the model intend to do something safe?”
Ask, “Is this exact tool call allowed?”
The second question is much easier for software to answer.
4. If the correct answer is already known, do not generate it
Project Rosie is a prototype for veterinary oncologists designing personalized mRNA cancer vaccines for dogs.
At one point, Gemma generated a synthesis specification that would be sent to a contract manufacturer.
A few hours later, that approach was removed.
The system switched to a template.
The reason is important.
The document goes to a manufacturer as-is.
If the model invents a catalog number or changes a quality-control threshold, the document can become unusable.
But the values that belong in the document are already known by the pipeline.
Only the patient-specific data changes.
That means there is no real benefit in asking a model to regenerate the parts that must stay exact.
Any checker would simply compare the model’s output with values the software already had.
So the project removed the model from that job.
The template writes the known values directly.
Gemma stays involved where generating language is actually useful, such as producing the clinical report and answering questions about the case.
This is the strongest version of the pattern.
If the software already knows the correct answer, do not ask a model to guess it.
The question to ask before writing the prompt
These four projects look different.
One helps people walk safely.
One scans software for security problems.
One lets an assistant call tools.
One helps prepare cancer-vaccine documents.
But they all make the same design move.
They do not ask the model to be perfectly reliable.
They ask what part of the model’s output can be checked by ordinary code before anything important happens.
A useful question to ask before writing the prompt is:
What can the software verify before this output is used, and what happens if the check fails?
Sometimes the answer is a number.
Sometimes it is an ID.
Sometimes it is a line the model was actually shown.
Sometimes it is a tool name from an approved list.
And sometimes the answer is already known, which means the model should not generate it at all.
None of these techniques make the model correct.
They do something more practical.
They reduce how much damage a wrong answer can cause.