The Gemma 4 Good Hackathon

The Gemma 4 Good Hackathon · Impact Track · Digital Equity & Inclusivity prize

Gem-Care

Built by Bilal Bounajma

A Gemma 4 speech-transcription fine-tune using structured clinical context. The repository records 19.6% word error rate under its evaluation method; the claimed baseline improvement and writeup’s 19.0% were not independently confirmed.

I also participated in this hackathon with Memory Moment. This coverage was prepared independently after results were announced.

The idea worth stealing

Clinical context only helps once the model is taught to use it.

Human listeners lean on what they know about a speaker’s condition, so the obvious move is to put that in the prompt. The builder reports that base Gemma 4 gets worse with clinical context, drifting from transcription into clinically flavoured text, and the longer the context the more it hallucinates. The fix was to make the context a dense severity profile, each dimension rated 1 to 7, and to LoRA-tune on audio paired with it, with a claimed improvement of more than a third on unseen speakers; the baseline runs are not available to confirm that comparison.

What we checked

Date checked: .

Public repository at the submission revision, Kaggle writeup, and linked artifacts; code was read, not executed. Deployment checks and limits are described below.

Labels apply to each finding and the evidence described, not to the project as a whole.

Not independently verified means the available evidence was insufficient to confirm a claim. It does not mean the claim is false.

  1. The adapter and its training curve are in the repo

    Verified

    A 12.5 MB LoRA on the language model’s Q, K, and V projections, 3.1 million of 5.1 billion parameters, with history.json recording test WER falling from 28.4 to 19.6 percent over six epochs and 8.5 hours. Source

  2. The baselines are not

    Not independently verified

    The 32.7 and 39.5 percent base figures appear only in the README and writeup; no result files for the audio-only or context-prompted base runs are committed. Source

    What we need: Per-sample outputs and scoring configs for both base-model runs on the same test split.

  3. Unseen speakers, lenient WER

    Partially verified

    Splits are speaker-disjoint, so 19.6 percent is on people the model never heard. But each sample’s WER is the minimum against the raw and the disfluency-cleaned transcript, and the test split was watched every epoch. Source

    What we need: A fresh speaker-disjoint evaluation with a fixed transcript policy and a test set unused during training.

Under the hood

Twenty-three rated dimensions
The prompt builder carries 13 tier-1 dimensions such as imprecise consonants and 10 tier-2 voice-quality dimensions, each rated 1 to 7; naturalness and intelligibility are excluded by default.
Loss only on the transcript
The collator tokenizes the prompt-only conversation to measure its length, then masks every prompt token to −100, so the clinical context is read but never learned as output.

Nuance

The writeup’s 19.0 percent is 19.6 in the repository’s own history; the German BF-SPRACHE set, per-word confidence scores, and the multi-pass suggestions are described but not in the code. Training data is the Speech Accessibility Project, which cannot be redistributed, so nothing here can be rerun. The demo video is withheld to protect a minor, which we respect.

Sources

Last updated: . Editorial updates do not imply a new technical check.

Builder credits reviewed: , using the official announcement, submission, and any linked credit sources.

Builder? Add context, request a correction, or ask for a re-check →