The Gemma 4 Good Hackathon

The Gemma 4 Good Hackathon · Main Track · Third prize

PenguinAgent

Built by And0Sha

A local video-analysis pipeline with SAM 3 penguin tracking, an 18-species classifier, and Gemma 4 report generation. We reviewed code and sample reports without running the tracker; literature grounding has gaps.

I also participated in this hackathon with Memory Moment. This coverage was prepared independently after results were announced.

The idea worth stealing

Give the model the telemetry, not just the frames.

PenguinAgent does the measuring before the reasoning. SAM 3 segments each bird, a Hungarian-algorithm stitch keeps identities across ten-second chunks, and a SigLIP 2 linear probe names the species; only then does Gemma 4 see frames paired with coordinates, area, and velocity. When it wants a chart it writes matplotlib code, and when it wants a citation it queries a FAISS index of PDFs on the same machine, to attach references to reports; the sample citations include gaps described below.

What we checked

Date checked: .

Public repository at the submission revision, Kaggle writeup, and linked artifacts; code was read, not executed. Deployment checks and limits are described below.

Labels apply to each finding and the evidence described, not to the project as a whole.

Not independently verified means the available evidence was insufficient to confirm a claim. It does not mean the claim is false.

  1. Ten-second chunks, stitched by the Hungarian algorithm

    Verified

    Video is downscaled to 720p at one frame per second and cut into 10-second segments; SAM 3 runs on each in its own subprocess, and scipy’s linear_sum_assignment joins identities across the seams. Source

  2. Eighteen species, one linear probe

    Verified

    The dataset folder holds eighteen species, a logistic-regression probe over SigLIP 2 features ships as penguin_classifier.pkl, and it runs only after SigLIP’s zero-shot label contains “penguin.” Source

  3. The model’s chart code runs unsandboxed

    Verified

    generate_custom_chart hands Gemma’s Python to exec() with an empty globals dict, which still grants the full builtins. A comment calls this a restricted namespace; it is not one. Source

Under the hood

The default brain is 26B
config.env sets gemma4:26b through Ollama, and the builder’s workstation has 512 GB of RAM and an RTX 5080. Each report names the model in its methodology section.
Anomaly index by prompt
The report prompt injects 100 general questions and 20 per species, then asks the model to score Uniqueness, Anormality, and Rarity and to compute √(0.3A² + 0.2R² + 0.5U²) itself, in prose.

Nuance

The pipeline is complete and the sample reports in the repository are real outputs, but the Antarctic-camp framing meets a 26B model that wants a serious GPU. The literature index is built from PDFs that are not redistributed, and one sample report cites the prompt’s own species question sheets as references [4] and [5]. No tests exist; we did not run the tracker.

Sources

Last updated: . Editorial updates do not imply a new technical check.

Builder credits reviewed: , using the official announcement, submission, and any linked credit sources.

Kaggle credits And0Sha; the linked GitHub profile says And Sha. A full name was not confirmed, so the submission credit is retained.

Builder? Add context, request a correction, or ask for a re-check →