Teaching a Model to Read Dog Barks

PetSpeak records a dog bark and gives you an honest, hedged read on what it might mean - a likely intent, an arousal level, and a plainly-worded interpretation with a confidence attached. This post is about how I built the classifier, and the decision I am proudest of, which was to train a real model, measure it honestly, and then not ship it yet.

The interesting engineering in a machine-learning product is rarely the model architecture. It is the labeling and the calibration, and knowing when something is good enough to put in front of a person.

Interpretation, not translation

The first and most important decision was a product one that constrains all the modeling: PetSpeak does interpretation, not translation. It never claims a bark "says" a sentence. It estimates the most likely behavioural intent from acoustics, attaches an explicit confidence, and always keeps an unknown outcome in reserve so it never claims to be sure when it is not. Every result carries a disclaimer that it is an interpretation and not veterinary advice.

That framing turns a fuzzy, borderline-impossible problem ("translate dog") into a bounded classification problem: audio clip → probability over nine dog intent classes, arranged in tiers.

The tiers matter for graceful failure. If the model cannot commit to a specific class, it can still fall back to the coarser family (social / request / warning / discomfort) or just the arousal band. Telling someone their dog is in pain when it is not is a bad outcome. Saying "high arousal, intent unclear" is at least honest.

Freeze the output, swap the brain

Before writing any model, I locked a contract: the classifier's only job is to emit a probability per class id. Everything downstream - deriving tone, urgency and stress, choosing the English phrasing, appending the disclaimer - reads that one probability vector and nothing else. It lives in a single mapper module and never needs to know what produced the numbers.

The placeholder shipping in production today is a hand-coded heuristic: it extracts six acoustic features from the clip (loudness, mean pitch, rising or falling inflection, repetition rate, onset harshness, and a jitter/shimmer "vocal stress" proxy), scores them against a hand-tuned prototype profile for each class with a radial-basis distance, and softmaxes the result. It is deterministic, explainable, and honestly not very smart - but it produces a valid probability vector, so the product is real and usable while the model catches up. The Swift version on iOS is a 1:1 port of the same math, so web and phone give the same answer.

One is infrastructure, the other is the labels

The infrastructure reason is quick: the backend runs on Cloudflare Workers, which cannot run a normal neural network - no native linear-algebra libraries, a tight CPU budget, no GPU. So model inference has to live somewhere else: exported to ONNX and run in the browser via WebAssembly, or exported to Core ML and run on the iPhone, with the Worker keeping its role as the record / quota / history layer. That is a solved design; it just means "serve the model" is a deliberate export step, not a given.

The real reason is labels, and it is the hard part of most ML products:

Spectrograms are images, so use an image model

The trained classifier turns audio into a picture and then uses a proven image network. Concretely:

audio (any format)
  → resample to 16 kHz mono
  → log-mel spectrogram   # 64 mels, 25ms window, 10ms hop
  → MobileNetV3-Small backbone (1-channel, ImageNet-pretrained)
  → linear head → 9 logits → softmax

A log-mel spectrogram is a 2D image of sound - time on one axis, perceptual frequency bands on the other, brightness for energy. Once audio is an image, decades of image-classification tooling apply. I used MobileNetV3-Small, pretrained on ImageNet, for a few reasons. It is tiny (~2.5M parameters, a few milliseconds on a phone), its ImageNet features transfer surprisingly well to spectrogram "images," and, the part that mattered most here, it exports cleanly to both ONNX and Core ML. I adapted its first convolution from three input channels to one (a spectrogram is single-channel) and swapped the final layer for a 9-way head. That is transfer learning. You keep the pretrained layers and replace the last one.

The training itself is unremarkable on purpose - MobileNetV3 fine-tuned with AdamW at a 3e-4 learning rate, small batches, inverse-frequency class weights so rare intents are not drowned out, checkpointing on the best macro-F1 rather than lowest loss. It is all standard stuff. The hard part is not in the training code.

Labeling data you have no labels for

With real barks but no intent labels, I used weak supervision: automatically generate imperfect labels from a "teacher," train the model (the "student") on them, and be brutally clear that the ceiling is set by the teacher's quality. I built two teachers, and the contrast between them is the lesson.

Teacher 1 - distill the heuristic. Run the existing hand-coded heuristic over real barks and use its softmax as the label. This works end-to-end and gets a model training immediately, but it has a trap hiding in it:

Teacher 2 - an AudioSet model. A far better-grounded teacher is an AST model pretrained on AudioSet that genuinely recognizes dog-sound types - Bark, Growl, Whimper, Howl, Bay, Yip. Run it on each clip, read the type probabilities, and map type → intent through a defined matrix (a growl leans toward warning; a whimper toward discomfort). This is still weak supervision, since acoustic type is not the same thing as intent. But it is grounded in something real about the sound instead of my own hand-tuned guesses, so it is the version worth paying for GPU time on.

Macro-F1 0.29, and why that number ships nothing

The trained model scored macro-F1 0.29 on a held-out split. Read that honestly. With nine classes, random guessing sits around 0.11, so 0.29 is roughly 2.6× better than chance - the model is genuinely learning something from the spectrograms. It is also nowhere near good enough to put in front of a person who wants to know if their dog is in pain.

Two things I built into the evaluation from the start, because on a product like this they matter as much as accuracy:

• Calibration, measured with ECE (expected calibration error). It is not enough to be right often; when the model says "80% sure," it should be right about 80% of the time. A confidently-wrong pet interpreter is worse than a hedged one, so I track calibration right alongside macro-F1.
• Unknown as a safety floor. dog-unknown is both a real trained class and a threshold: if the top confident class falls below a calibrated bar, the result surfaces "Unclear" instead of a specific guess. Low confidence can never graduate into a confident wrong answer. That threshold is the most important safety feature in the app.

The feature frontend has to match across all three runtimes

There is a subtle failure mode unique to shipping the same model to multiple runtimes. The model is trained on log-mel spectrograms computed with a specific recipe: 16 kHz, 64 mel bands, a 25ms window, a 10ms hop. At inference time, that spectrogram gets recomputed - in Python during training, in WebAssembly in the browser, in Swift on iOS. If any one of those three computes the spectrogram even slightly differently, the model receives an input it was never trained on and quietly gets worse, with no error anywhere to tell you why.

Ship the heuristic and collect labels until the model is ready

So the trained model exists, exports to ONNX and Core ML, and beats chance - and it is not in production, because macro-F1 0.29 on weak auto-generated labels is not a number I will stand behind in front of a real person and their real dog. Shipping it would trade an honest, explainable heuristic for a black box that is wrong more confidently. I do not want to make that trade.

What ships instead is the whole product on the heuristic, plus the thing that actually fixes the ceiling: a feedback loop. Every time a user taps "Was this accurate?" and optionally corrects the class, that lands in a table as a human-verified label - the one kind of data no public dataset gave me. Those corrections fold back into the training manifest, the model retrains against a frozen evaluation set, and the version tag bumps so every result is traceable to the brain that produced it. The heuristic is not the end state. It is what generates the labels I need.

That is the part I would want a hiring manager to notice. Fine-tuning MobileNet is the easy part. The harder part was building it so the model can be swapped in whenever it is good enough, measuring it honestly enough to know that day has not arrived, and shipping something truthful until then.

The companion piece

If you want the other side of my model-building - a generative model rather than a classifier, images rather than audio, and a case where the model did turn out worth shipping - I wrote up building a diffusion model from scratch: a DDPM written by hand in PyTorch and trained for about the price of a sandwich. The code for both lives back on my portfolio.