An AI product that runs on your laptop, not ours
HuddleOwl coaches you through a live call and scores how you ran it about forty seconds after you hang up. We designed and built it end to end, and re-engineered the open-source inference stack underneath so the whole thing answers in time, offline, on a machine you already own.
The product
What HuddleOwl is
Every AI meeting tool tells you what happened after it happened. HuddleOwl tells you what to do about it while you can still do it. A card appears while you are still talking, quoting the exact line that triggered it: the objection you talked past, the question you skipped, the buying signal you missed. Then, about forty seconds after you hang up, a scored review of how you ran the call, with the sentence behind every score.
The part that makes it hard is where it runs. Nothing joins the call, and on a Mac with the on-device engines nothing leaves the laptop, wifi off included. That is not a privacy checkbox bolted on at the end. It is the constraint the whole product was engineered around, and it is the reason this case study is mostly about models.
The hard part
Making open models fast enough to coach you live
Live coaching is a latency problem before it is an AI problem. A cue that arrives after the moment has passed is just a transcript. So the real engineering here is not the prompt. It is everything underneath it: two open-source inference stacks, customised, re-tuned and reverse engineered until a laptop can transcribe both sides of a call and write a coaching cue while the sentence that triggered it is still in the air.
Ship the audio to somebody's API
- Audio leaves the machine before anything can be said about it
- One general model asked to do transcription and judgement at once
- Latency you do not control, billed per minute and per token
- Whatever fits one context window, which on a 40 minute call is the last few minutes
- Off the table entirely for NDA, legal, health and HR calls
Two open engines, compiled in and re-tuned
- whisper.cpp and llama.cpp built from source into the app, not called over a network
- Each engine tuned to its own latency budget and switched independently
- Quantised and sized to real hardware: 3.1 GB on disk, 8 GB RAM floor
- Map-reduced across the entire transcript, chunked so a quote is never half a sentence
- Wifi off and nothing changes: same rubric, same evidence quotes, same speed
Two engines, two different jobs
Transcription and coaching were split apart on purpose, because they fail in opposite directions. A Whisper-family model runs through whisper.cpp on both audio tracks at once; a separate local language model runs through llama.cpp and has to write a usable cue before the speaker has moved on. Each is optimised, sized and swapped against its own budget instead of one compromise serving both.
Tuned for time to first cue
The number that matters is not tokens per second on a benchmark, it is whether the card lands while the moment is still open. The runtime was rebuilt around that: a live loop that scores the call as it moves, rather than a request that waits for a turn to end.
Sized to the machine, not the leaderboard
574 MB for transcription, 2.5 GB for the coach, 3.1 GB together. The app reads the hardware before it offers you a model and tells you what will not fit, instead of downloading three gigabytes and then reporting itself unavailable. 8 GB of RAM is the floor, 16 GB is comfortable.
Both sides, and nothing joins the call
No bot, no extra participant, no calendar connection. Both halves of the conversation are captured on the machine itself, natively on macOS and through loopback on Windows, so nobody on the call sees anything different from a normal one.
Language detected once, then locked
23 languages, detected per audio track and then held. Left to itself a streaming transcription model will drift mid-call and start rendering one speaker as a different language. Locking it per track was a small change with a large effect on whether the transcript is usable at all.
The whole call, not the last context window
Most tools summarise whatever fits in one window, which on a long call means the last few minutes and a confident tone. This map-reduces the full transcript, and the chunker never splits a speaking turn, so a quote is never half a sentence.
The evidence gate lives in code, not in a prompt
A small local model will happily invent a plausible-sounding coaching note. So a cue that cannot point at the sentence it is reacting to is discarded before it reaches the screen, and that rule sits in the backend where a model cannot talk its way past it.
Your runtime, if you already have one
Already running Ollama or LM Studio? HuddleOwl uses it, and you pick which model answers rather than the app taking whatever the server listed first. Prefer a hosted model? One interface fronts 100+ providers or your own endpoint, on your key.
Local is the default, not the degraded mode
Offline modes usually mean a worse product. Here the rubric, the evidence quotes and the roughly forty seconds to a brief are identical with the wifi off. On-device engines ship on macOS today; the Windows local engines are still being built, and the product says so plainly.
The build
What we built, layer by layer
Not a wrapper around an API. A signed desktop application, every screen in it, and the inference stack underneath, all built by us.
The meeting record
Every call you have run, searchable full text across transcripts, cues, decisions and follow-up emails, all of it on your own disk. Transcript, summary, decisions, owned action items and a follow-up email in your voice: the notetaker features, without the per-seat bill.
The live coaching engine
Both sides of the call captured natively on the machine, transcribed live, and read continuously for the things worth interrupting you about. Signal scores move as the call moves, so you can see it slipping before it has slipped.
The post-meeting brief
Not a summary. A scored rubric for the role you were playing, a verdict, and the one next step that matters, about forty seconds after you hang up. It is map-reduced across the whole transcript, not the last few minutes that fit a context window, and a dimension the call never tested is left unscored rather than quietly given a zero.
Evidence behind every score
Every dimension is a button. What it opens is the reasoning and the exact line you said. You can disagree with a score, which is the whole point. You cannot disagree with a transcript.
The on-device model manager
Each model shows its size, its licence and what it is for before you download it, and the app checks your hardware first rather than handing you three gigabytes that will not run. Install both, switch the engines to this machine, and the wifi can go off.
The stealth overlay
Coaching nobody else in the call can see: absent from the recording, the Zoom share and the Loom, with clicks passing straight through to whatever is underneath. At 50% opacity over an unknown desktop a themed palette composites to a smear, so the whole app drops to the highest contrast pair it has.
How we shipped it
Built to be given to strangers
A downloadable app is a harder promise than a web app. It has to install on machines you will never see, and the first thirty seconds decide whether anyone gets to the product at all.
Built with Claude Code
The desktop app, the local backend and the release tooling were built with Claude Code, the same loop we run on client work.
Signed and notarised
Apple Developer ID signed and notarised by Apple from 0.4.0, so Gatekeeper passes it. One dialog with an Open button, and nothing to paste into Terminal.
One workflow ships everything
A GitHub Action pulls the three installers, writes SHA256SUMS, publishes the release, rewrites the version everywhere in the docs and pokes the site to rebuild. Nothing is edited by hand afterwards.
Notices generated, not written
Every bundled component, its licence and its copyright holder are generated from real dependency metadata and ship inside every installer, because shipping other people's open source properly is part of the job.
Built with
The stack underneath it
Claude Code
An Electron shell over a React interface, with a Python service frozen into the bundle, and the two inference engines compiled from source rather than pulled from a package registry. Every component is permissively licensed and every notice ships with the binary.
More work
More we've built and shipped
Want AI that runs where your data already lives?
Local-first, open-weight, privacy-first: it is a real engineering choice, not a marketing one, and it is buildable today. Tell us what you have in mind and in 30 minutes we will tell you whether it should run on device, in your cloud, or not be built at all.
Free 30-minute call. Honest take, even if that means “you don't need us yet.”