Dev blog #4. The one where I delete a small mountain of work and feel weirdly good about it.
To catch the moment of impact, the app has to be recording before you swing. You don't get a heads-up that the good frame is coming. So the phone has to quietly hold onto the last second or two of high-speed video at all times, and then, the instant you hit, keep only the little slice around the strike and throw the rest away.
That always-on rolling window is the heart of the whole thing. It's also where I face-planted for about a week. Here's what actually went wrong, because the real problems are more interesting than "it was hard."
The naive version, which of course I built first
The obvious approach is a ring buffer: keep a rolling list of the last couple seconds of frames in memory, newest in, oldest out, and freeze whatever's in there when you swing. I had a version of that running on my couch in an afternoon, and I felt great about it. That feeling did not survive the range.
Problem one: a second of high-speed video is gigantic
Do the napkin math, because it's worse than you'd guess. A single frame at the resolution and quality I need is several megabytes of raw, uncompressed pixels. Now run that at a high-speed frame rate, hundreds of frames every second, and try to hold a second or two of them at once. You're suddenly asking to keep something on the order of half a gigabyte to over a gigabyte of raw image data live in memory, and not just once, but refreshed constantly as new frames pour in.
A phone app does not get to use memory like that. It gets nowhere close before the operating system decides you're being greedy and kills the app outright. So the simplest possible idea, "just keep the frames," dies the moment you put real numbers on it.
Problem two: even loading the buffer is a firehose
Set the size issue aside for a second, because there's a second problem hiding behind it. To store a frame, you first have to move it out of the camera's pipeline and into wherever you're keeping it. At a high-speed frame rate that's a new frame every few milliseconds, relentlessly, and each one has to be copied out before the next one lands. That copying is not free. It costs time and it costs heat.
When the copy can't keep pace with the camera, frames get dropped. And dropped frames are the one thing this whole project cannot tolerate, because the single frame you care about, the one with the ball at impact, might be one of the ones that hit the floor. You don't get to choose which frames you lose.
Problem three: it only broke when it actually mattered
Here's the cruel part, and the real answer to "why did leaning on it expose the problem." On the couch I'd record for a few seconds, freeze, and everything looked flawless. But a few seconds never fills the buffer to its limit and never heats the phone up. A real range session is the exact opposite. You're recording continuously for minutes, shot after shot, and two slow-moving problems creep up on you.
First, memory pressure builds as the system strains to juggle that enormous rolling buffer. Second, the phone gets hot, because sustained high-speed capture plus all that copying is genuinely hard work, and a hot phone protects itself by throttling, quietly running slower right when you need it at full tilt. So the copy that kept up fine for ten seconds starts falling behind at minute four. Frames begin slipping, and every so often the frame that slips is the strike.
No crash. No error. Just an occasional "huh, it missed that one." Intermittent failures that depend on accumulated heat and memory pressure are the worst bugs in existence, because you can't make them happen on command. They only show up when you've half forgotten you were even testing for them.
The rethink
The mistake underneath all three problems was the same: I was insisting on hoarding raw, uncompressed frames. That is the single most expensive way to hold onto a moment in time. The fix, in concept and not in code, was to stop doing the expensive thing.
Your phone already knows how to keep video in a compact, compressed form. That's what every clip in your camera roll is. So instead of keeping a mountain of raw pixels, I keep the rolling window in that kind of compact form, and only expand the tiny slice around the strike back into full detail after the trigger fires, once, for the handful of frames I actually need. The buffer shrinks to a fraction of its old size, the phone runs far cooler because it isn't shoving giant raw frames around nonstop, and the firehose mostly stops being a firehose.
I'm not going to hand over the exact build, but that's the honest shape of it. The breakthrough wasn't a clever trick. It was admitting that my whole storage strategy was wrong, and that the phone had already solved most of the problem for me the moment I stopped trying to brute-force it.
The delete
So I did the thing nobody enjoys. I tore the entire raw-frame approach out, days of work, gone, and rebuilt around the compact version. It stung for about an hour and then felt fantastic, because the replacement actually survives a real session instead of slowly cooking itself into dropped frames.
That's the part of building in public worth being honest about. Progress isn't a straight line. Sometimes the most valuable move you make all week is admitting the road you're on doesn't get there, and turning around.
Next up: the phone is on the ground behind you with no buttons to press, so how does it know the exact instant you swing?
Want to follow along? Pre-register and stick around. See you on the range.
0 comments