The Biteshot Blueprint
Our engineering tenets, trade-offs and target architecture
Biteshot reads a photo of a meal, estimates what's on the plate, plans the next meal, and runs a small social feed of home cooking, natively on iOS and Android. This post is for engineers. It isn't a tour of today's servers. It's the target architecture we're steering toward: the constraints we hold ourselves to, the design choices that follow from them, and what's next on the roadmap.
1. Our engineering tenets
These are the rules we use to make decisions when the answer isn't obvious. They guide the rest of this post. They are ours until we learn better ones.
- Users' data stays with users. Health and diet data lives on the person's device. The server is designed so it never needs it, and every request sends the least it can.
- Pay only when people use it. Nothing runs or bills while the app is idle. Any always-on cost has to earn its place with a measured improvement users can feel.
- Fail closed on safety and spend. Fail open on convenience. If we can't verify moderation or a spending cap, we stop. If we can't write a convenience counter, we let the person carry on.
- Never a dead end. When something upstream is busy, the app retries once, saves the work for later, or answers from what's already on the phone. "Try again later" is the last resort, not the plan.
- Rules live in code, once. Safety, diet and allergy rules are code, never prompt text, and the same tests pin them on every platform and the server.
- Measure, then decide. Latency budgets, accuracy on real plates and cost per request decide our trade-offs. Opinions start the discussion and numbers end it.
- Generation is cheap. Verification is the product. Whoever writes a change, person or agent, never approves it. Every change gets an adversarial review and tests that cross module boundaries.
- No regressions. Something that worked yesterday must work tomorrow. Every deploy is followed by reading live behaviour, not just a green test run.
- Delete as you go. When a design changes, the old path, its strings and its tests leave in the same change. We would rather have less code than clever code.
- Move fast through two-way doors, slowly through one-way doors. Reversible choices are made quickly and revisited with data. Irreversible ones, such as where data lives or what we promise users, get the most scrutiny.
- Parity is a feature. iPhone and Android users get the same app, the same rules and the same safety, in all five languages.
- People own the calls that matter. Agents build and review. People decide product, pricing, privacy and anything said in public.
2. Target topology
We build on serverless Google Cloud and Firebase because scale-to-zero is a hard requirement. The trade-offs are vendor lock-in and a single region. We accept both until real traffic says otherwise, and keep provider-specific code behind small interfaces so either can change later.
3. Latency budgets and how we plan to meet them
We set a budget per interaction type, then design each path to fit it.
| Interaction | Target p50 | Target p95 | Main lever |
|---|---|---|---|
| Open the feed, warm | < 400 ms | < 1 s | Parallel reads, paging, client cache |
| Open the feed, cold | < 1.5 s | < 2.5 s | Lazy route loading, a small warm pool |
| Scan a meal | < 3 s | < 6 s | Lite models, temperature 0, short per-hop timeouts |
| Companion reply | < 3 s | < 8 s | Busy-model memory, cross-vendor fallback |
| Week plan | < 6 s | < 15 s | Streamed progress, first day shown early |
- One HTTP surface, lazy routes. One function shares warm instances across every route, where many small functions would each cold-start on their own. To keep that one function light, a route table maps each path to the module that serves it, and the module is imported only on first use. A test pins the table to the real routes so nothing is forgotten.
- Fan out, never chain. Feed assembly issues its independent reads concurrently and pages results, so the first screen never waits for the tenth.
- A small warm pool where it pays. Cold starts dominate a scale-to-zero service's tail latency. The plan is a minimal warm instance for the user-facing surface only, sized by measured tail latency, with rarely used admin code moved into its own function so cold starts stay lean.
- Edge-cache what's the same for everyone. The curated part of the feed is identical for every user of a language on a given day. It fits CDN caching with a daily key, which takes it off the function entirely.
- Stream long answers. A week plan should show its first day while the rest are written, instead of one long spinner.
4. Data placement and schema strategy
The most important decision is that the food diary is not a server table. Meals, body details, goals and health data live in on-device stores: SwiftData on iOS, Room on Android. There's no per-user diary schema to migrate, no health-data breach surface, and no database bill that grows with usage. Every feature that needs the diary either runs on the device or sends only what a request needs. A plan request carries goals, diet, allergies and recent meal names, never the diary.
Schema rules we follow
- Denormalize for the read path. A feed card document carries what the card shows (author handle, avatar, tags, allergens), so a page is one query plus batched counts, never a join.
- Counters are transactional and floored at zero. One document per counter until it gets hot. The transaction boundary makes sharding a local change when it's needed.
- Every privacy promise has an expiry and a sweeper. Hashed addresses, inbox items and rejected comments carry an explicit expiry time, and a scheduled job deletes them, so no promise depends on a database setting.
- Identifiers are random, secrets are hashed. Nothing user-facing is sequential, and no secret or address is stored in a reversible form.
- Cross-device sync, if we add it, will be a bridge, not a table. The AI-app inbox is the pattern: a narrow, expiring channel the device drains, never a server copy of the diary.
5. Redundancy in the AI path
Free-tier model endpoints fail in bursts, and models get retired with little notice. The path is designed so no single provider, model or bad afternoon stops the app.
- Remember who's struggling. Providers publish no live load signal, so each instance remembers recent failures (429, 500, 503, timeouts) for a few minutes and tries those models last.
- Respect retry hints. A provider that says "retry in 23 s" is skipped until then, instead of paying a failed round trip on every request.
- Per-hop timeouts inside one request deadline. A fast fallback beats a slow answer, and no hop may outlive the request.
- Isolate workloads. Moderation, chat and scans use different models and quota buckets, so one can't starve another.
- Cross-vendor, not just cross-model. At least one fallback always sits at a different company.
- A capped, pre-charged overflow with priority. A billed overflow can answer anyone when free tiers are saturated. Each attempt is charged before it's sent, at a deliberately high estimate, and settled to real usage afterwards. The cap fails closed, and paying users always keep headroom.
- Degrade on the device. The app retries once, and only on the server's busy signal, never on client timeouts, which would double cost. After that it saves the scan to finish later, or answers common questions locally from the user's own plan and recipes.
6. Accuracy as an engineering loop
- Choose models on measured error. Models are picked by mean absolute calorie error on reference plates from the public Nutrition5k dataset. A lite model beat larger ones on error for its latency, so lite is the default. Next: a larger held-out set, re-run on every model or prompt change.
- Never evaluate on production quota. Evaluations run on separate keys. We learned that one the hard way.
- Determinism where it matters. Scans run at temperature 0, so the same photo gives the same estimate. Plans run warmer, so days vary.
- Structured output, validated twice. Models return JSON. The server parses defensively: one object with stray text is accepted, two competing objects fail, so a safety check can never pick one of two verdicts. Values are clamped, then the app validates again.
- Corrections are data. Every estimate is editable. Thumbs up or down feed back as item names only, which improves prompts without storing meals.
- Safety rules are code. Allergens, diet rules and medical modes are enforced on the server and the device, never left to the model.
7. One rulebook, two platforms
We build natively on both platforms because the camera, health platforms, widgets and billing are where this app lives, and cross-platform layers are weakest exactly there. UI is written twice. Logic can't drift: each platform has a core library that owns nutrition math, safety rules, feed mixing and goal limits, and both are tested against shared fixture files that the server suite loads too.
8. Scaling and cost controls
- Instance caps are a budget. Every function has a maximum instance count, so a spike or a bug has a known worst-case cost. Raising it is a decision, not a surprise.
- Quotas keyed several ways, expiring on their own. Allowances are counted per install, network and account, with automatic expiry. Counter-store errors fail open: a database hiccup shouldn't stop someone logging lunch, and spend caps are the real backstop.
- Per-user cost ceilings route instead of failing. A heavy user past their ceiling moves to the free chain instead of getting an error.
- Global paid caps fail closed. Overflow AI, image moderation and other paid calls have hard caps that are read strictly: unreadable means stop.
- Push work to the device. Barcodes, finding food in the camera roll and photo de-duplication run on-device with Vision and ML Kit, and never touch the server.
- Shard when measured. Hot counters, a second region and read replicas wait for real load. Each is designed to be a local change when it comes.
9. Moderation that fails closed
The feed is used by teenagers and by people in eating-disorder recovery, so moderation must be strict without making posting feel slow. The design is publish-then-verify:
- New content is written as pending.
- A create trigger screens text and images on separate models and a separate quota.
- Flagged content is taken down and queued for a human with a reason, and the author sees why.
- A scheduled sweep re-checks anything stuck, so a missed trigger can't leave content unscreened.
- If screening is unavailable, content is held, never published unchecked.
False positives are fixed in code, not prompts. Tagging a friend ("With @bluey") was once flagged as personal information. The fix replaces handles the server has already resolved to real accounts with a neutral token before screening. Anything else after an @, such as an insult or a phone number, is still screened exactly as typed.
10. Identity and auth
- Pseudonymous by default. Social identity is an account id plus a device-held secret. An optional platform sign-in is used only to restore it on a new phone, and it's stored as a peppered hash. No email, no password.
- Standards for third parties. AI apps connect over OAuth 2.1 with PKCE (S256 only), exact redirect matching, discovery metadata, client metadata documents with dynamic registration as a fallback, short-lived access tokens, and rotating refresh tokens whose reuse revokes the grant.
- Approve on the phone, not in a web login. The user confirms a short code in the app. We never build, store or defend a password.
11. Decision log: the trade-offs, and how we chose
Every decision above had a credible alternative. We score options against the same criteria, in this order: user safety and privacy, then idle and marginal cost, then latency, then build effort for a very small team. We also ask how hard each choice is to reverse. Easily reversed choices (two-way doors) we make quickly and revisit with data. Hard-to-reverse ones (one-way doors, such as where user data lives) get the most scrutiny, because undoing them later means a migration and a broken promise.
Where the food diary lives: on the device or on a server
| Option | Pros | Cons |
|---|---|---|
| On the device (chosen) | No health-data breach surface; no per-user schema or backups to run; cost doesn't grow with users; works offline; simple privacy story | No automatic cross-device sync; server features must ask the device; analytics on eating patterns are impossible by design |
| On a server | Sync and web access for free; server-side insights; easier support | Health data at rest we must protect forever; a database bill that scales with every meal; heavier compliance and deletion obligations |
How we chose: privacy and idle cost both pointed the same way, and this is a one-way door: moving health data off a server later is much harder than adding an optional sync to one. We'd revisit if people ask for multiple devices, and then only as end-to-end encrypted sync the server can't read.
Native on each platform, or one cross-platform codebase
| Option | Pros | Cons |
|---|---|---|
| Native: SwiftUI and Compose (chosen) | First-class camera, health platforms, widgets, billing; platform look and accessibility for free; no bridge layer to debug | UI written twice; risk that logic drifts between platforms |
| Cross-platform UI (Flutter, React Native) | One UI codebase; one team skill set | Plugins lag on exactly the APIs we rely on most; harder platform polish; an extra runtime |
| Shared logic only (Kotlin Multiplatform) | One copy of the rules | Toolchain complexity on iOS; another build system for a tiny team |
How we chose: the app's value sits in platform APIs, so native won on product quality. We contained the main con, logic drift, with core libraries on each side and shared test fixtures. That gets most of the benefit of shared logic without a new toolchain. We'd revisit if the UI-duplication cost starts slowing releases more than platform polish helps them.
One function with lazy routes, many functions, or an always-on server
| Option | Pros | Cons |
|---|---|---|
| One function, lazy routes (chosen) | Warm instances shared by every route; one deploy; one place for quotas and validation | One large deployable; a cold start pays for whatever the first route imports |
| A function per route | Small, isolated deploys | Many cold starts and idle instances; duplicated middleware |
| Always-on container | No cold starts; predictable latency | Pays 24/7 regardless of use; capacity planning |
How we chose: idle cost ruled out always-on. Between the serverless options, sharing warm instances beat isolation, and lazy imports removed most of the one-function penalty. We'd revisit with a small warm pool once measured tail latency justifies its fixed cost.
Document store or relational database
| Option | Pros | Cons |
|---|---|---|
| Serverless document store (chosen) | Nothing to size or pay for while idle; transactions for counters; built-in triggers for moderation | No joins, so we denormalize; ad-hoc queries are limited; hot documents need sharding |
| Managed Postgres | Joins, constraints, rich queries; familiar schema migrations | An instance billed while idle, or a serverless tier with its own cold starts; connection management from functions |
How we chose: with the diary on the device, the server's data is small, social and read-heavy: feeds, counters and short-lived inboxes. That shape fits documents, and the idle cost decided it. We'd revisit if we need reporting across many entities, and would add an analytical store rather than move the live data.
Free models with fallbacks, or one paid provider
| Option | Pros | Cons |
|---|---|---|
| Free tiers + cross-vendor fallback + capped paid overflow (chosen) | Near-zero cost per free user; no single-provider outage; paid capacity reserved for subscribers | More code paths to test; free-tier bursts; data-use terms differ by tier |
| One paid provider for everyone | Simple; consistent latency and terms | Cost grows with every free user; a single point of failure |
How we chose: a free app can't carry a per-user AI bill, so free tiers came first. The redundancy then had to be designed in. Subscribers go to the billed tier, which also keeps their photos out of model training. We'd revisit if paid volume makes one provider's committed-use pricing cheaper than the complexity.
Moderate before publishing, or publish and verify
| Option | Pros | Cons |
|---|---|---|
| Pre-moderation | Nothing harmful is ever visible | Posting feels slow; a model outage blocks all posting |
| Publish, then verify within seconds (chosen) | Posting feels instant; outages degrade to "held", not "broken" | A short window where content is visible before screening |
How we chose: we narrowed the con instead of accepting it. New photos stay hidden until they're screened, text is checked within seconds by a trigger, a sweep catches misses, and anything unscreenable is held. We'd revisit if abuse rises, and could switch text to pre-moderation for new accounts only.
Approving an AI app: a code on the phone, or a web login
| Option | Pros | Cons |
|---|---|---|
| Short code approved in the app (chosen) | No passwords to build, store or defend; works with pseudonymous accounts; the phone already proves identity | An extra app switch, which people can miss; code-phishing risk needs explicit warnings |
| Web sign-in (email or platform login) | A familiar single-screen flow | A whole login system to secure; ties accounts to email |
How we chose: security and our no-password principle outweighed the extra step. We then worked on the cons. The approval screen names the requesting app, flags unverified apps, shows where and when the request started, and tells people to go back to finish. The page also completes as soon as they return. We'd revisit if drop-off at the app switch stays high.
12. How we build: AI agents, review gates and tests
Most of Biteshot's code is written by AI coding agents, directed by one person. That makes engineering discipline more important, not less. An agent can produce a thousand plausible lines in minutes, and plausible is exactly what slips past a tired reviewer. Our development process is built around one idea: generation is cheap, verification is the product.
Multiple agents, separated by role
- Builders and reviewers never mark their own work. Builder agents implement a scoped brief on both platforms. A separate reviewer, on a stronger model with read-only access, checks the diff for regressions, safety and security, and answers GO or NO-GO with concrete failure cases.
- Parallel where independent, serialized where shared. Agents work at the same time on unrelated areas, such as the server connector and the goals screens. They share one branch and commit only their own changes, and anything that touches the same files waits its turn.
- Briefs are specs, not wishes. Each brief names the files, the constraints (parity, five languages, safety rules, no deploys), what's already been ruled out, and what "done" means, including which tests must pass.
- Humans make the product calls. Agents propose and implement. Product, pricing, privacy wording and anything public need an explicit human yes.
Token management is cost management
- Right-size the model to the job. Faster, cheaper models build. The strongest model reviews and hunts hard bugs, where a missed issue costs far more than tokens.
- Keep contexts small. Agents get a focused brief and pointers to the files that matter, not the whole repository. Reviews read diffs, not codebases.
- Targeted tests, not whole suites. Agents run the tests for what they touched; the full suite runs in the background. A green full suite isn't a reason to burn tokens re-running it.
- No polling, no busywork. Long jobs notify when they finish instead of being checked in a loop.
Keeping slop out
"Slop" is code that looks right and isn't: dead branches, duplicated logic, comments that lie, tests that test nothing. Our defenses:
- Every review is adversarial. Reviewers are asked for concrete failures, not impressions. In one week, reviews stopped a gentle-mode screen that still showed calorie numbers, an app-update migration that would have dropped an eating-disorder safety floor for every user aged exactly 30, a fallback parser that could have published a comment when the moderator returned two contradictory verdicts, and a fix that would have hidden insults written as @tags from moderation. Each was fixed and re-reviewed before shipping.
- Delete as you go. When a design changes, the old path, its strings and its tests are removed in the same change. Briefs say so explicitly.
- One source of truth. The spec is updated with every change, and the code is checked against it. Two platforms stay in step because shared fixtures make drift a failing test.
- Tests that test the wiring. Our worst bug passed every unit test: each part worked, and the parts didn't work together. Now critical paths have tests that cross module boundaries, and every deploy is followed by reading live logs and exercising the real endpoint.
- Pre-existing failures are named, not ignored. A known failing test is listed until it's fixed, so a new failure can never hide behind an old one.
13. Roadmap
- A warm pool sized by measured tail latency, for the user-facing surface only.
- A CDN-cached curated feed, keyed by language and day.
- Streaming week plans.
- A larger held-out accuracy set, re-run automatically on every model or prompt change.
- Per-route latency dashboards from day one of every new feature.
- Optional end-to-end encrypted sync between a user's own devices, built as a bridge the server can't read, if people ask for it.
- A second region, when an outage would cost more than running one.
If you're building something small: decide where your users' data lives before the first screen, cap every paid call, and make every safety path fail closed. Most of Biteshot's design follows from those three decisions.
— Ani, founder of Biteshot
Comments
Loading comments…