Garage:
AI across a workspace platform
A workspace and commerce platform — offices, meetings, webinars and a storefront. I built its AI layer: assistants, meeting intelligence and generative media.
Commercial product — the AI features sit behind sign-in.
CORE CAPABILITIES
Betty — Workplace Assistant
Provider-agnostic tool calling implemented three times over, reading time-tracking and leave history and creating real leave requests.
Ask Cabinet
Upload a video or PDF and ask questions about it — Gemini File API upload, poll to ready, one-shot Q&A.
Meeting Note-Taker
A hidden bot joins the LiveKit room subscribe-only and opens one Deepgram socket per participant.
Transcript Preprocessing
Filler removal, disfluency collapsing and same-speaker merging cut 20–40% of input tokens before the model sees them.
AI Ad Video
A product image becomes a rendered short-form reel, with vision analysis feeding script generation.
Per-Org Key Vault
Organisations bring their own encrypted AI provider keys, with a reported fallback to the platform key.
Architecture
No product screenshots — this is a backend system. The diagram below is the system itself.
Figure 05 — AI surface across the workspace product.
Open full size ↗Betty
Workplace assistant
Three tool schemas, one shape
OpenAI, Anthropic and Gemini tool-calling normalised to a single internal shape, so the assistant is provider-agnostic.
Reads real data
Time-tracking records and leave history, not a canned knowledge base.
Writes real records
Creates actual leave requests, which notify founders over Socket.IO.
Tool calling is implemented three times over — one adapter per provider — because the three vendors disagree on schema shape, and the assistant should not.
Ask Cabinet
Question a file directly
Gemini File API upload
The video or PDF is uploaded, polled until ready, then answered in one shot.
Recording transcription
Produces a timestamped transcript, a summary and action items.
Cached back onto the file
Results are written back to the file record and fired automatically on upload.
Meeting intelligence
The note-taker
CAPTURE
Hidden subscribe-only bot
Joins the LiveKit room without publishing, so it never appears as a participant with media.
One socket per participant
A dedicated Deepgram nova-2 socket per participant, giving attribution without a diarization model.
Incremental persistence
Transcript segments are persisted as they arrive, so a crash can rehydrate rather than lose the meeting.
Resolved-participant email
Distribution goes out by email to the resolved participant list.
ADAPTIVE SUMMARIZATION
Single-pass under budget
One gpt-4o pass when the transcript fits a 15,000-token budget.
Map-reduce above it
Speaker-boundary chunks through gpt-4o-mini, merged by gpt-4o.
Per-chunk success floor
The merge only proceeds when enough chunks succeeded, rather than silently summarising a hole.
The pipeline picks its own shape from the transcript length rather than always paying for map-reduce.
TRANSCRIPT PREPROCESSING AS A COST LEVER
Filler-word set
A known filler set is stripped before the transcript reaches the model.
Disfluency collapsing
Repeated disfluencies are collapsed rather than repeated verbatim.
Same-speaker merging
Consecutive turns from one speaker are merged into a single block.
20–40% fewer input tokens
The cut lands before inference, so it is a cost lever rather than a quality trade.
GENERATIVE MEDIA
AI ad video
Product image to a rendered short-form reel; vision analysis feeds script generation behind a swappable provider interface.
Simulated audience
Proposes plausible attendees and a timed chat script for a webinar.
Schema re-validation
Generated output is re-validated against a schema and messages are truncated before render.
Roster attribution guard
Any chat line attributed to a name outside the generated roster is dropped rather than displayed.
PER-ORG ENCRYPTED KEY VAULT
Bring your own key
Organisations store their own AI provider keys, encrypted at rest.
Graceful fallback
A stored key that is expired or out of quota falls back to the platform key.
Reported provenance
The response reports which key was actually used, so the fallback is never silent.
SIMULATED AUDIENCE
A webinar can be rehearsed against a plausible audience. The output is treated as untrusted: it is re-validated against a schema, messages are truncated, and any line attributed to a name outside the generated roster is dropped before it renders.
TECHNICAL ARCHITECTURE
backend
Node + TypeScript
Express API surface
MongoDB
Primary datastore
BullMQ on Redis
Async media and AI jobs
realtime
LiveKit + mediasoup
Meetings, webinars and the note-taker bot
Socket.IO
Live notifications and assistant events
Deepgram nova-2
One socket per meeting participant
ai
OpenAI
gpt-4o and gpt-4o-mini summarization
Anthropic + Gemini
Normalised into one tool-calling shape
Per-org key vault
Encrypted provider keys with platform fallback