On-Device AI: Why Your Meeting Data Should Never Leave Your Computer

When you use a cloud-based meeting tool, here's what happens: your audio gets captured, uploaded to a server you don't control, processed by an API you can't audit, and stored in a database you have no visibility into. Some tools keep recordings for 90 days. Some keep them indefinitely. Some use your data to train their models. Most don't tell you which.

For a personal conversation, maybe that's fine. For a business conversation where you're discussing deal terms, client strategies, competitive intelligence, or personnel decisions? That's a different calculation.

The Cloud Model

Most AI meeting tools follow the same architecture:

1. A bot joins your call (or you upload the recording)
2. Audio is sent to a cloud transcription service (typically a third-party API)
3. The transcript is processed by a language model (often another third-party API)
4. Results are stored in the vendor's database
5. You access your data through their web app

At minimum, your conversation passes through 3 different systems owned by 3 different companies. Each one has their own privacy policy, data retention rules, and security posture.

This isn't necessarily bad. Some companies do this responsibly. But you're trusting a chain, and a chain is only as strong as its weakest link.

The On-Device Alternative

On-device AI means the entire pipeline runs locally on your computer. No cloud uploads. No third-party APIs for the core processing. Your audio stays on your machine.

Here's how Sixthly implements this:

Parakeet for transcription. Parakeet is a speech-to-text model that runs directly on your Mac's hardware. It converts audio to text without any network request. The audio file is never created, just processed in memory and discarded. Zero bytes leave your machine.

LanceDB for context retrieval. When Sixthly surfaces a coaching tip, it needs to search across frameworks (negotiation tactics, rapport signals, past meeting context). LanceDB is an embedded vector database that runs locally. No Pinecone, no cloud vector store. The search happens on your SSD.

LLM via your own API key. The one part that does involve a network call is the language model. Sixthly sends a text prompt (not audio) to whichever LLM you choose (Claude, GPT, or Gemini) via OpenRouter, using your own API key. You control which model, you control the billing, and the prompt contains only the specific context needed for coaching, not your full transcript.

What "Private" Actually Means

Many tools claim to be "private" or "secure." These words mean different things:

• Encrypted in transit means your data is encrypted while it travels over the internet. This is the bare minimum. Every cloud service does this.
• Encrypted at rest means your data is encrypted on the server's disk. This protects against a specific attack (someone physically stealing the hard drive) but doesn't prevent the company from reading your data.
• On-device processing means your data never leaves your machine in the first place. There's nothing to encrypt in transit because nothing is transmitted.

On-device is the strongest privacy guarantee because it eliminates the question entirely. You don't need to trust the company's security team, their data retention policy, their employee access controls, or their third-party agreements. Your data is on your machine and nowhere else.

The Tradeoffs

On-device AI isn't free of tradeoffs:

• Hardware requirements. Running ML models locally requires decent hardware. Sixthly requires an Apple Silicon Mac (M1 or later). If you're on an older Intel Mac or Windows, you'll need to wait for broader platform support.
• Model size. Local models are typically smaller than cloud models. Parakeet is excellent for transcription but it's not the absolute largest model available. The quality is more than sufficient for real-time coaching, but it's a design choice.
• Feature velocity. Cloud-based tools can update their models and pipelines instantly. On-device updates require app updates. We ship frequently, but there's an inherent lag.

We think these tradeoffs are worth it. Your meeting conversations are some of the most sensitive data in your business. Keeping them on your device is the right default.

How to Evaluate Meeting Tool Privacy

When evaluating any meeting tool, ask:

1. Where is my audio processed? (On-device or cloud?)
2. Is my audio stored? (If so, for how long? Can I delete it?)
3. Is my data used to train models? (Most cloud tools do this unless you opt out.)
4. How many third parties touch my data? (Transcription API, LLM API, hosting provider, analytics provider?)
5. Can I verify the privacy claims? (Open-source components? Audit reports? Network traffic inspection?)

If the answers aren't clear, that tells you something.

---

Want meeting coaching that stays on your device? Join the Sixthly waitlist. Your conversations never leave your Mac.