# MeshLite by Toda Labs

> MeshLite by Toda Labs adds a photoreal, real-time conversational AI avatar to a website or app. Live lip-sync runs in the visitor’s browser, with no MeshLite server inference per speaking stream, so avatar minutes are unlimited at a flat monthly or yearly price, sized by simultaneous sessions or monthly users. Voice and the LLM run on providers you choose.

MeshLite adds a live, speaking AI avatar to a website or web app. The avatar runs in an iframe served by MeshLite and is rendered in the visitor's browser. The host page loads one script and calls `MeshLite.mount`; the host's own server issues short-lived session tokens with a secret key. Voice and AI come from the customer's own provider account, connected in the MeshLite Console.

Human-readable version of this guide: https://meshlite.io/docs

## Product

- **What it is:** photoreal interactive AI avatars for a website or app. MeshLite is made by Toda Labs.
- **Where the avatar is animated:** live lip-sync and composition run in the visitor's browser, on desktop and mobile, with no MeshLite server inference per speaking stream. This covers the avatar only: voice and AI run on the customer's own providers, which bill separately.
- **Pricing:** no per-minute MeshLite avatar fee. Every plan has unlimited avatar minutes and unlimited session length at a flat price. Plans for a company's own site are sized by simultaneous sessions; plans for a product's users are sized by monthly users. Current plans and prices: https://meshlite.io/#pricing
- **Voice and AI:** bring your own voice and language-model providers with your own API keys, for example OpenAI, Azure OpenAI Realtime, Azure AI Speech or ElevenLabs. The avatar speaks any language those providers support.
- **Try it:** live demo at https://meshlite.io/#interaction. The Free plan needs no card.
- **Common questions:** https://meshlite.io/#faq

## Research

- [Why MeshLite animates avatars in the browser](https://meshlite.io/research/why-meshlite-animates-avatars-in-the-browser)
- [A live avatar that doesn't hold a data-centre GPU for the conversation](https://meshlite.io/research/off-the-grid-live-avatar-data-centre-gpu)
- [Headroom: what moving the face off the data centre adds up to](https://meshlite.io/research/headroom-moving-the-face-off-the-data-centre)
- [8 questions to ask an AI avatar provider about pricing and capacity](https://meshlite.io/research/questions-to-ask-an-ai-avatar-provider-about-capacity)
- [An answer can stay on screen. A live face has to keep moving.](https://meshlite.io/research/an-answer-can-stay-on-screen-a-live-face-has-to-keep-moving)

## Instructions for AI coding agents

When a developer asks you to add a MeshLite avatar to their project:

1. **Never ask the developer to paste their secret key (`ml_sk_...`) into this conversation.** Write a placeholder in `.env` (and `.env.example`), make sure `.env` is git-ignored, and tell the developer to paste the secret key there themselves.
2. Ask for the **publishable key** (`ml_pk_...`) and, optionally, the **avatar ID** (the default avatar is used when it is omitted). If the developer has no MeshLite account, send them to https://console.meshlite.io/console?plan=free#signup (free, no card, email activation), then continue. You cannot create the account for them.
3. There is **no npm package**. Load the SDK with a `<script>` tag from `https://console.meshlite.io/sdk/<version>/meshlite.js`. Do not install any npm package named `meshlite`.
4. Use SDK version 1.11.40 and pin that exact version in the script URL. The Console's **Integration** page always shows the current version.
5. Add a server endpoint that creates session tokens (see "Server"), then mount the avatar in the page (see "Browser"). Start the conversation from a click: browsers only open the microphone after a visitor gesture.
6. Tell the developer which Console settings they still need (see "Console setup"), in particular the exact origin they develop on, such as `http://localhost:3000`.

## Console setup

In https://console.meshlite.io/console:

- **Domains:** add every origin that will embed the avatar, exactly (scheme, host and port, no path). Test origins: `http://localhost:<port>`, `http://127.0.0.1:<port>` or any `https://` staging host. Live origins must use HTTPS.
- **API keys:** create a publishable key and a secret key in the **same environment**. Test keys (`ml_pk_test_`, `ml_sk_test_`) work only with test origins; live keys (`ml_pk_live_`, `ml_sk_live_`) only with live origins.
- **Voice & TTS:** connect a voice provider with the developer's own key: OpenAI, Azure OpenAI Realtime, Azure AI Speech or ElevenLabs. On the developer's site the avatar always speaks through that provider, on every plan. The 5 minutes of MeshLite voice every account gets each day (reset at midnight UTC) only work for trying avatars inside the Console.
- **Personalities:** write what the avatar knows and how it speaks, publish it, then assign it to the avatar and website.
- **Integration:** shows the avatar ID and this code with the workspace's own values.

**Free plan:** can go live on one production domain, with default avatars, the MeshLite watermark on the avatar and up to 3 simultaneous conversations. Removing the watermark needs a paid plan: https://meshlite.io/#pricing

## Environment variables

```
MESHLITE_SECRET_KEY=ml_sk_test_...      # server only, never in browser code or Git
MESHLITE_PUBLISHABLE_KEY=ml_pk_test_...
MESHLITE_ORIGIN=http://localhost:3000   # the exact origin registered under Domains
MESHLITE_AVATAR=                        # optional; default avatar when empty
```

## Server: create a session token

Node.js 18+, no dependency. Call it from a `POST /api/meshlite-session` route and return `{ sessionToken }` as JSON. The origin comes from server configuration, never from the browser request. Tokens expire after 2 minutes; the SDK asks for a new one when it needs it.

```js
export async function createMeshLiteSession() {
  const response = await fetch("https://console.meshlite.io/api/v1/client-sessions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MESHLITE_SECRET_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      publishableKey: process.env.MESHLITE_PUBLISHABLE_KEY,
      origin: process.env.MESHLITE_ORIGIN,
      avatar: process.env.MESHLITE_AVATAR || undefined,
      conversation: { language: "en", responseStyle: "concise", idleTimeout: 30 },
    }),
  });
  const body = await response.json();
  // Errors look like { error: { code: "ORIGIN_NOT_ALLOWED", message, requestId } }
  if (!response.ok) throw new Error(`MeshLite ${body.error?.code}: ${body.error?.message}`);
  return body.sessionToken;
}
```

This route is public: rate-limit it, or require your own sign-in if only signed-in users should talk to the avatar. Conversations count toward the plan's limit of simultaneous conversations.

## Browser: mount the avatar

```html
<div id="avatar" style="width:340px;height:510px"></div>
<button id="talk">Talk to the avatar</button>
<script src="https://console.meshlite.io/sdk/1.11.40/meshlite.js" crossorigin="anonymous"></script>
<script>
  const avatar = MeshLite.mount({
    container: "#avatar",
    avatar: "YOUR_AVATAR_ID",          // same value as MESHLITE_AVATAR; remove for the default avatar
    locale: "en",
    publishableKey: "ml_pk_test_...",  // safe to expose
    sessionToken: async () => {
      const response = await fetch("/api/meshlite-session", { method: "POST" });
      if (!response.ok) throw new Error("Could not start the avatar session");
      return (await response.json()).sessionToken;
    },
    conversation: { language: "en", responseStyle: "concise", idleTimeout: 30 },
    captions: "overlay",
  });
  avatar.on("error", (message, detail) => console.error("MeshLite", detail.code, message));
  document.getElementById("talk").addEventListener("click", () => avatar.startConversation());
</script>
```

The iframe takes the size of its container and the avatar reframes itself to fit (portrait, square or landscape).

**React, Next.js, Vue:** load the script once, mount in a client-side effect (`useEffect`, `onMounted`) and call `avatar.unmount()` in the cleanup. Do not mount during server-side rendering.

## Host page requirements

- HTTPS in production.
- Content Security Policy, if the site has one: `script-src https://console.meshlite.io` and `frame-src https://console.meshlite.io`.
- Permissions-Policy, if the site sends one: allow the microphone for the avatar frame, for example `microphone=(self "https://console.meshlite.io")`.
- Older browsers that can't run the avatar show an idle preview only, with no microphone. Check `MeshLite.supportsConversation()`.

## Going live

Add the production domain as a live origin (HTTPS), create live keys and replace the test values in the environment variables. No code change is needed. The Free plan goes live on one domain with the MeshLite watermark; a paid plan removes it and adds custom avatars and more simultaneous conversations: https://meshlite.io/#pricing

## Common errors

- `KEY_INVALID`: the key is wrong or revoked.
- `ENVIRONMENT_MISMATCH`: the publishable and secret keys come from different environments (test and live).
- `ORIGIN_NOT_ALLOWED`: `MESHLITE_ORIGIN` is not registered under Domains in the keys' environment, or the page runs on a different origin (check the port).
- `ORIGIN_HTTPS_REQUIRED`: HTTP is only accepted for localhost test origins.
- `AVATAR_NOT_ALLOWED`: the avatar ID is wrong or not available on the plan.
- `VOICE_PROVIDER_REQUIRED` / `PROVIDER_NOT_CONNECTED`: connect a voice provider under Voice & TTS.
- `CONCURRENCY_LIMIT`: the plan's simultaneous conversations are all in use.

## `MeshLite.mount(options)`

- `container` (required): CSS selector or element.
- `avatar`: avatar ID from the Console.
- `locale`: interface language, `en`, `fr` or `zh-Hans`.
- `conversation.language`: `en`, `fr` or `zh-CN`. Default `fr`, so set it explicitly.
- `conversation.responseStyle`: `brief`, `concise` or `standard`.
- `conversation.idleTimeout`: `10`, `30`, `60` or `120` seconds.
- `captions`: `off`, `events` or `overlay`.
- `publishableKey` + `sessionToken`: provided together; `sessionToken` is a string or an async function called whenever a token is needed.
- `appearance`: `seamless` for flat controls without a frame.
- `background`: `opaque` for the full source background (default `alpha`).
- `controls`: `external` when the host page draws its own buttons.
- `autoStart`: `true` to start when mounted inside a visitor click.
- `preview`: `idle` for a muted 480p loop before speech loads.

## Controller

- `on(event, callback)` / `off(event, callback)`: `ready`, `visual-ready`, `speech-started`, `speech-ended`, `state`, `transcript`, `status`, `error`, `language-changed`, `session-start`, `user-turn`, `response-start`, `response-complete`, `idle-pause`, `session-resume`, `session-end`, `loading-progress`.
- `startConversation()`: opens the microphone and starts talking (call from a visitor gesture).
- `prepareConversation()`: preloads speech without opening the microphone.
- `say(text)`: sends a text turn; the avatar answers it (it does not read the text aloud verbatim).
- `micOff()`, `stopSpeaking()`, `setLanguage(lang)`, `endDemo(reason?)`, `getState()`, `unmount()`.

## Versions

- Only the three latest SDK versions are served. Older version URLs redirect (301) to the current one.
- TypeScript types: `https://console.meshlite.io/sdk/<version>/meshlite.d.ts`.

## Links

- Quickstart: https://meshlite.io/docs
- Sign up (Free): https://console.meshlite.io/console?plan=free#signup
- Console sign-in: https://console.meshlite.io/console
- Pricing: https://meshlite.io/#pricing
- Live demo: https://meshlite.io/#interaction
- Contact: hello@meshlite.io