Gemini 3.5 Live Translate：实时语音翻译 API 指南

import { GoogleGenAI, Modality } from "@google/genai";

const ai = new GoogleGenAI({});

const session = await ai.live.connect({
  model: "gemini-3.5-live-translate-preview",
  config: {
    responseModalities: [Modality.AUDIO],
    inputAudioTranscription: {},
    outputAudioTranscription: {},
    translationConfig: {
      targetLanguageCode: "es",
      echoTargetLanguage: false,
    },
  },
  callbacks: {
    onmessage: (message) => {
      const content = message.serverContent;
      const transcript = content?.outputTranscription?.text;
      const translatedAudio = content?.modelTurn?.parts?.find((part) => part.inlineData);

      if (transcript) console.log("Translated transcript:", transcript);
      if (translatedAudio) {
        // Decode and play the translated PCM audio chunk.
      }
    },
  },
});

// Browser microphone audio usually needs conversion before sending.
// Target input for Live Translate:
// - raw PCM
// - 16-bit
// - little-endian
// - mono
// - 16kHz sample rate
// - roughly 100ms chunks

session.sendRealtimeInput({
  audio: {
    data: pcm16MonoChunk.toString("base64"),
    mimeType: "audio/pcm;rate=16000",
  },
});

Organizer microphone
  -> realtime room audio
  -> translation bridge per target language
  -> Gemini Live API translationConfig
  -> translated 24kHz audio
  -> attendee playback + optional transcript

1. Start in Google AI Studio to test target languages.
2. Use gemini-3.5-live-translate-preview behind a feature flag.
3. Capture microphone audio and convert to 16kHz mono PCM.
4. Send roughly 100ms chunks over the Live API session.
5. Request input and output transcripts for debugging.
6. Keep API keys on the server; use ephemeral tokens for browser clients.
7. Decide whether target language is locked server-side or user-selectable.
8. Test accents, background music, overlapping speakers, long pauses, and rapid language switches.
9. Add visible latency and transcript status in the UI.
10. Disclose AI-generated translated audio and preserve SynthID expectations.