ElevenLabs

Answer phone calls with an ElevenLabs voice agent that uses a Squid agent as its brain

The ElevenLabs connector's functionality

The ElevenLabs connector makes ElevenLabs Agents host an agent's phone calls:

• ElevenLabs voices. Every voice of your ElevenLabs account, cloned and designed ones included, with the eleven_v3_conversational model.
• The Squid agent as the LLM. ElevenLabs runs the conversation and sends every turn to the Squid agent, which answers with its knowledge bases, connectors and AI functions.
• Multilingual calls. The call starts in the line's language and can switch to the languages you allow, or any language ElevenLabs supports, when the caller speaks one.
• Call manners built in. A filler phrase when an answer takes a moment, hanging up after 20 seconds of silence, ending the call when the caller says goodbye, and voicemail detection on outbound calls.
• Outbound calls from the agent's number through ElevenLabs.

Squid provisions everything at ElevenLabs for you when the agent's phone line picks the ElevenLabs engine: the ElevenLabs agent, the imported Twilio number and the credential the ElevenLabs agent uses to reach Squid.

Before you begin

• An ElevenLabs account with access to the Agents platform.
• An ElevenLabs API key allowed to manage agents, phone numbers, workspace secrets and voices.
• A Twilio connector with a number for the agent. ElevenLabs imports that number into its own account using the Twilio credentials of the connector, so the number must not already be imported there.

Adding the ElevenLabs connector to your Squid application

1. Navigate to the Connectors tab in the Squid Console.

2. Click Available Connectors.

3. Find the ElevenLabs connector, and select Add Connector.

4. Provide the following configuration details:

Connector ID: A string that uniquely identifies the connector in your code, such as elevenlabs.

API Key: The ElevenLabs API key. Stored as a secret.

Default Voice ID: Optional. The voice for phone lines that pick none. Without it, ElevenLabs' default voice is used.

Public Base URL: Optional. The public base URL ElevenLabs reaches this application at, only needed behind a proxy, a tunnel or a custom domain. Defaults to the application's backend URL.

API Base URL: Advanced. ElevenLabs' API base URL, default https://api.elevenlabs.io.

The connector configures nothing per agent. Which agents answer with it, with which voice and languages, is decided on each agent's phone line.

Using ElevenLabs on a phone line

1. In Agent Studio, add the Twilio ability to the agent (or open it), and pick a Phone number.

2. Set the Voice engine to ElevenLabs and choose the ElevenLabs connector to use.

3. Pick a Voice from the account's voices, or leave the default.

4. Set the Language the call starts in and, under Languages, whether the caller may switch to selected languages or to any language.

5. Write the Greeting, and optionally a Filler message and after how many seconds of waiting it is spoken (default 4).

6. Save. The status line reports that the number rings this agent through an ElevenLabs agent.

The full list of options is in the Twilio connector's phone line settings.

What Squid creates at ElevenLabs

Saving the line provisions, in your ElevenLabs account:

• An agent named Squid <agentId>, configured with the greeting as its first message, the voice, the language and its switchable languages, and a system prompt that only carries phone manners: everything the agent knows lives on the Squid agent.
• The agent's LLM set to Squid's OpenAI-compatible endpoint for the Squid agent, https://[APP_ID].[REGION].squid.cloud/ai/openai/agents/<agentId>/v1, authenticated with a workspace secret holding your application API key.
• The phone number, imported from Twilio and assigned to the ElevenLabs agent, so ElevenLabs answers calls to it.
• The system tools end_call, voicemail_detection (which leaves a short message in the call's language on a voicemail and hangs up) and, when the caller may switch language, language_detection.

Changing the line updates the ElevenLabs agent in place. Disconnecting the ability, or switching the line to another engine, deletes the ElevenLabs agent and releases the number, which then goes back to Twilio's control. The ids of what was created are on the line as provisioning.elevenLabs.

Languages

The call can start in any language ElevenLabs speaks. The languages a caller can switch to mid-call are limited to the ones ElevenLabs supports as agent presets: en, ja, zh, de, hi, fr, ko, pt, pt-br, it, es, id, nl, tr, fil, tl, pl, sv, bg, ro, ar, cs, el, fi, hr, ms, sk, da, ta, uk, ru, hu, no and vi. Saving a line that allows switching to another language fails with "ElevenLabs cannot switch a call to …"; that language can only be the one the call starts in. Switching is deliberate: a name, a single word or a number never changes the language, a whole sentence in an allowed language does.

Outbound calls

placeCall on the Twilio connector, and the agent's placeTwilioCall function, place the call through ElevenLabs when the agent's line uses the ElevenLabs engine. The ElevenLabs agent greets whoever answers and runs the conversation; a voicemail gets a short message. Per-call instructions are not supported on this engine: put what the agent should accomplish in its own instructions instead.

Connecting the ElevenLabs connector itself as an ability of the agent adds one AI function, placeElevenLabsCall, which does the same from a prompt ("Call +1 555 010 0100").

Limitations

ElevenLabs hosts the call, so some of what the Twilio and OpenAI engines offer is not available:

• No transcript or call record in Squid. Transcripts and recordings are in the ElevenLabs dashboard. Squid's twilio_calls collection only holds outbound calls placed from Squid, without their outcome.
• No SMS on that number. Squid does not wire the number's messaging webhook up; incoming texts are not delivered to the agent. Keep texting on a number run by the Twilio or OpenAI engine.
• No opening line composed by the agent. The first message is always the greeting as written.
• No transfers and no call context in AI functions. The ElevenLabs agent ends calls with its own tool. transferTwilioCall and endTwilioCall do nothing on this engine, and the agent's functions do not receive a twilioCallSid.
• No browser calls through this engine.

Using the ElevenLabs connector from code

npm install @squidcloud/elevenlabs-client

Backend code

import { SquidElevenLabsClient } from '@squidcloud/elevenlabs-client';

const elevenLabs = new SquidElevenLabsClient(this.squid, 'elevenlabs'); // your connector id

// The account's voices, cloned and designed ones included: what the Twilio ability lists.
const { voices } = await elevenLabs.listVoices();

Method	Executable name	Description
listVoices()	elevenLabsListVoices	Voices of the account.
provisionAgent({ agentId, phoneNumber, twilio, voice?, existing? })	elevenLabsProvisionAgent	Creates or updates the ElevenLabs agent for a Squid agent and imports the number. Needs the API key.
releaseAgent({ agentId, phoneNumberId? })	elevenLabsReleaseAgent	Deletes an ElevenLabs agent and releases its number. Needs the API key.
placeCall({ agentId, phoneNumberId, to })	elevenLabsPlaceCall	Places an outbound call answered by an ElevenLabs agent. Needs the API key.

provisionAgent, releaseAgent and placeCall take ElevenLabs ids, not Squid agent ids; they are the building blocks the Twilio connector uses, and you rarely need them directly.

Troubleshooting

Symptom	Cause and fix
"ElevenLabs cannot switch a call to …" when saving the line	One of the additional languages is not an ElevenLabs preset language. Remove it, or make it the language the call starts in.
Saving fails with an ElevenLabs error about the phone number	The number is already imported in the ElevenLabs account, by hand or by an earlier line. Remove it in the ElevenLabs dashboard and save again.
Saving fails with a 404 from ElevenLabs	The ElevenLabs agent or number Squid created was deleted in the ElevenLabs dashboard. Disconnect the ability and connect it again so they are recreated.
The voice answers but every reply is an apology or silence	ElevenLabs cannot reach the Squid agent: the application is not deployed, or Public Base URL points at the wrong place. Check the application log for the requests to /ai/openai/agents/….
A different voice than expected	The line has no voice and the connector no default voice, so ElevenLabs picked its own. Set one of the two.