Twilio

Give an AI agent a phone number: it answers calls, places calls and texts on it

The Twilio connector's functionality

The Twilio connector turns a Twilio account into phone lines for your agents:

A number per agent. Connect the Twilio ability to an agent and pick one of the account's numbers. Calls and texts to that number reach that agent.
Inbound calls. The agent answers in real time, with its instructions, knowledge bases, connectors and AI functions. It knows the caller's number, the number that was called and the time.
Outbound calls. The agent calls someone from its number, with a purpose, when asked to or when your backend tells it to.
SMS. Incoming texts are answered by the agent, or handed to your own handler. The agent can text confirmations and links, and your backend can send texts from the agent's number.
In-call actions. The agent can end the call after a farewell or transfer it to another number, and your application can take a live call over for a human.
Records. Every call and every spoken turn lands in the application's built-in database.
A choice of voice engines. Twilio's own speech services work out of the box. Add an OpenAI Voice or ElevenLabs connector and pick it as the engine of any phone line. See choosing a voice engine.

Before you begin

A Twilio account with at least one phone number that has voice and SMS capabilities. Note that a trial account can only call and text verified numbers.
The account's Account SID and Auth Token (Twilio Console, Account Info). The Auth Token is required: it is the only credential that can verify the signature of the webhooks Twilio sends.
Optional: a Twilio API Key (Console, Account > API keys & tokens). When you provide one, Squid uses it instead of the Auth Token for REST calls, and it is required for browser calls through Twilio.
For the Twilio speech engine in its default real-time mode, Twilio must have ConversationRelay enabled on the account: in the Twilio Console, under Voice > Settings > General, accept the Predictive and Generative AI/ML Features Addendum. The turn-based mode and the OpenAI and ElevenLabs engines do not need it.
Sending texts to US numbers needs a number registered for A2P 10DLC (a brand and a campaign) in the Twilio Console, or a toll-free number that has been verified. Twilio blocks unregistered traffic.

Adding the Twilio connector to your Squid application

Navigate to the Connectors tab in the Squid Console.
Click Available Connectors.
Find the Twilio connector, and select Add Connector.
Provide the following configuration details:

Connector ID: A string that uniquely identifies the connector in your code, such as twilio.

Account SID: The Twilio Account SID (AC…).

Auth Token: The Twilio Auth Token. Stored as a secret.

API Key SID: Optional. A Twilio API Key SID (SK…). With an API key, every REST call uses it instead of the Auth Token.

API Key Secret: The secret of that API key. Stored as a secret. Required together with the API Key SID.

The Advanced section holds two settings you only need behind a proxy, a tunnel or a custom domain:

Webhook Base URL: The public base URL Twilio reaches this application at. Defaults to the application's backend URL, which is public on Squid, in production as in local development.

API Base URL: Twilio's API base URL. Defaults to https://api.twilio.com.

Nothing is configured per connector beyond the account. The phone number, the voice engine and everything about how the agent sounds belong to the agent's phone line, described next. Saving the connector again re-checks every phone line that uses it.

Connecting an agent to a phone number

An agent gets its phone line by adding the Twilio ability in Agent Studio:

Navigate to the Studio tab in the Squid Console and open the agent, or create one.
Click Add Abilities, scroll to the Communication section and select the Twilio connector you created.
Pick a Phone number. The list holds the numbers of the account that no other agent uses yet: a number rings one agent only.
Choose the Voice engine. Twilio speech is always available; ElevenLabs and OpenAI voice appear once the application has an ElevenLabs or OpenAI Voice connector. The settings below the engine adapt to it.
Set the Language the call starts in, the Greeting, and the engine's own settings, all described in phone line settings.
Save. The connector now wires the number up: it points the number at this application (or, on the ElevenLabs engine, hands it to an ElevenLabs agent) and reports the outcome in the ability's status line, such as "+1 555 123 4567 rings this agent (OpenAI voice)". A failed setup shows why, for example a number that another agent already uses.
With the OpenAI engine, the status line also shows a webhook URL to register once in your OpenAI project. See registering the incoming-call webhook.

Call the number. Disconnecting the ability releases the number again.

Where the backend runs

Twilio calls your application's webhooks, so the application must be reachable: deploy the backend, or run it locally with squid start, which also serves a public URL of the form https://[APP_ID]-dev-[DEVELOPER_ID].[REGION].squid.cloud. The connector's own code is hosted by Squid and needs no deployment.

Phone line settings

These are the per-agent options of the Twilio ability. In code they live on the agent's connection to the integration as connectedIntegrations[].options, typed as AiAgentTwilioIntegrationOptions in @squidcloud/twilio-client.

Option	Engines	Description
`phoneNumber`	all	The agent's number on the account, in E.164 format (`+15551234567`). Required.
`voiceEngine`	all	`twilio` (default), `openai` or `elevenlabs`.
`openAiVoiceIntegrationId`	OpenAI	The OpenAI Voice connector that answers the calls.
`voiceModel`	OpenAI	The realtime model that talks on the call. Defaults to the connector's default model, then to the newest realtime model.
`voice`	all	The voice: an OpenAI voice name (`marin`, `cedar`, `alloy`, …), an ElevenLabs voice id, or a Twilio provider voice id for the turn-based Twilio mode. Defaults to the engine's default voice.
`allowInterruptions`	OpenAI	Whether the caller's speech interrupts the model. Set from code; omitted values keep OpenAI's default.
`interruptionSensitivity`	OpenAI	`low`, `normal` or `high`: how readily caller speech is detected. Set from code.
`elevenLabsIntegrationId`	ElevenLabs	The ElevenLabs connector that runs the calls.
`language`	all	The BCP-47 tag of the language the call starts in, such as `en-US`, `es-MX` or `he-IL`. Defaults to `en-US`. It drives the speech services and tells the agent which language to answer in.
`welcomeGreeting`	all	Spoken as soon as the call is answered. Defaults to a short greeting in the call's language, when Squid has one; a language without one makes the line wait for the caller to speak first.
`greetingMode`	Twilio, OpenAI	`phrase` (default) speaks the greeting as written. `agent` has the agent compose its opening line once the call connects, so it can look the caller up and greet a known caller by name, with the greeting as its fallback. The price is a moment of silence after pickup. The console calls it Opening line.
`languageMode`	OpenAI, ElevenLabs	`single` (default) keeps the whole call in `language`; `selected` lets the caller switch between it and `additionalLanguages`; `any` follows the caller into any language the engine can switch to.
`additionalLanguages`	OpenAI, ElevenLabs	ISO codes (`en`, `es`, `fr`) the caller may switch to in `selected` mode.
`fillerMessage`, `fillerAfterSeconds`	ElevenLabs	A phrase spoken when an answer takes more than a moment, and after how many seconds of waiting (default 4). Defaults to a phrase in the call's language.
`voiceMode`	Twilio	`relay` (default) streams the conversation through Twilio ConversationRelay; `gather` is turn-based, speaking one complete answer per caller utterance and ending the call after two silent turns.
`ttsProvider`, `transcriptionProvider`	Twilio (relay)	Twilio's text-to-speech provider (`ElevenLabs`, the default, `Google` or `Amazon`) and speech-to-text provider (`Deepgram`, the default, or `Google`).
`disableBuiltInSmsReply`	all	When `true`, the connector does not answer incoming texts with the agent; your own SMS handler does. Set from code.
`provisioning`	all	Written by the connector after every save: `{ state: 'ready' \| 'error', message?, updatedAt, openAi?, elevenLabs? }`. Read it to learn whether the number is wired up.

Setting up the phone line from code

Everything the ability does is also available to backend code holding the application API key, which is how you provision agents at scale. Setting the connection wires the number up exactly as saving the ability does; the outcome lands in provisioning a few seconds later.

Backend code
import { executable, SquidService } from '@squidcloud/backend';
import { AiAgentTwilioIntegrationOptions, SquidTwilioClient } from '@squidcloud/twilio-client';

export class PhoneLinesService extends SquidService {
  /** Gives the agent a number of the Twilio account, answered by an OpenAI realtime voice. */
  @executable()
  async assignPhoneLine(agentId: string, phoneNumber: string): Promise<AiAgentTwilioIntegrationOptions> {
    this.assertIsAuthenticated();
    const line: AiAgentTwilioIntegrationOptions = {
      phoneNumber,
      voiceEngine: 'openai',
      openAiVoiceIntegrationId: 'openai_voice',
      voice: 'marin',
      language: 'en-US',
      welcomeGreeting: 'Thanks for calling Bright Smile Dental. How can I help you today?',
      // The agent composes the opening line itself, so a known caller is greeted by name.
      greetingMode: 'agent',
    };
    // `connectedIntegrations` is set as a whole: keep the agent's other connections and replace its Twilio one.
    const agent = await this.squid.ai().agent(agentId).get();
    const others = (agent?.options.connectedIntegrations ?? []).filter((item) => item.integrationId !== 'twilio');
    await this.squid
      .ai()
      .agent(agentId)
      .setAgentOptionInPath('connectedIntegrations', [...others, { integrationId: 'twilio', integrationType: 'twilio', options: line }]);

    // Ask the connector to wire the number up now and report the outcome, instead of polling `provisioning`.
    const twilio = new SquidTwilioClient(this.squid, 'twilio');
    const { provisioning } = await twilio.provisionAgent({ agentId });
    if (provisioning.state === 'error') throw new Error(`The number could not be connected: ${provisioning.message}`);
    return { ...line, provisioning };
  }
}

setAgentOptionInPath('connectedIntegrations', [...]) replaces the agent's whole list of connections, which is why the example carries the other connections over. To change one option of an existing line, address that connection directly: setAgentOptionInPath('connectedIntegrations[integrationId=twilio].options', { ...currentOptions, greetingMode: 'agent' }). That path only updates a connection that exists; it does not add one.

To see which numbers are free, call listPhoneNumbers(): every number of the account comes back with usedByAgentId when an agent already has it.

What happens on a call

However the audio is handled, the agent takes part in the call the way it takes part in a chat:

It knows the call. Every turn carries phone manners (short spoken replies, no markdown, confirm names and numbers by repeating them) and the call facts: the caller's number, the number that was called and the current time. The agent answers in the call's language.
It remembers the call. All turns of one call share one conversation memory, so the agent keeps the context from the first word to the last.
Its tools see the call. Every AI function the agent calls receives agentContext.twilioCallSid and agentContext.twilioIntegrationId, so a function can read the call record, for example to look the caller up by number. See using the call context in AI functions.
It can end and transfer. The Twilio ability adds two in-call AI functions to the agent: endTwilioCall, which speaks a farewell and hangs up, and transferTwilioCall, which speaks an announcement and dials another number, connecting the caller to it with the agent's number as caller ID. Tell the agent in its instructions when to use them, such as "When the caller asks for a person, transfer the call to +1 555 010 0100 and tell them you are connecting them."

The ElevenLabs engine is the exception: ElevenLabs hosts the call, ends it with its own tool, and does not pass the call context to the agent's functions, so transfers are not available there.

When no agent is connected to the called number, the caller hears "Sorry, no assistant is available to take this call right now" and the call ends. A disabled connector answers new calls with "Sorry, this number is not taking calls right now" and stays silent on texts.

Outbound calls

The agent calls out with the placeTwilioCall AI function ("Call +1 555 010 0100 and confirm tomorrow's appointment"), and your backend does the same with placeCall. The call is placed from the agent's number; when it is answered, the agent greets the person and follows the purpose you gave it, which is added to its instructions for that call. Whoever answers can interrupt, ask questions and be transferred like an inbound caller.

Backend code
import { executable, SquidService } from '@squidcloud/backend';
import { SquidTwilioClient } from '@squidcloud/twilio-client';

export class RemindersService extends SquidService {
  @executable()
  async callToConfirm(patientPhone: string, appointmentText: string): Promise<{ callSid: string }> {
    this.assertIsAuthenticated();
    const twilio = new SquidTwilioClient(this.squid, 'twilio');
    return twilio.placeCall({
      agentId: 'front-desk',
      to: patientPhone,
      // What the agent should accomplish once the call is answered.
      instructions: `Confirm the appointment ${appointmentText}. Offer to reschedule if it no longer works.`,
    });
  }
}

instructions apply on the Twilio and OpenAI engines. The ElevenLabs engine places the call through ElevenLabs and ignores them.

SMS

Texts to the agent's number are verified, deduplicated and broadcast on the application's event bus as an SMS_EVENT_TYPE event. By default the connector then answers them itself: it asks the agent, in a conversation kept per sender, and texts the answer back from the agent's number. The agent is told it is texting, so it answers in plain text under 320 characters.

To answer texts your own way, set disableBuiltInSmsReply: true on the phone line and handle the event in your backend:

Backend code
import { eventHandler, SquidService, TriggerEvent } from '@squidcloud/backend';
import { SMS_EVENT_TYPE, SquidTwilioClient, TwilioSmsEvent } from '@squidcloud/twilio-client';

export class TextsService extends SquidService {
  @eventHandler<TwilioSmsEvent>(SMS_EVENT_TYPE)
  async onText(event: TriggerEvent<TwilioSmsEvent>): Promise<void> {
    const { integrationId, agentId, from, to, body } = event.payload;
    // A text to a number no agent has carries no agentId; ignore it or route it yourself.
    if (!agentId) return;
    const answer = await this.squid
      .ai()
      .agent(agentId)
      .ask(body, {
        instructions: 'You are answering an SMS. Plain text only, under 320 characters.',
        memoryOptions: { memoryMode: 'read-write', memoryId: `sms_${from}` },
      });
    const twilio = new SquidTwilioClient(this.squid, integrationId);
    await twilio.sendSms({ to: from, from: to, body: answer });
  }
}

The agent sends texts on its own with the sendTwilioSms AI function, for example a booking confirmation with the address, and your backend sends them with sendSms({ to, body, agentId }), from the agent's number.

Call records and transcripts

The connector keeps two collections in the application's built-in database:

Collection	Document
`twilio_calls`	One `TwilioCallRecord` per call, keyed by the Twilio call SID: `agentId`, `direction`, `from`, `to`, `status` (`in-progress`, `completed`, `busy`, `failed`, `no-answer`, `canceled`, `transferred`), `voiceEngine`, `channel` (`phone` or `web`), `startedAt`, `endedAt`, `durationSeconds`, `instructions`.
`twilio_call_transcripts`	One `TwilioCallTranscriptEntry` per spoken turn: `callSid`, `role` (`user` or `agent`), `text`, `at`.

Read them from backend code with getCallTranscript({ callSid }), which returns the record and its turns in order, or query the collections directly, for instance to list today's calls of an agent. To react while a call is happening, subscribe to the voice session events that the transcript is built from.

The ElevenLabs engine hosts its calls outside Squid; their transcripts are in the ElevenLabs dashboard.

Browser calls through Twilio

A visitor of your web page can call the agent without a phone, through Twilio's browser client. Your backend issues a credential bound to the agent, the browser connects with it, and the call reaches the agent like a call to its number, with channel: 'web' on the call record. This works on the Twilio and OpenAI engines and needs a Twilio API Key on the connector.

Backend code
import { executable, SquidService } from '@squidcloud/backend';
import { CreateTwilioCustomerVoiceTokenResponse, SquidTwilioClient } from '@squidcloud/twilio-client';

export class BrowserCallsService extends SquidService {
  /** A credential for one call with the agent, valid for five minutes. */
  @executable()
  async createBrowserCallToken(): Promise<CreateTwilioCustomerVoiceTokenResponse> {
    this.assertIsAuthenticated();
    const twilio = new SquidTwilioClient(this.squid, 'twilio');
    // webCallerKey is echoed on the call record and in the agent's function context, so you can tell who called.
    return twilio.createCustomerVoiceToken({ agentId: 'front-desk', webCallerKey: this.getUserAuth()?.userId ?? '' });
  }
}

Client code
import { Device } from '@twilio/voice-sdk';

const { token } = await squid.executeFunction('createBrowserCallToken');
const device = new Device(token);
const call = await device.connect();
call.on('disconnect', () => console.log('The call ended'));

The credential allows one outgoing call and nothing else, and expires after five minutes if unused. getCustomerVoiceCall({ identity }) tells your backend which call SID a credential was used for, and the call's status.

tip

On the OpenAI engine, a browser can also call the agent over WebRTC without any Twilio involvement, with squid.ai().voice().startWebCall(). Choose the Twilio route when you want browser calls recorded and routed exactly like phone calls, or handed to a human.

Handing a call to a human

Your application can take a live call over, for example to join the caller to a conference with a staff member, without the agent hanging up on the way out. setHumanHandoff records who owns the call:

Backend code
import { SquidTwilioClient } from '@squidcloud/twilio-client';

const twilio = new SquidTwilioClient(this.squid, 'twilio');

// 1. Claim the call: from now on Squid routes it to your handoff URL instead of ending it.
await twilio.setHumanHandoff({
  callSid,
  handoffId: ticketId,
  action: 'claim',
  // A webhook of yours that answers with the TwiML joining the caller to the human, e.g. <Dial><Conference>.
  handoffUrl: this.squid.getWebhookUrl('joinConference') + `?ticket=${ticketId}`,
});

// 2. Once the human is on the line, confirm; a confirmed claim can no longer be released.
await twilio.setHumanHandoff({ callSid, handoffId: ticketId, action: 'confirm' });

// Or give the call back to the agent while the human has not joined yet.
await twilio.setHumanHandoff({ callSid, handoffId: ticketId, action: 'release' });

A claimed call behaves as follows:

The agent's endTwilioCall and transferTwilioCall functions refuse ("A human handoff owns this call"), and the end of the agent's voice session no longer hangs the caller up.
The next callback Twilio makes for the call is answered with a redirect to your handoffUrl. On the OpenAI engine that happens as soon as the agent ends its side of the call, so a function of yours can claim the call and then let the agent say goodbye. On the Twilio engine, move the call yourself with Twilio's REST API (calls(callSid).update({ url: handoffUrl })) after claiming it.
handoffUrl must be https:// (or http://localhost in development) and carry no credentials. A second claim with a different handoffId or URL is rejected, as is a claim on a call that already ended.

webCallerKey on the call record identifies a browser caller across the handoff, so your conference page can show the human who is waiting.

Using the Twilio connector from code

Install the client package to call the connector's operations with typed methods. Every method needs the application API key, so call them from backend code such as executables.

npm install @squidcloud/twilio-client

Backend code
import { SquidTwilioClient } from '@squidcloud/twilio-client';

const twilio = new SquidTwilioClient(this.squid, 'twilio'); // your connector id

Method	Executable name	Description
`sendSms({ to, body, agentId?, from? })`	`sendSms`	Sends a text from the agent's number (the first connected agent's without `agentId`).
`placeCall({ to, agentId, from?, instructions? })`	`placeCall`	Places a call the agent talks on once it is answered.
`getCallTranscript({ callSid })`	`getCallTranscript`	The call record and its transcript in order.
`listPhoneNumbers()`	`twilioListPhoneNumbers`	The account's numbers, each with the agent it already rings.
`provisionAgent({ agentId })`	`twilioProvisionAgent`	Wires the agent's number up again and returns the outcome.
`createCustomerVoiceToken({ agentId, webCallerKey })`	`twilioCreateCustomerVoiceToken`	A Twilio Voice SDK credential for one browser call with the agent.
`getCustomerVoiceCall({ identity })`	`twilioGetCustomerVoiceCall`	The call a browser credential was used for.
`setHumanHandoff({ callSid, handoffId, action?, handoffUrl? })`	`twilioSetHumanHandoff`	Claims, confirms or releases human ownership of a live call.

The executable names are what squid.executeFunction() takes when you prefer not to install the package; pass integrationId in the request then.

In Agent Studio and through ask(), the agent has these AI functions on any Twilio-connected agent: sendTwilioSms, placeTwilioCall, endTwilioCall and transferTwilioCall.

Troubleshooting

Symptom	Cause and fix
The ability says "Choose a phone number for this agent"	The line has no `phoneNumber`. Pick one and save again.
"… is not a phone number of Twilio account …"	The number is not on the account the connector holds, or is written differently. Use the E.164 form Twilio shows, such as `+15551234567`.
"… already rings agent …"	Another agent has the number. A number rings one agent only; free it there or pick another.
"Choose the OpenAI Voice integration …" or "… ElevenLabs integration …"	The engine needs its connector: set `openAiVoiceIntegrationId` or `elevenLabsIntegrationId` (the console picks the first one for you).
Calls are answered with "no assistant is available"	The called number is not connected to an agent, or its setup failed. Open the ability and check its status line.
Twilio's debugger reports an `INVALID_TWILIO_SIGNATURE` error from the webhook	The URL Twilio requests is not the URL the connector expects, typically a proxy or tunnel in front of the backend. Set Webhook Base URL to the public URL and save.
The call connects but nothing is said, then it drops (Twilio speech, real-time mode)	ConversationRelay is not enabled on the account. Accept the AI/ML addendum in the Twilio Console (see before you begin), or switch the line to the turn-based mode.
Silence for a few seconds after pickup	Expected with the Opening line set to "composed by the agent": the agent runs a turn before speaking. Use the fixed greeting for an instant answer.
Texts are not delivered to US numbers	The number is not registered for A2P 10DLC. Twilio's Messaging logs show the error code.
`CUSTOMER_VOICE_REQUIRES_TWILIO_API_KEY` on `createCustomerVoiceToken`	Browser calls mint Twilio access tokens, which need an API Key SID and secret on the connector.
`CALL_ALREADY_ENDED` or `CALL_HAS_ANOTHER_HANDOFF` on `setHumanHandoff`	The call ended before the claim, or another `handoffId` already owns it. Read the call record first.

Give an AI agent a phone number: it answers calls, places calls and texts on it​

The Twilio connector's functionality​

Before you begin​

Adding the Twilio connector to your Squid application​

Connecting an agent to a phone number​

Phone line settings​

Setting up the phone line from code​

What happens on a call​

Outbound calls​

SMS​

Call records and transcripts​

Browser calls through Twilio​

Handing a call to a human​

Using the Twilio connector from code​

Troubleshooting​