Twilio Integration Guide

Integrating a Conversational AI agent with Twilio allows you to create seamless, human-like voice interactions. This guide will walk you through the process of setting up your environment, configuring your agent, and implementing the integration using JavaScript.

Prerequisites

Before you begin, ensure you have the following:

• An ElevenLabs account
• A configured ElevenLabs Conversational Agent
• A Twilio account with an active phone number
• Node.js 16+ installed
• ngrok for local development

Agent Configuration

To integrate with Twilio, your agent must be configured to use the correct audio format supported by Twilio.

1. Configure TTS Output

   • Navigate to your agent settings.
   • Go to the Voice Section.
   • Select “μ-law 8000 Hz” from the dropdown.

2. Set Input Format

   • Navigate to your agent settings.
   • Go to the Advanced Section.
   • Select “μ-law 8000 Hz” for the input format.

Implementation

Initialize the Project

First, set up a new Node.js project:

mkdir conversational-ai-twilio
cd conversational-ai-twilio
npm init -y
npm pkg set type="module"

Install Dependencies

Next, install the required dependencies for the project.

npm install @fastify/formbody @fastify/websocket dotenv fastify ws

Create Project Files

Create a .env and index.js file with the following code:

# .env
ELEVENLABS_AGENT_ID=your_agent_id

// index.js
import Fastify from 'fastify';
import websocketPlugin from '@fastify/websocket';
import dotenv from 'dotenv';
 
dotenv.config();
 
const fastify = Fastify();
fastify.register(websocketPlugin);
 
fastify.get('/', { websocket: true }, (connection, req) => {
  connection.socket.on('message', message => {
    console.log('Received:', message);
    connection.socket.send('Hello from server');
  });
});
 
fastify.listen(8000, err => {
  if (err) {
    console.error(err);
    process.exit(1);
  }
  console.log('Server listening on port 8000');
});

Run the Server

You can now run the server with the following command:

node index.js

If the server starts successfully, you should see the message [Server] Listening on port 8000 in your terminal.

Twilio Setup

Create a Public URL

Use ngrok to make your local server accessible:

ngrok http 8000

Configure Twilio

1. Go to the Twilio Console.
2. Navigate to Phone Numbers → Manage → Active numbers.
3. Select your phone number.
4. Under “Voice Configuration”, set the webhook for incoming calls to: https://your-ngrok-url.ngrok.app/incoming-call-eleven
5. Set the HTTP method to POST.

Testing

Call your Twilio phone number. Start speaking - you’ll see the transcripts in the ElevenLabs console.

Troubleshooting

Connection Issues

• Verify your ngrok URL is correct in Twilio settings.
• Check that your server is running and accessible.
• Ensure your firewall isn’t blocking WebSocket connections.

Audio Problems

• Confirm your ElevenLabs API key is valid.
• Verify the AGENT_ID is correct.
• Check audio format settings match Twilio’s requirements (μ-law 8kHz).

Security Best Practices

• Use environment variables for sensitive information.
• Implement proper authentication for your endpoints.
• Use HTTPS for all communications.
• Regularly rotate API keys.
• Monitor usage to prevent abuse.

Reference: ElevenLabs by ElevenLabs Team