Skip to main content

Elasticsearch

Squid can connect to an Elasticsearch cluster to serve as a data source for your Squid application.​

With the Elasticsearch connector, you can query your indices through the Squid Client SDK, run native Elasticsearch queries, and let AI agents answer questions using your data.

note

The Elasticsearch connector is read-only. Search, count, aggregations, and other read operations are supported; document writes, _bulk, _update_by_query, and _delete_by_query are rejected.

Elasticsearch-side prerequisites​

  • A reachable node URL including the scheme and port, such as https://your-elasticsearch-host:9200.
  • Optionally, a user with read privileges on the indices you want to expose. The connector authenticates with HTTP basic auth (username + password); if omitted, it connects unauthenticated.
  • For clusters on a private network, you can route the connection through a Squidlet, a proxy that runs on your machine and lets Squid access local resources.

Adding the Elasticsearch connector​

  1. Open the Squid Console and select your application to use with Elasticsearch.
  2. Click the Connectors tab, and then click Available connectors.
  3. Select the Elasticsearch connector.
  4. Provide the following configuration information:
  • Connector ID - Choose an ID that is brief and helps identify the connector.
  • Database Endpoint - The cluster URL. For example, https://your-elasticsearch-host:9200.
  • Database Username - Optional. A user with read access.
  • Database Password - Optional. To keep it private, store the password in Squid Secrets.
  • Squidlet ID - Optional. The ID of a Squidlet proxy for reaching private clusters.
  • Tables to Process - Optional. A comma-separated list of indices to include in schema discovery. Use forward slashes to specify a regex pattern (e.g. /^logs-.*/).

Discovering indices​

After adding the connector, use the Schema tab to discover your indices and data streams. Squid reads each index's mapping to build the schema, and collapses numbered index series (like time-based indices) into glob patterns. System indices are excluded.

Querying your data​

Query indices from the Client SDK the same way as any other Squid collection, passing the connector ID:

Client code
const articles = await squid
  .collection<Article>('articles', 'YOUR_ELASTICSEARCH_CONNECTOR_ID')
  .query()
  .like('title', '%observability%')
  .snapshot();

For full access to the Elasticsearch query DSL, run a native query with executeNativeElasticQuery, passing the index, the request body, and optionally the endpoint and HTTP method (defaults: _search and GET):

Client code
const result = await squid.executeNativeElasticQuery(
  'YOUR_ELASTICSEARCH_CONNECTOR_ID',
  'articles',
  {
    query: { match: { title: 'observability' } },
    size: 10,
  },
);

Native queries are restricted to read-only endpoints such as _search, _count, _msearch, _mget, _mapping, _sql, and _eql. Requests to any mutating endpoint return an error.

A few Elasticsearch-specific behaviors to be aware of:

  • When no limit is set, standard queries return at most 1000 results.
  • Search requests that don't specify their own timeout get a 30-second server-side timeout, so runaway queries are canceled by the cluster.
  • Each connector allows up to 15 concurrent native queries; requests beyond that are rejected rather than queued.

Securing your database​

By default, security rules are required, and in a production environment, it's essential to never allow access to the connector without them. Learn more about securing your database data in the security rules documentation.

To learn how to work with your data in Squid, view the Client SDK documentation.