Skip to main content

Use unstructured and structured data in an AI agent

Integrate documents to your AI agent's knowledge base, and then take actions on your data​

What you'll build​

  • An app with an AI agent that uses product manuals as context to answer questions about your appliances, and creates a list of maintenance tasks when requested.

What you'll learn​

  • How to create an AI agent with a custom knowledge base.
  • How to add functions to the AI agent that allow it to take actions when prompted.

Environment setup​

  1. In the Squid Console, switch to the dev environment.

switch environment

  1. Download the ai-home-maintenance code sample using the following command. Replace the placeholders with the values from your Squid application as shown in the console.
    2. npx @squidcloud/cli init-sample ai-home-maintenance --template ai-home-maintenance --appId YOUR_SQUID_APP_ID --apiKey YOUR_SQUID_API_KEY --environmentId dev --squidDeveloperId YOUR_SQUID_DEVELOPER_ID --region YOUR_REGION
  1. Open the project in the IDE of your choice.
  2. Start the app locally by runnning the following command in the project folder:
npm run start
  1. To view the app, navigate to localhost:PORT where PORT is logged in the terminal. The address will likely be http://localhost:5173.

Add the AI agent​

  1. In the Squid Console, navigate to the AI Agents tab.

Each agent is like a different persona or use case, distinguished by its own set of instructions and context.

  1. Under Agents, click the + to add an agent. Provide the following details, and then click Add:
  • Agent ID: maintenance-scheduler
  • Model name: GPT-4o
  1. Click the + next to Instructions, and add the following instructions to the agent:
Respond to queries about appliances. Only use the provided context to generate responses.
  1. Click Add instructions to save.

These instructions tell the AI agent what actions to take.

Add context​

Context tells the agent what knowledge to pull from when answering questions. Adding context allows the agent to provide relevant answers on specific topics that may not be part of the underlying AI model.

  1. Click the Plus sign (+) next to Context to open a new modal where you can add context.

  2. Input Range as the context title.

  3. Select URL as the Context type from the dropdown menu.

  4. Input the following URL as the context:

https://media3.bosch-home.com/Documents/8001203758_A.pdf

  1. Click Add Context to save.

  2. Repeat steps 1 to 5, adding the following links:

Microwave:

https://media3.bosch-home.com/Documents/9001152447_B.pdf

Dishwasher:

https://media3.bosch-home.com/Documents/9001641455_A.pdf

Washer/dryer:

https://products-salsify.geappliances.com/image/upload/s--pdIsKW0u--/izid0w85hg3mbckeyyo1.pdf

Alternatively, you can search online for the manuals for your own appliances and provide them as context. This will allow you to ask questions you might have about them.

Explore your Squid application​

With both servers running, visit http://localhost:5173/ in your web browser. You're now ready to explore the functionalities of your Squid application:

  1. Try asking questions about your appliances. For example, you can ask the following:
How do I clean my cooktop?

The AI agent will respond in the chat with details obtained from the manuals.

  1. Instruct the AI agent to create a list of tasks for maintaining your appliances:
Create a list of maintenance tasks.

After some time, the new tasks will appear as a list on the right of the application. This means the AI agent was able to use the manuals as context to create maintenance tasks, and then write the tasks to the database on your behalf.

  1. Squid's Query with AI feature uses knowledge of your schema to generate database queries based on your prompts. To save your app's auto-discovered schema, navigate to the Squid Console, navigate to the Integrations tab and click the button on built_in_db to open a dropdown menu. Select Schema from the dropdown.

  2. Click Save Schema to save your app's auto-discovered schema.

  3. Check off some of the tasks to mark them as completed, then ask the AI agent the following:

Which maintenance tasks have I completed?

The AI agent can query your database and discover which items have been completed. This means the AI agent is able to generate database queries on the fly!

  1. The AI agent can recognize and respond to more complex instructions. For example, prompt the AI agent with the following:
Look up in the manual to determine how often I should clean my washer-dryer lint filter. Based on that interval, if I run two loads of laundry a day, how often should I clean the filter?

The AI agent can determine how often to clean the lint filter, and then apply that to your question to give a logical response.

View the code​

You've successfully built a full-stack app with an AI agent powered by Squid! Curious what's going on under the hood? In this section, we'll dive into the code.

Configuring the AI agent on the frontend​

Using the Squid React SDK's useAiChatbot hook, you can connect to your AI agent with the following code:

frontend/src/components/HomeAI.tsx
const { history, chat, complete } = useAiChatbot(
  'ai_agents',
  'maintenance-scheduler'
);

Adding functionality to the AI agent​

A Squid AI Agent can take different actions based on prompts. This is achieved by designating a function with the @aiFunction decorator in the Squid backend, and then passing the function with your prompt as part of the options parameter.

For example, the following AI function creates a list of maintenance tasks created from the appliance manuals, and then writes the tasks to the built-in database using the logMaintenanceScheduleInternal() function:

backend/src/service/example-service.ts
  @aiFunction(
    'If someone asks for a list of maintenance tasks, run this function. This function creates a maintenance task for a list of tasks. Pass each created task to this function. If the user asks how to complete a specific task, then do not run this function. !Important: if a user asks questions about **existing** tasks in the database, then do not run this function.',
    [
      {
        name: 'task',
        description: 'the task to complete',
        type: 'string',
        required: true,
      },
      {
        name: 'interval',
        description: 'the frequency or interval at which the task needs to be done',
        type: 'string',
        required: true,
      },
      {
        name: 'appliance',
        description:
          'the appliance that requires the given maintenance. Provide the manufacturer name of the appliance, along with the type of appliance',
        type: 'string',
        required: true,
      },
    ],
  )
  async createTaskItem(params: { task: string; interval: string; appliance: string }) {
    const { task, interval, appliance } = params;
    await this.logMaintenanceScheduleInternal({ task, interval, appliance, completed: false });
  }

Notice that the @aiFunction decorator includes a parameter where you describe the purpose of the function. The AI agent uses this information to determine when to call the function.

A second AI function queries the database for information based on the prompt:

backend/src/service/example-service.ts
  @aiFunction(
    " !Important: Call this function when a client prompt requests information about the **existing** tasks in their database. This function queries the database for tasks that meet the client's request. Include the exected query in your response",
    [
      {
        name: 'prompt',
        description: 'the request from the client',
        type: 'string',
        required: true,
      },
    ],
  )
  async checkTasksAI(params: { prompt: string }): Promise<string> {
    const { prompt } = params;
    const response = await this.squid.ai().executeAiQuery('built_in_db', prompt);
    return `${response.answer} \n ${response.executedQuery}`;
  }

In the frontend, the functions are passed with each prompt using the following code:

frontend/src/components/HomeAI.tsx
chat(question, { functions: ['checkTasksAI', 'createTaskItem'] }); // Add functions in the second parameter

The AI agent then determines when to run a function depending on your prompt.

For more information on AI functions, view the AI functions documentation

Next steps​

Congratulations! You added an AI agent to an application that can read documents an answer questions, write to a database based on the content of the documents, and read from the database based on the client prompt.