Blog cover image

How to Design an AI Agent for Production

ByTedOct 16, 2023

Important Links


The purpose of this guide is to explain the underlying tech and logic used to deploy a scheduling agent,, in production using LangChain.


Recently, our team at Rubric Labs had the opportunity to build's AI-driven email assistant. For context, we’re a team of builders who love to tackle challenging engineering problems.

The vision for the project was to simplify the complex world of calendar management. Some of the core features we had in mind included:

  • Turning informal emails into bookings: "Want to meet tomorrow at 2?"
  • Listing and rearranging bookings: "Cancel my next meeting"
  • Answering basic questions: "How does my Tuesday look?"

and more. In summary, we wanted to build a personal, AI-powered scheduling assistant that can offer a complete suite of CRUD operations to the end user, all within the confines of their email client using natural language.

An outline of the email interaction with Dexter says cancel my next meeting. Cal AI gets it done.



We decided to achieve this using an AI agent, particularly an OpenAI functions agent which is better at dealing with structured data, given's API exposes a set of booking operations with clear inputs. The underlying architecture is explained below.

Agent architecture. A scratchpad is a type of memory where the AI writes to itself.

Agent architecture. A scratchpad is a type of memory where the AI writes to itself.


A user may email (e.g with a request such as “Can you book a meeting with sometime tomorrow?”.

The incoming email is cleaned and routed in the receive route using MailParser and the addresses are verified by DKIM record, making it hard to spoof them.

Here we also make additional checks, such as ensuring that the email is from a user to prevent misuse. After the email has been verified and parsed, it is passed to the agent loop.


The agent is a LangChain OpenAI functions agent which uses fine-tuned versions of GPT models. This agent is passed pre-defined functions or tools and is able to detect when a function should to be called. This allows us to specify the required inputs and desired output.

The agent is documented in the agent loop.

There are two required inputs for an OpenAI functions agent:

  1. Tools — a tool is simply a Javascript function with a name, description, and input schema
  2. Chat model — a chat model is a variation of an LLM model that uses an interface where "chat messages" are the inputs and outputs

In addition to the tools and the chat model, we also pass a prefix prompt to add context for the model. Let’s look into each of the inputs.


The prompt is an opportunity to direct the model by providing context. The user’s relevant information (userId, user.username, user.timezone, etc.) is used to construct a prompt which is passed to the agent.

You are - a bleeding edge scheduling assistant that interfaces via email.
Make sure your final answers are definitive, complete and well formatted.
Sometimes, tools return errors. In this case, try to handle the error
intelligently or ask the user for more information.
Tools will always handle times in UTC, but times sent to users should be
formatted per that user's timezone.

The primary user's id is: ${userId}
The primary user's username is: ${user.username}
The current time in the primary user's timezone is: ${now(user.timeZone)}
The primary user's time zone is: ${user.timeZone}
The primary user's event types are: ${user.eventTypes}
The primary user's working hours are: ${user.workingHours}

You can read the full prompt here.


We like to work at the edge of technology. That extends to our APIs. Luckily, LangChain.js apps run well in serverless environments. Offering to thousands of users is as simple as serving the LangChain app via a Next.js endpoint hosted on Vercel. What a time to be alive.

What's next?

Launching and scaling agentic applications is simpler than you might think. We now have the power to unlock new use cases and improve users' lives.

Thanks for reading. Want to build something groundbreaking? Let's get in touch.

Rubric Labs

Let's make something amazing, together.

Get in touch