This week we’re releasing our second video on Performance-Driven Development, or PDD—a new methodology for building systems powered by large language models. Today’s topic is GenAI application architecture.

Many teams start by integrating LLMs into their existing software libraries. It’s a quick and easy way to get going, since you can make API calls to LLM providers like OpenAI with just a few lines of code. While this approach works for simple tasks, tightly integrating LLMs into your existing code creates scaling challenges.

A better approach is to isolate your AI solutions from the rest of your infrastructure using interfaces. The AI solution consists of components that optimize the LLM’s performance, such as prompts, data preprocessing, prompt orchestrations, and tools.

This design offers several advantages which I cover in this video.