How to Develop an Enterprise GPT Strategy

So your CEO has been testing ChatGPT, and now they want you to develop a large language model strategy for your company. Here are two approaches other companies have used to get started.

The Big Strategy Approach

The big strategy approach is necessary when you're going to approach your board or CEO and ask for a significant budget to begin implementing an AI transformation strategy. This involves developing a comprehensive plan, including program design, roadmaps, timelines, and organizational structure. The foundation of this strategy is identifying business use cases within your organization that will yield significant business value. Usually, the best way to begin this process is by talking to various business units about how they work today and searching for opportunities to apply large language model technologies to improve business processes. This works a lot like any early-stage customer discovery interview process, a part of product management best practices for the past fifteen or twenty years.

However, your approach to the interviews will make or break your success. The teams conducting the interviews need to ask questions in a way that elicits the kind of responses your product strategy team needs to develop a plan. Your goal isn't to develop a pile of disconnected business use cases - the use cases must map back to scalable capabilities and software functions. Example capabilities are information extraction, text summarization, document Q&A, or chat.

The worst possible outcome is a pile of generic buzzwords like "predictive analytics." Unfortunately, I've had to redo AI strategies for large companies who spent a lot of money on an AI strategy only to get a beautiful slide deck of generic drivel.

Quick Proof of Concept (POC) Approach

This approach involves executing a short-term project, ideally within four months, to demonstrate the feasibility of large language models in your organization. The goal is to gain organizational momentum and familiarity with the technology, which will be helpful if you later pursue a more extensive strategy. The challenge is selecting the right POC because you have so many options.

Selecting the best POC

To select the best use case for a quick POC, consider the following guidelines:

1. Select a use case with motivated business partners, as their responsiveness and cooperation are essential for success. This is probably the most important decision you can make.
2. Choose a model architecture that aligns with your organization's security and policy constraints. Currently, GPT-4 is the most advanced model, but if that's not feasible, consider BERT or other NLP transformers. It may seem odd to start by considering a model instead of the business use case, but the limitations and customization challenges of the model architecture will dictate the best place to start.
3. Pick a business process where you have access to data with a low sensitivity level, avoiding policy or security concerns.
4. Aim for a use case where you can generate training data programmatically, as high-quality, task-specific training data is critical for NLP models. Avoid use cases where lots of end-user engagement is necessary, such as user-driven prompt engineering.

Which approach is best for you? On balance, I've seen more success from starting with a POC before doing a bigger strategy engagement. Starting with a POC gives the organization a baseline understanding of the technology and challenges in building it.