1 Year Since ChatGPT Launch

It's been almost 1 year since ChatGPT first launched for public use. The world was taken by storm by the possibilities of what Large Language Models (LLMs) could do to every industry.

Tech giants like Microsoft, Google, and Amazon as well as SaaS players like Salesforce, Freshworks, and Hubspot refocused their attention on using these LLMs to improve customer experience and enhance internal productivity.

We have been closely following the updates from industry leaders and engaging with 200+ product and technology leaders to understand their motivation, approach, and vision for integrating Generative AI applications into their products and workflows.

Here, we attempt to summarize our learnings over the past year:-

Two Decades of Enterprise Software

Over the past few decades, technology-first teams have built their software for a range of customer personas, use cases, and capabilities. Naturally, these have become complex and simply navigating through the product to get the job done has become difficult.-

How exactly does a user use such a SaaS product?

Step 1: Deciding Objectives & Intent

Depending on the user's KPIs and the product's capabilities, the user starts using the product with a particular intent

eg. A marketer running email campaigns considers “Email Conversions” as a KPI and uses Hubspot to create, track, and analyze the performance. The intent here is to increase the KPI from x% to y%

Step 2: Analysing Data from Multiple Sources to Reach a Decision

To achieve the intent, a number of questions need to be answered. These answers come from the data present within the product + data outside the product + subjective input from the user and other stakeholders.

eg. The marketer will understand how previous campaigns have performed, what the levers of improving performance are (content, creative, CTA, positioning, audience, time of delivery, etc.), external factors (holiday season, macroeconomic conditions, etc.), and subjective input about the priorities of the organization.

Based on this analysis which consists of fetching and analyzing multiple data points, adding company-specific and industry-specific context, and approval from stakeholders, the marketer decides on an action: Change the CTA for 18-25-year-olds to “Sign Up Now”

Step 3: Executing the Action on the Product

Once the actionable is clear, the user executes the action on the SaaS product as well as on 3rd party tools

eg. changing the creative with updated CTA, taking approvals for the stakeholders, creating a new campaign on Hubspot, ensuring all technical details are correct and functional, etc.

What's wrong with this process?

1. Can take weeks to go from intent to execution

• Simply analyzing past performance to understand which levers affect a metric the most for a cohort can take days
• Most softwares provide a standard dashboard that may not cover the data that the user is looking for. Even with dashboards available, important information gets lost in a sea of charts

2. Subjective Decision-Making

• Key patterns and trends can get lost when the intelligence is highly dependent on the user
• Human bias gets incorporated into the decision resulting in sub-par output

3. Opportunity Cost

• Users of the software are important resources to any organization. If their time is largely consumed in operating the product, the organization is losing out on high-value tasks that the user could have completed. This is ultimately a loss in efficiency and revenue opportunities

Entering the era of AI Copilots

Generative AI has given unique capabilities to the world - ones that allow intelligent reasoning, objective decision-making, and contextual assistance to every user.

Technology leaders like Microsoft are revamping their entire product suite with AI Copilots that understand your business, your objectives and provide assistance to help you complete tasks in the fastest way possible.

Here is how the industry leaders are reinventing software with LLMs:

1) Microsoft 365 Copilot

Microsoft, on the shoulders of their tightly-knit partnership with OpenAI, was the first to get into the Copilot mix by announcing the revamp of the entire Microsoft 365 suite including Word, Excel, PowerPoint, Outlook, etc. They believe strongly in the vision of empowering humans to do creative work while the Copilot takes care of redundant tasks like creating summaries or presentations, doing operations on data, and never having to start a document from scratch.

They have a strong thesis about creating a 25% upsell ($10B) on their existing revenue base of $40B/year over the next 3 years which opens up our mind about the possibilities of this Copilot.

2) Salesforce Einstein Copilot

Salesforce is going a step beyond by providing industry-specific Copilots that bring the ability to answer customer questions with sufficient context of their sector. It has also focused on a developer-friendly approach by opening up Copilot Studio - which allows developers to configure and train their own AI assistants by choosing their own models, writing custom prompts, and connecting with external data sources via APIs

3) Shopify Sidekick

Shopify's focus on helping e-commerce entrepreneurs has intensified through the Sidekick which promises the ability to help an operator with everything that they need to run their business online. In an industry that is almost always low on tech bandwidth, Sidekick will be a game changer by allowing everything from crafting content, adding products on display, applying discounts, or learning more about the change in metrics

The Why Behind Introducing AI Copilots

Beyond industry leaders, almost every SaaS team has expressed interest in exploring the power of LLMs to revamp their product suite. Over the last 6 months, we've spoken to 200+ product and technology leaders in SaaS to understand their motivation, approach, and vision towards building AI Copilots.

Here are 5 main reasons that came up:

1) “A Better Customer Experience Can Lead to Upsell Opportunities”

Customer-focused teams are obsessed with providing customers with the best-in-class experience even without a direct short-term ROI in terms of revenue increase. They believe that delighted customers are likely to stick longer and contribute higher LTVs in the long run.

Leaders believe that better customer experience will translate to an increase in ticket size. Even if Microsoft's estimate of a 25% upsell opportunity is a stretch, by all means, a 5-10% increase in ticket size over the next 3 years is what everyone is aiming for.

2) “This could Improve Internal Productivity”

The AI Copilots launched by SaaS market leaders have forced technology leaders to take a hard look at the redundancies in their internal workflows. There are thousands of hours wasted in every business function where time is taken up in answering ad-hoc queries, maintaining databases, crunching numbers on sheets, or summarizing documents - all of which can be done by AI Copilots. This can enable teams to take up more creative tasks.

3) “Because our Competitors and Industry Leaders are doing it”

In competitive markets, SaaS teams have to constantly keep up with their competitors. If one of the competitors starts offering an AI Copilot, it becomes a hygiene feature to retain customers who are on the edge of switching.

4) “Because it comes with great PR value”

In a world where there is a massive ongoing fight to claim customers' attention, AI Copilots or any Generative AI application serves as a great way to get people talking about your organization.

5) “Our Customers have an AI budget and they are asking for it”

The SaaS teams that serve Fortune 500 Enterprises are receiving suggestions and demands from their customers to experiment with GenAI use cases. They are willing to set aside a budget for these experiments and given that their own technical bandwidth is limited, they expect assistance from the SaaS provider to guide them along the way.

Development Stage of AI Copilots

Even though most teams have publicly announced their intention to serve AI Copilots, many are far from deploying a production-ready application. Given that engineers are still getting used to the new LLM tech and product managers are still figuring out the right use cases, there's a long way to go.

Here is how the market is divided:

1) Use Case Identification and Research (1-3 months)

A large section of companies are currently in this phase. This includes identifying what's broken in the workflow and what's a business priority for the company. The next step is to build an understanding of what Generative AI and LLMs have to offer and which use cases have the best balance between potential and feasibility.

Most companies organize hackathons to get all their minds working on identifying use cases.

2) Exploring/Playground Stage (3-6 months)

Once a set of use cases are identified, the product and engineering team experiments with the tools available at the disposal (eg. OpenAI's GPT APIs, Libraries like LangChain, and No-code LLM App builders) and plays around with it to get an understanding of how it works.

At this stage, a dedicated team/pod is set up to put in more serious effort.

3) Prototype/POC/Beta Stage (6 months - 1 year)

Once the use cases are defined and a team is put in place, a fancy-looking demo for each use case is created. This is when the real work (and trouble) begins and that's also where a large section of the market is stuck.

Most teams are hardly able to push beyond the prototype, given that engineers are already tasked with running the business as usual and have the LLM initiatives in addition.

At this stage, a dedicated AI Product Manager or AI Project Head is brought on board to lead the initiative. An announcement marking the launch of a capability is made.

4) Production-ready LLM application (1 year +)

There are struggles with achieving accuracy, working with unstructured data, controlling LLM hallucination, controlling privacy concerns with OpenAI, maintaining a balance between cost and latency, etc.

Engineering teams (and thus, CTOs and CPOs) realize that they need dedicated bandwidth of engineers to work on these projects and need a different approach for each use case (structured analytics vs content generation vs operations on unstructured data).

At this stage, project leaders have two options - either ask for more time and budget to deliver a production-grade application OR turn to specialized vendors to get assistance and launch in time. Our partners have been quick to see the value in the latter.

5) AI-first Innovation (Unknown)

There are hardly a few teams we've spoken to who are rethinking their products from the ground up using AI and are fortunate to partner with some of them. AI-first innovation would mean questioning each workflow and redesigning it in the best way possible using the GenAI tools at disposal.

There may be a new set of problems that prop up at this stage, that are yet to be fully discovered.