AI CoPilot vs Pilot
AI CoPilot vs Pilot
How do you think about using AI to enhance your product experience?
Welcome to all the new subscribers that have joined us over the last few weeks! If you haven’t already, you should check out my last article here:
If you like what you read, please subscribe to Superfluid here:
In every job, some tasks are just a chore to get done. They're boring, tedious, and should theoretically be automated, but for whatever reason, haven't been.
Over the last few months, every time I come across a task that fits that description, I've turned to ChatGPT. I start every prompt off with "Please" and end it with a "Thank you" in the hopes that I'll get a response that gives me a complete solution and that when AI does take over, I'll be shown some mercy for being nice.
And in almost all instances, the output is never 100% what I want. And so starts the iterative cycle where I'll make minor edits to my prompts, including random caveats and triggers that I've learned from the prompt engineering gurus on Twitter.
At times, it's so frustrating that I'll just bite the bullet and start from scratch.
This is a pretty familiar experience for me.
Dealing with crypto dApps and interfaces is a similar challenge at times. They can be unintuitive and often require a steep learning curve for new users. While AI isn't as bad, in its current form, there is a learning curve associated with making it do the things you want it to do.
In contemporary society, this just isn't good enough. When we place comfort and convenience above all else, it's a stretch to imagine everyone becoming skilled prompt engineers, but it's also a stretch to imagine AI replacing everyone in its current form.
Generative AI is only really good at a few things. Typically, these tend to be tasks rooted in logic or that follow a rules-based system, e.g., writing or editing code, interpreting legal contracts, or answering questions from a set of documents. It makes sense for them to be co-pilots rather than pilots.
However, a large majority of current off-the-shelf generative AI tools are focused on quantity rather than quality. Understandably, being so early in the lifecycle of these tools means that some level of iteration is necessary, but at the moment, there is a vast ocean of possible iterations which inherently distract users from their core goal.
To an extent, consumers will be forgiving of this; however, when thinking about how this actually infiltrates a professional B2B environment, nearly good enough is not enough. Moreover, in a B2B context, it's all about workflows, data provisioning, and integrations.
As such, if we think about the needs of a B2B user versus a consumer, they are wildly different, yet many generative AI products aim to cater to both.
So, as a founder, how should you be thinking about building with AI?
I believe there are three principles that founders need to consider:
Niche Focus is Key
Historically, new technology has led to the development of broad-based tools with progressive specialization over time. However, this doesn’t ring true with generative AI. In this case, there is a strong necessity to start specialized and stay niche to compound both product and distribution benefits over time.
AI has undoubtedly opened up a world of possibilities for founders, but as I spoke about above, mindless integration doesn't really add much value. True value is really created when problems are clearly defined, and the power of AI is tightly focused.
Given their existing distribution prowess and access to data, general AI applications are best left to incumbents such as Microsoft and Google. Instead, emerging AI-centric startups should look to carve out a niche where they can achieve outcomes that are far superior to what incumbents can achieve.
My recommendation here is to niche down at least two layers. For example, the first layer could be sales enablement tools, and the second layer could be focusing on nurturing customer relationships. The more layers you go down, the closer you get to solving problems from a first-principles level.
Taking a Goal-Oriented Approach
At the moment, generative AI products are largely focused on creating outputs and a lot of them. Using Leonardo AI, I can create four images for every prompt I give it. However, after I get four images back, I need to sift through them and actually determine which one is best or start the process again. Sure, the output is slightly better than if I used the bare-bones Stable Diffusion model, yet it still isn't good enough.
Optimizing for outputs is fine for consumers who might find it a novelty; however, for a B2B use case, these are just a waste of time for workers who are likely under pressure to deliver. Instead, it makes more sense to actually think smaller when considering how to integrate generative AI into a business. By using the "Jobs to be done" framework, you can break down individual jobs and think clearly about which parts of the job make sense to be completed by AI and which don't.
From this, you can start thinking about the goals that need to be met or achieved through using your product and work backward from there. There is a strong likelihood that high-value problems that need to be solved will take more than just a simple generative AI generator. Pair this with a niche focus, and you'll have a killer product on your hands.
Take Intercom, a customer service bot SaaS business, which has had a history of testing and implementing AI within its products since 2018. More recently, they've built out a solution called Fin, which ingests a business's support content and uses GPT-4 to parse through this and answer support queries on behalf of a customer support agent.
However, in the early days of testing generative AI, they focused on a risk-free implementation method, meaning testing out the powers of LLMs in an area of the business where it wouldn't have too much of a negative impact if things went wrong. By focusing on solving problems where expectations were low, then if it was useful at all, customers would be happy.
In the case of Intercom, this was through summarizing conversations that customer support agents were having through the Intercom product. This wasn't a crucial feature for people, so if things went wrong, the downside was minimized. Moreover, this served as a testing ground for Intercom as they built in guardrails for the AI to stop hallucinating, which is particularly important in the context of answering customer support inquiries.
By taking a risk-averse, goal-oriented approach to embed AI within their product, Intercom was able to resolve some of their customer's low-hanging fruit problems while also learning more about the way LLMs function and how they can build better products that actually accomplish their customer's goals rather than give them outputs that are low quality.
Performance is Tightly Correlated with Data Capture
The guiding principle in data collection is that Garbage in = Garbage out, and this is even more true when building with AI. The performance of your product will be tightly correlated to the type of data you capture, the context in which you capture that data, and how you use that data to inform further product development and improvements.
When quizzing founders over their data capture practices, I typically find that they either aren't capturing any data or they're capturing anything and everything they can. There doesn't seem to be an in-between state. Obviously, not capturing any data is a missed opportunity, but capturing too much data can actually cause you to overlook interesting patterns or actions being conducted in your product.
Instead, I'd urge you to really think about the inputs and actions your customers take that lead to needle-moving outcomes. This is the data that absolutely needs to be captured. By being targeted but mindful of the data you capture, you'll be able to avoid spurious correlations, bias, and potential model degradation, which will all affect the performance of your product over time.
Like the Intercom example above, think about how you can capture or use data that will help inform productive improvements of your product and how that data can better inform you of new problems that your customers are facing.
Far too many founders right now are throwing everything they’ve built to pivot their product to incorporate AI in some way. For a minority of them, this tactic will work, but for the majority, this will just be a futile hype-chasing exercise. Instead, I would urge founders to take a first-principles approach towards deciding if AI will incrementally improve your customers’ experience, and whether you can solve problems to a high standard.
From my perspective, in its current form, it’s clear that AI should be used as a co-pilot and that changes how it is integrated into a product. Perhaps with GPT-5 or 6, there is a visible pathway for AI to become a self-sufficient pilot, but until then, a solid and reliable co-pilot is 100% greater than a struggling pilot.
Make sure to subscribe now to not miss the next article.
How did you like today’s article? Your feedback helps me make this amazing.
Thanks for reading and see you next time!
Abhi