AI Infrastructure vs Apps

If theres one thing that I want to do better at in my life it’s to be more consistent with my routine and output.

So in that vein I decided to try and create a habit of typing out what ever is on my mind in 30 minutes or less a few times per week. Taking note of what Nat Eliason said on X recently, no editing until the end and see where we end up.

Naturally, we’re going to be talking about AI.

Over the last 12 months we’ve gone through one of the most transformational changes in technology in a a very long time. No longer than a few hours go by that something new comes out on X that will without doubt effect multiple industries, thousands of jobs both positively and negatively.

For precisely this reason, there has also been a lot of fear at the capabilities of what AI can do and which jobs it will replace.

But as someone who is default techno-optimistic my feeling is that artificial intelligence will create a lot more abundance that people may think. I will speak about this in more depth one day but to try and sum in up in once short sentence, there is a lot of human hours going into performing tasks that are better designed to be done by a computer.

More on exactly where you should be focussing your efforts soon.

Now as we move into 2024 it’s clear to me that as the angle of the expontential technology curve got even steeper through this year, the gap between what terminally online people on Twitter know about and what the general public know about is vastly different.

This is both down to the speed of advancement and also the focus on the infrastructure layer with the app layer lagging behind. I’ll explain this a bit more.

OpenAI came out with a bang with ChatGPT and became the faster growing app in terms of users of all time surpassing 100m users in just 2 months. In the coming months after this happened we saw billions of dollars in venture capital flow into companies building powerful models dubbed to power the next version of the internet. Some examples of this would be Anthropic and Mistral.

Keeping to the same pace, the shipping machine that is OpenAI then release new APIs to allow other people to build apps on top of what they have built. We’ve now entered the season of the wrapper.

The ongoing joke here is that if you’re building on top of an OpenAI API like GPT4 for instance you are just a wrapper and have no moat as people can just use ChatGPT. and to some degree this is right. What this lead to huge amounts of apps with terrible UIs that let people enter a query and get a result back. It felt like magic to the user for the first few times but churn for AI apps is huge and due to shifting expectations of the human species in general the novelty soon wore off.

OpenAI and other companies are still shipping changes art lightening speed but the app layer is still lagging behind.

There is still one more reason why I think this happened. SMEs and Enterprise businesses see the value in using LLMs to handle things like customer support but if they just went ahead and replaced there customer support rep with a GPT3.5 endpoint then all hell would break lose and the customer would likely have a bad experience.

So how would a company actually start to use LLMs to enhance their customer support function? 

Firstly they would have to build a robust knowledge base of their business. This should include FAQ’s, internal documents, website materials, knowledge about privacy policies and any other proprietary information that reps would use to make sure that support requests we’re handled and closed effectively.

That’s the first step.

Next who ever is in charge of building the implementation would have to ensure through good system prompting that edge cases were handled and the Assistant would respond in a way that no only aligned with the business but also that it would vear outside of these guidelines.

There are other things to consider too like prompt injections and hallucinations. Prompt Injections are where bad actors can use creative prompting to the LLM to extract sensitive information that is trained on (this is something like ChatGPT is vulnerable too).

Hallucinations is also a topic for another day but tldr is that LLMs are designed to hallucinate to some degree through a parameter called Temperature but in the case of recalling set information to handle a business customer, this isn’t always a good thing.

That was a long tangent into just a couple of things a business needs to think about when implementing an LLM for customer support and is one of the reasons why the app layer is lagging behind.

Side note - this is also one of the reasons why new startups have an advantage over incumbents as it’s easier and often better to build generative AI first rather than ferry rig it into an already complex business.

Ok seeing as I’ve put a strict time deadline on this post I’d like to wrap it up with one more things.

We now live in a multi-modal AI world.

The most powerful AI models now can see, watch, listen, talk then create incredible text, code, music, poems, images, 3d models, video, expressions, texture and way more in mere seconds.

This will have a transformational impact on the world unlike anything we have seen before and we have not yet even touched the surface of capabilities.

It’s an incredible time to be alive and the positive impact about to be felt buy the world is yet to be realised and the convergence of many of these new technological steps forward it yet to be felt.

But we will start to see it soon.

Final note - over the moon to have joined Interface, an early stage investment fund by Christian Reber and Niklas Jensen where I’ll be focussing on AI.