Our belief at pi Ventures is to back disruptive innovation, and companies focused on developing 10x differentiated businesses and new technologies to solve large global problems.
Technology is changing rapidly in every domain — be it through all the moves and shakes AI is making or through the creation of alternative proteins in the lab. We meet a lot of entrepreneurs from around the globe building companies in these spaces. We learn from each of these conversations. We are bringing these learnings in a mini-series called ‘DeepTech: Emerging Frontiers’ which will discuss various emerging trends within DeepTech ranging from core AI/ML to recent trends in physical innovation. The mini series is being presented in a reader friendly bite-sized format.
Here is our first chapter in the series on AI trends, the hottest talk of the town!
A Decade of AI: From Latent to Inevitable
In recent months, there has been so much buzz around generative AI and chatGPT and if ‘machines are finally coming for our jobs?’ Not since the large-scale adoption of smartphones in the late 2000s has a new tech emergence felt this sweeping.
However, while some developments in the world of AI seem unexpected, quite a lot has happened in the past decade. The boom in recent years has primarily been attributed to improvements in computing infrastructure and the availability of vast amounts of data that can be used for training purposes. The result: artificial intelligence has touched various parts of our lives with some prominent, visible developments such as self-driving cars, autonomous systems, voice assistants, and computer vision; and some more subtle progressions hidden in papers, publications, models, and frameworks.
The timeline depicts a (non-exhaustive) list of some of the critical moments in AI in the past decade. The 2010s saw the rise of deep learning, which, amongst other things, helped make sense of unstructured data. The emergence of GANs started the generative AI movement which got accelerated when transformers led to the formation of Foundational Models.
In parallel, as more and more ML models went to production through the 2010s, it gave its own momentum to the Infrastructure layer for AI. The term MLOps emerged and led to the creation of the foundation of frameworks. Today the models in production are asking for Observability and drift detection to be done in real time. We will also slowly see the shift from batch based systems to a more real time approach of doing ML.
Investible Deeptech Themes in AI
We looked at how the AI landscape has evolved over the last decade or so. Now, we are presenting our thoughts on which are the Investible themes within AI, especially from a deeptech lens.
The first wave of companies solved well-defined numerical problems such as churn and LTV prediction, OCR, etc. using ML. These spaces quickly became crowded, making the creation of differentiated solutions increasingly challenging. However, it opened an opportunity for a category of startups offering “AI tools/ modules” pretty much in an off-the-shelf manner.
Next wave of companies were built on the back of advances in NLP technology. We saw the rise of chatbots capable of answering simple queries and guiding humans through a process flow. Technology that was able to deliver the right customer experience — balancing intuitiveness and technical complexity — defined the winners. Now, with the emergence of large language models, however, the right-to-win of these companies might need a rethink.
Simultaneously, in fields such as healthcare, where obtaining large data sets was challenging, companies that built verticalized custom models also began emerging. Beyond the underlying technology, data was also a strong moat for these businesses. This continues to be an area where valuable companies can be built.
Today Generative AI is creating a significant buzz — a lot of that is coming from experimentation / building on top of foundational models. While that is unlocking interesting use cases, we are looking for more disruptive companies in the space. We will cover them in the next blog.
While the above was going on in the Applied AI side, the drive to get the models to production quicker fueled the early growth of the AI frameworks in the Infrastructure space. These were mainly dominated by big companies. However, newer needs such as drift detection and observability which need a differentiated technology are forming an interesting proposition for investments. One of the other trends we are watching closely is how the ML & AI systems which are largely batch based will transform themselves into real time continual learning frameworks.
To summarise the investible themes for us @ pi, we looked at differentiated Chatbots, Vertical use cases and some early use cases of Gen AI in our Fund 1 while now in Fund 2, we are looking at Infra use cases as well in a strong manner as shown in the diagram.
Evolution of Generative AI: A brief timeline
Generative AI has taken the world by storm. Over the next few posts, we will share a few thoughts on how we’re thinking about the space here at pi
Machines capable of defeating humans at Go, creating original artwork, and predicting structures of proteins. How did we get here?
The first development that gave machines the power of creation dates back to 2014, with the invention of generative adversarial networks (GANs), capable of producing different variants of a seed image.
The next couple of years saw the emergence of language models that could perform tasks such as transcription and basic summarization. However, these models were not capable of understanding contextual nuances that occur so often in language, limiting large scale applications.
Transformers upended this paradigm. Introduced by Google in 2017, transformers are a neural network architecture capable of understanding context and thus, meaning in language. They serve as a building block in several models even today, and paved the way for ‘foundation’ models such as GPT (Generative pre-trained transformer), which performed much better on a wider variety of language tasks.
Over the next few years, much larger models (both computer vision and language) trained on a vast corpus of data became increasingly performant on language and computer vision tasks — culminating in the launch of GPT-3, a 175 billion parameter model trained on text corpora of the entire web and more.
These models are heavy and expensive to train, arguably possible at the time only by well-funded labs or Big Tech. Massive improvements in hardware have led to AI scientists getting more bang for their buck, allowing them to train even larger models and we began to see results approach human performance with breakneck pace.
2022 proved to be a watershed moment for Generative AI, with large-language models (LLMs) being adapted to solve problems across various industries. Models like ChatGPT now allow us to perform advanced language tasks such as generalised natural language queries with high accuracy. In the computer vision domain, diffusion techniques led to the launch of photorealistic text-to-image generators such as DALL-E 2 and Midjourney
To add to this, open-source LLMs such as Stable Diffusion (alternative to DALL-E 2) and Openjourney (alternative to Midjourney) have cropped up. Their deployment has been made easy via platforms such as Huggingface — making generative AI infrastructure accessible to devs and creators alike. Empowered with the right toolset, they can build applications with the appropriate LLM for their data to power a litany of use cases across industries.
We will go over some use cases and investable themes in future posts, stay tuned!