The Birth of Transformers
Jakob Uszkoreit, co-founder of biotech startup Inceptive, played a pivotal role in the development of generative AI. While at Google in 2017, Uszkoreit proposed the concept of self-attention, which led to the creation of the transformer architecture. This breakthrough forms the foundation of today’s generative AI systems, including popular applications like ChatGPT.
Key Insights:
- Transformers were deployed in production at Google and other companies before gaining widespread attention
- OpenAI’s ChatGPT brought transformers into the spotlight in late 2022
- All eight authors of the seminal “Attention Is All You Need” paper have since left Google
- Uszkoreit suggests that Google’s size may have limited its ability to apply the technology as boldly as smaller companies like OpenAI
From AI to Biotech
In 2021, Uszkoreit co-founded Inceptive, a biological software company aiming to revolutionize drug development using AI. The startup recently secured $100 million in funding from investors including Andreessen Horowitz and Nvidia. Inceptive’s focus is on designing RNA molecules using generative AI to create medicines that work in harmony with biological systems. This innovative approach has the potential to transform the pharmaceutical industry and improve patient outcomes.
