Llion Jones, recognized as the “Pioneer of Transformers,” devoted nearly 12 years to his tenure at Google. He assumed a pivotal role in shaping the foundational Transformers research paper, which stands as a cornerstone in the realm of modern generative AI. Now he found Sakana AI with David Ha.
Jones was a member of an eight-person team at Google, collaborating on the development of transformer software, a foundational element in today’s generative AI tools. This software has enabled the creation of chatbots such as ChatGPT and Bard, as well as image generators including Stability AI, Midjourney, and Dall-E.
In a parallel trajectory, mirroring his co-authors, Jones embarked on his departure from Google to co-establish an independent AI enterprise with a fellow former Google researcher. Drawing inspiration from the intricacies of nature, Jones joined forces with his former Google colleague David Ha to establish Sakana AI, a research laboratory dedicated to generative AI, situated in Tokyo.
Since the debut of the Transformers research paper in June 2017, each co-author has transitioned from Google, entering the realm of their individual startup endeavors. This departure stems from the burgeoning worldwide demand for expertise in generative AI.
Additionally, the co-founders have offered insight into the significance of their company name, “Sakana.” Rooted in the Japanese term さかな (sa-ka-na) for fish, it encapsulates the notion of “a collective entity formed by simple rules, resembling a school of fish coming together.” This emblem carries the essence of natural principles such as evolution and collective intelligence, which inspire their pursuits.
Prior to his departure from Google, Ha occupied the role of overseeing the AI research division for the tech conglomerate in Japan. Now serving as Sakana AI’s CEO, Ha was also at the helm of research efforts for the image AI startup Stability AI.
In relation to his exit from Google, Jones elucidated that he bears no ill will towards the tech behemoth. However, he acknowledged that the immense scope of the company impeded his capacity to delve into the particular work he aspired to pursue.
From his perspective as Sakana AI’s current CTO, he believes that Google has centered its entire corporate focus around one solitary technology, resulting in a constraining framework for innovation. This comprehensive alignment, he explained, can hinder the pursuit of innovation due to its rigid parameters.
Adding to this sentiment, Ha emphasized that he and Jones have engaged in dialogues with likeminded individuals who share an interest in large language models (LLMs), even though their precise strategies are still in the process of being crystallized. Ha expressed, “I would be taken aback if language models were not a pivotal facet of the future.”
Sakana AI is steering its trajectory toward cultivating an autonomous generative AI model, capable of crafting a diverse array of content encompassing text, images, code, and multimedia. This journey ventures into a fiercely competitive realm inhabited by prominent AI heavyweights like Google, Microsoft, and OpenAI, as well as well-funded startups including Cohere, Character.ai, and Anthropic. The notable $10 billion investment by Microsoft in OpenAI this year, coupled with valuations of $2 billion for Cohere and $1 billion for Character.ai, underscores the formidable financial commitment within this domain.
Jones and Ha firmly assert that prevailing AI models encounter limitations due to their rigid and inflexible architecture, akin to fixed structures such as bridges or buildings.
Simultaneously, both Jones and Ha voice dissenting opinions about OpenAI, the startup acclaimed for propelling the concept of generative AI into the mainstream, while securing substantial investments from Microsoft and other backers. Ha characterizes OpenAI as having undergone “considerable expansion and displaying a hint of bureaucracy,” drawing parallels with specific factions within Google.
Jones, on the other hand, engages in a discourse surrounding the extent of OpenAI’s innovativeness. He posits that OpenAI’s significant milestones, such as the ChatGPT and DALL-E image generation services, have their roots in the research he conducted during his tenure at Google. OpenAI subsequently elevated and scaled these innovations, incorporating enhancements along the way, while choosing not to openly share these developments with the community. While refraining from releasing these technologies under an open-source license, OpenAI has, however, disseminated research papers that offer insights into the underlying mechanisms.
