Google overlooking the paper in 2017 has cost them a staggering $6.2 billion – a collective valuation of all the AI startups built on transformers today by the technology’s foundation team. 

The last of the eights authors of the pivotal transformers paper to remain at Google is (also) leaving the big tech to start his own company. Llion Jones said he will be joining his fellow ex-Google research scientist David Ha in the new endeavour to start Sakana AI. 

The paper titled, ‘Attention Is All You Need’ was discussed at Google’s meal time when the authors were discussing ways to make computers generate text efficiently. After running tests for over the next five months, the team wrote the paper having no idea of what’s coming next. Their work made the once-dominant architecture Recurrent Neural Nets (RNNs) in language processing largely obsolete. 

Notably, the transformer architecture represents the T in ChatGPT and chatbots akin. The 2017 paper laid the foundation to the generative AI wave initiated by OpenAI after making GPT models accessible to the public (and enterprises). Jones along with the co-authors are finally getting the due credit for their billion dollar paper which Google miscalculated six years ago. 

“It’s only recently that I’ve felt … famous?” Jones recently told the Washington Post. “No one knows my face or my name, but it takes five seconds to explain: ‘I was on the team that created the ‘T’ in ChatGPT.’

The $6.2 billion worth underdogs

One standout is ‘Cohere’, founded in 2019 by Aidan N. Gomez, Ivan Zhang and Nick Frosst. This NLP-focused enterprise startup has soared to a valuation of $2.2 billion. Reports reveal that Tiger Global, having invested initially, will sell a stake, reaping a 40% markup on the current valuation — reflecting on the startup’s rise.

‘Adept’, another fledgling startup with a mere year under its belt, has not only secured a valuation of $1 billion but has also managed to raise an impressive $350 million in venture capital this year. Founded by lead author Ashish Vaswani and Niki Parmar, the startup’s minimalist workforce of 25 belies its profound impact. The duo’s entrepreneurship goes beyond Adept. Their undisclosed startup, still in stealth mode, has raked in $8 million in funding led by Thrive Capital.

Noam Shazeer, another author of the transformative paper, co-founded ‘Character.AI,’ a chatbot startup that rose to a valuation of $1 billion. A $150 million funding round led by Andreessen Horowitz solidified the company’s status as a unicorn earlier in 2023.

The narrative doesn’t end here. Illia Polosukhin founded the high-performance blockchain called ‘NEAR Protocol,’ in 2017, which has a valuation of $2 billion. The latest fundraising tally of $350 million was the second nine-figure raise within four months by Polosukhin’s brainchild. 

Jakob Uszkoreit who thought of the name Transformer founded Inceptive in 2021 to use deep learning and learn life’s languages. During his 13 years stint as a senior software engineer he built the language understanding team of the Google Assistant and worked on Google Translate during its early days.  Inceptive’s value currently remains undisclosed. As the recognition finally trickles in, it is a fitting closure to see Lukasz Kaiser, the eighth author of the paper, found his place as a member of the technical staff at OpenAI two years ago. 

Attention was all they needed

Back in 2017, least did Google know its paper would eventually be cited around 90,000 times by other researchers, and the transformer architecture would underpin ChatGPT, Midjourney and the rest. While Google shared the discovery with the world, the company itself didn’t put the content generation technology to use straight away. Google kept it in hibernation till the CEO Sundar took the stage at Google I/O 2021 to announce LaMDA. The company did not strike gold unlike OpenAI due to the ‘Is AI sentient?’ controversy that summer. 

The Washington Post pointed out that the company went wrong with its scaling strategy. Out of the estimated 140,000 employees, over 7000 people are dedicatedly working on AI as per Bloomberg Opinion. Comparatively OpenAI, has a much smaller workforce — about 150 AI researchers out of approximately 375 employees in 2023. 

The company’s gigantic workforce adds to certain layers to sign off on ideas half a decade ago several former employees have confessed. Furthermore, the search giant has a very high bar for turning ideas into products, Polosukhin said in an interview. The company’s inertia and intolerance towards experiments with dead ends and iterations is another reason the transformers team decided to walk through the exit door. Moreover, there were more exciting innovations happening outside the Mountain View HQ which led to the talent exodus. 

While the company has been experimenting with models like PaLM, LaMDA and Bard, there is still no hint of a significant product. While Google stashed it away, startups capitalised on the technology to remove language barriers between humans and machines. The ground-breaking paper overlooked by the company has made Google a victim of its own success.

The post Google’s $6.2 Bn Missed AI Opportunity  appeared first on Analytics India Magazine.