Illia Polosukhin On Inventing The Tech Behind Generative AI At Google

Illia Polosukhin is one of the “Transformer 8,” a group that many call the founding fathers of generative AI. They co-wrote a paper at Google in 2017 that established the core concept that makes generative AI possible: transformers. Polosukhin left Google before the paper was published to co-found Near, an AI startup and blockchain ecosystem with millions of users. CNBC's Katie Tarasov sat down with Polosukhin to talk about how the transformer was invented and what he sees as the pitfalls and possibilities of the tech revolution he helped kick off.
Produced by: Katie Tarasov
Shot and Edited by: Lisa Setyon
Supervising Producer: Jeniece Pettitt
Additional Footage: Near, Nvidia
» Subscribe to CNBC: cnb.cx/SubscribeCNBC
» Subscribe to CNBC TV: cnb.cx/SubscribeCNBCtelevision
About CNBC: From 'Wall Street' to 'Main Street' to award winning original documentaries and Reality TV series, CNBC has you covered. Experience special sneak peeks of your favorite shows, exclusive video and more.
Want to be a successful, confident communicator? Take CNBC’s new online course Become an Effective Communicator: Master Public Speaking. We’ll teach you how to speak clearly and confidently, calm your nerves, what to say and not say, and body language techniques to make a great first impression. Sign up today and use code EARLYBIRD for an introductory discount of 30% off through July 10, 2024: cnb.cx/4aryNgM
Connect with CNBC News Online
Get the latest news: www.cnbc.com/
Follow CNBC on LinkedIn: cnb.cx/LinkedInCNBC
Follow CNBC News on Instagram: cnb.cx/InstagramCNBC
Follow CNBC News on Facebook: cnb.cx/LikeCNBC
Follow CNBC on Threads: cnb.cx/threads
Follow CNBC News on X: cnb.cx/FollowCNBC
Follow CNBC on WhatsApp: cnb.cx/WhatsAppCNBC
#CNBC
Illia Polosukhin On Inventing The Tech Behind Generative AI At Google

[@velo1337]

with how easy his explanations are, you can clearly see that he was one of the first ppl who coded it.

[@elmermarko5126]

Near should make great gains the next few months. Near is at the forefront of Ai and crypto.

[@TommyChong677]

I recently read an article about a man who identified AI stocks before the AI boom, highlighting the importance of information and insight. I believe AI is poised to enter a new phase, and I aim to position my $200k investment portfolio to capitalize on significant gains.

[@FraserRyan-oz2bj]

Absolutely crucial in the stock market: information, insight, and predictability. As an early investor in NVDA, AMD, ANSS, and LRCX, my advisor's guidance was invaluable.

[@Cynthia-mm1cv]

People often underestimate financial advisors' importance. Over 50 years of data reveal that those who work with advisors typically earn more than those who go it alone. I've been fortunate to work with one for 13 years, resulting in a $1 million portfolio, largely from early investments in AI and other growth stocks.

[@tinsleyLuna]

How can i reach this person?

[@Cynthia-mm1cv]

I'm cautious about giving specific recommendations as everyone's situation varies. Consider independent financial advisors like "LUCIA ALICIA CRUZ" I've worked with her for years and highly recommend her. Check if she meets your criteria.

[@tinsleyLuna]

I looked up her name online and found her page. I emailed and made an appointment to talk with her. Thanks for the tip.

[@attilapataki6639]

Ok! While I don't necessarily like how he describes AI in the beginning (and this is just a matter of taste and subjectivity) I do appreciate him for how he answered the doomsday question. Congrats 👏🏻and for humanities sake: I do hope there are more Ai devs like him.

[@sha_la007]

This is great! I would love to understand how they handle words like: hill, heal, and heel. They all sound alike but have different meaning depending on the context of the sentence. How are they training to understand meaning, considering English has a lot of ambiguity in the language. It's often one of the hardest languages to learn because of this.

[@yashraghuwanshi931]

The computer are reading it , and not hearing it so it can understand the meaning using the spelling

[@muzehack]

When he talks about "attention" in AI models that is basically the mechanism that determines the context based on its training. It works very much like humans do. When he talked about the cat and the wall and whether "it was tall" referred to the cat or the wall, the AI develops weights that relate to the probability that "it was tall" was referring to the cat, because in the examples it was trained on it was more likely that the cat was the subject of the sentence and "it" referred to the cat.

[@urbanstrencan]

He is the one who started our next revolution with AI.

[@cryptomotivation2639]

Near is the top Ai blockchain and will be a top 5 coin this crypto cycle. Easy 10X from here.

[@kfique]

Great interview, tks!

[@ricnyc2759]

I have a question: are the transformers based on software or hardware? Is it like an "if - else statement"?

[@anb4351]

If else but more complex, i.e if probability of x happening given y based on training data set is greater than z then find the most plausible next token

[@___________02]

Hi Katie,
Congratulations to you for giving birth to a new life. 🎉🎉

[@sputnik8543]

I think what we learned is you have to be named Ilya or Illia to lead the AI race; so I’ll be changing my name to Ilya or some variant

[@pacanosiu]

God will .... you up for this

[@RealSmall]

Um, interesting

[@Nickspizza413]

Seriously Bullish on Near.

[@komodo_04]

The future is $NEAR

[@aberba]

These Russians are killing it in the AI space

[@sputnik8543]

I think what we learned is you have to be named Ilya or Illia to lead the AI race

[@ODevaneador]

😂😂😂😂

[@ODevaneador]

It's a name thing, wtf

[@anipacify1163]

Ashish Vaswani goat 🐐. The main guy behind the transformer paper

[@mizan05h]

❤

[@DmitryVinogradovjr]

Катя и Илья

[@auro1986]

this technology may have been from darpa

[@happeningkorea]

😀😀😀

[@JohnDoe-mx3rr]

The doomsday saying there no economic has not seen industrial war machine

[@dad.things]

The title makes it sounds like he was almost a cofounder 🤷🏻‍♂️

[@NadimShaikh-qv7zj]

He was the co-author of the monumental paper, so more important for the specific tech than any other co-founder or ceo.

[@tjm4702]

What the heck u talking about ? He is the cofounder.

[@NadimShaikh-qv7zj]

@@tjm4702 I think op probably mean to say that the he is not the co-founder of Google or OpenAI or one of the other big AI companies that are building these foundational models.

[@dad.things]

Look at the thumbnail. It says Near cofounder. Sounds like he was almost a cofounder. It’s a joke tech bros!

[@NadimShaikh-qv7zj]

@@dad.things Oh, I interpreted it in a totally different way. I'm sorry, my bad!

[@user-pq3uz2zb9i]

Brawndo - its what plants CRAVE!

[@amatiphone9644]

Both Russian lol

[@Rugged-Mongol]

And the host is an American of Russian-stock/descent.

[@rho992]

The primary author is Indian

[@IgorFlysta]

He is Ukrainian

[@FATmenDRIVEtrucks]

We’re all human doesn’t matter where you were born. Being pro America is dead we do nothing outsource everything this isn’t the 1950’s anymore

[@blingberry]

He is ukranian

[@donson3326]

Contributed the least to the paper. Another rug pull.

[@littlebrit]

They are both Ukrainians. Ukraine has done so much for blockchain.

[@romanyelisov4710]

Слава Україні

[@invisiblian]

so glad illia didn't do a joe biden on this interview. lol