MoroccoAI Conference 2022 Honorary Keynote Prof. Geoffrey Hinton - The Forward-Forward Algorithm

In this Honorary Keynote, Prof. Geoffrey Hinton, Full Professor at University of Toronto, discusses "Learning Deep Neural Networks without Propagating Derivatives".
This is a new learning algorithm for artificial neural networks, called Forward-Forward algorithm, inspired by our knowledge about neural activations in the brain. While backpropagation is extremely useful, it is very different from what we know about the brain. Prof. Hinton explains how this new algorithm has the potential to replace backprop in the future.
The Q&As is moderated by Dr. Salim Chemlal, MoroccoAI co-founder.
MoroccoAI Conference 2022/2023 aims to formulate a Promising National AI Strategy to support sustainable development across key sectors in Morocco.
MoroccoAI is a non-profit association led by AI scientists and researchers in Morocco and abroad.
MoroccoAI vision is to promote AI education and excellence in Research in Morocco.
Follow us on LinkedIn at: / morocco-ai
Subscribe and support us by sharing with friends and colleagues.
Content of this video:
0:00 Introduction
2:20 Prof. Hinton's presentation
35:15 Q&A

[@KahurangiSteez]

It's kinda crazy that these videos only have 9 comments, 2.6k views etc. This guy will be remembered for centuries. Hello future

[@aymaneboutahir]

This is too much for us ❤❤

[@Soracontent]

Thanks for sharing this knowledge with us looking forward to watch more content like this one, very interested to learn more about the AI and be a part of Moroccan AI community

[@oussamahamal9870]

Interesting thank you Prof. Hinton

[@zouhirelmezraoui1336]

محاضرة قوية لكم جزيل الشكر و التقدير و الاحترام و لا يسعنا الا طلب المزيد من الترجمة التلقائية للفيديو NPL لتعميم الفائدة للجميع AI For Everyone

[@zouhirelmezraoui1336]

علمني يا سيدي علمني و خذ بيدي Deep Learning Supervised 🎓🎓🎓

[@zouhirelmezraoui1336]

Thanks Prof for This Great Concepts

[@abbaoui3213]

Top

[@DmytroBronzebeard]

🎯 Key Takeaways for quick navigation:
00:50 *🏅 Geoffrey Hinton, a renowned AI researcher, made significant contributions to neural network research, including inventing the backpropagation algorithm and various deep learning techniques.*
03:08 *🧠 Hinton questions whether the brain uses backpropagation for learning, suggesting the brain likely uses different mechanisms due to anatomical and computational constraints.*
05:14 *🔍 The "Forward-Forward Algorithm" is proposed as an alternative to backpropagation, designed to be robust to hardware variability and inspired by the brain's recurrent nature.*
13:06 *💡 The Forward-Forward Algorithm achieves similar performance to backpropagation in supervised learning tasks, albeit slightly slower, by incorporating hard negative cases during training.*
19:30 *🔄 To address top-down effects in perception, the algorithm treats static images as videos, leveraging recurrent networks to model predictive interactions between different layers of processing.*
25:25 *🧠 In training, the algorithm adjusts to prefer low activity for positive data and high activity for negative data, resembling predictive coding principles observed in the brain.*
26:00 *🔄 The forward-forward algorithm suggests collecting gradients from the beginning and decaying the learning rate gradually, resulting in comparable performance to backpropagation.*
32:28 *🧠 The forward relaxation process, unlike backpropagation, involves information diffusion rather than direct propagation, influencing learning across layers in neural networks.*
35:02 *⚙️ The forward-forward algorithm introduces a potential paradigm shift towards analog computation, enabling hardware-agnostic neural network execution and promising low-power, high-efficiency systems.*
41:16 *🔀 A bifurcation between traditional digital hardware and novel analog low-power devices is foreseen, where shared weights and positive-negative data contrast offer advantages in the latter.*
45:28 *🌍 Collaborative efforts, including mentoring, talks, and focusing on Africa-specific AI applications, can leverage local talent and address unique challenges to foster AI advancements in the region.*
Made with HARPA AI

[@dianes6245]

One idea stands out - the enormous info content of matter and the physical world. The brain understand the physical world and is trained on that giant content. Not so much the content of the internet.
This explains the disparity - that an LLM seems to know more than a human with fewer wts or connections. The LLM does not know more - just internet scraped data. It may not know that a cactus had sharp points because that is data about the physical world.
It is very hard to compare info the brain knows with info that an LLM knows. If that were possible, we may find that the brain vastly exceeds the knowledge in an LLM.
It does not know the number of rocks in my front yard. I dont either, but its easy to estimate. Count rocks in 1 sq meter, then multiply. You cant do the count with a picture because rocks are layered.
That kind of thing goes on and on. An LLM does not know how a ball is caught. Keep a constant angle - its that simple. But it dont know that. It does not know that the fall of an apple is subject to uncertainty, and how much that might be for a certain kind of tree. More and more and more.
This affects AGI. Because an AGI need to function as a human and have that info from the physical world. To pound nails. To fold laundry. The hard stuff.

[@zouhirelmezraoui1336]

Waarom Doe Je Gestoord Je Mag Kiezen Meet Geduld Geen Geluk Zonder geld ( Back Propagation)

[@zouhirelmezraoui1336]

💖👆💖🎓🎓🎓👍👍👍🔭🔭🔭🌍🌍🌍🐙🐙🐙🏄🏄🏄🌴🌴🌴⛅⛅⛅💻💻💻👉👆👈🎅🎅🎅👊👊👊

[@zouhirelmezraoui1336]

NPL