- Neural Network began in 70s.
- AI in vogue in 80s, mainly Knowledge-based Expert System, inference engine only but NO self-learning capability.
- “AI Winter” in 90s.
- He could not get AI funding in UK.
- Refused the USA military funding, he moved to Toronto University with Canadian funding on pure Basic AI research.
- 4 decades of perseverence in Neural Network, he invented “DeepLearning” Algorithm using new approach (Machine ‘Self-learning’ capability by training in Big Data, learn from variance between output vs actual by using 19CE French mathematician Cauchy’s Calculus “Gradient Descent“. )
- Hinton thanks Canada for Basic Research Funding.
- Now working for Google.
Notes: The success of Hinton:
- Cross-discipline of 3 skills : (Psychiatrist + Math + IT) – Chinese proverb : 三个臭皮匠, 胜过一个诸葛亮 (3 ‘smelly’ cobblers beat the smartest military strategist Zhuge Liang in Chinese Three Kingdoms)
- Failures but with perseverence (4 decades)
- Courage (withstand loneliness) but with vision (see light at the end of tunnel)
- Look for condusive Research Environment : Canada Basic Research Funding
- Stick to his personal principle : Science for Peace of mankind, no ‘Military’ involvement.
Gradient Descent in Neural Network (Video here) :
Turing Award = “Nobel Prize” in Computing
Award Amount = US$ 1 million (sponsored by Google)
1950s old idea of “Neural Network” (咸鱼翻身) – Old wine put into new “bottle” (“Deep Learning”) by Prof Hinton.
The Register: Google finally touts $150 pint-sized Linux dev board with Edge TPU AI math copro brains.
“Certainly having a strong background in mathematics (eg. Linear Algebra, Multi-variables Calculus, Baeysian Probability, etc) will make it easier to understand machine learning at aconceptual level.“
“If the math seems tough, focus on the practical first, learn through analogies and by building something yourself.
But if the math comes easy, you’re starting with a solid foundation.”