Python Big Data Counter

Useful python Big Data “counter” technique.

https://realpython.com/python-counter/

Naïve Bayes Model

https://v.ixigua.com/ebbtHmc/

AI & Big Data are Related

https://searchenterpriseai.techtarget.com/tip/How-do-big-data-and-AI-work-together

Scaling Up System For 100-milion Users

https://levelup.gitconnected.com/how-to-design-a-system-to-scale-to-your-first-100-million-users-4450a2f9703d

3 Mathematical Laws Data Scientists Need To Know :Zipf’s Law

https://www.kdnuggets.com/2021/03/3-mathematical-laws.html

Zipf’s Law (Word Frequency)

Python Libraries for Machine Learning

Key Python Packages for Data Science

Build First Data Science Apps

https://www.kdnuggets.com/2021/02/build-first-data-science-application.html

Naive Bayes | Gaussian Naive Bayes with Hyperpameter Tuning in Python

https://www.analyticsvidhya.com/blog/2021/01/gaussian-naive-bayes-with-hyperpameter-tuning/

朴素贝叶斯分类ML : Naive Bayes Probability

看动画，成为机器学习工程师，朴素贝叶斯分类，原来数学这么有趣

https://m.toutiaoimg.cn/a6906067192412897796/?app=news_article&is_hit_share_recommend=0&tt_from=android_share&utm_medium=toutiao_android&utm_campaign=client_share

Homology

Part 1: “Homology” without Pre-requisites, except “Function” (he un-rigourously interchanges with “Mapping”, although Function is stricter with 1-and-only-1 Image) .

Part 2: Simplex (单纯形)

Topology History : Euler Characteristic eg. (V – E + R = 2) ,
Poincaré invention

This video uses Algebra of point, line, triangle… to explain a Simplex (plural: Simplices) in R^{n} Space, that is organizing the n-Dimensional “Big Data” data points into Simplices, then (future Part 3, 4…) compute the “holes” (or pattern called Persistent Homology).

Part 3: Boundary

Part 3 justifies why triangles (formed by any 3 data points) called “Simplex” 单纯形(plural: Simplices) are best to fill any Big Data Space.

Python for naive Bayes algorithm

A Guide to the Naive Bayes Algorithm

Information Theory for Data Science

https://www.kdnuggets.com/2021/01/essential-math-data-science-information-theory.html

Inferential Statistics

https://www.analyticsvidhya.com/blog/2021/01/inferential-statistics-sampling-distribution-central-limit-theorem-and-confidence-interval/

Differential Equations Versus Machine Learning

Differential Equations Versus Machine Learning | by Col Jung | Nov, 2020 | Medium

View at Medium.com

TDA : Topological Data Analysis

Data Science now applies Algebraic Topology : Persistent Homology.

https://en.m.wikipedia.org/wiki/Topological_data_analysis

Source (Quora) :
https://www.quora.com/Is-advanced-mathematics-useless/

Why Use Ensemble Learning?

https://machinelearningmastery.com/why-use-ensemble-learning/

Data Science Minimum: 10 Essential Skills You Need to Know to Start Doing Data Science

https://www.kdnuggets.com/2020/10/data-science-minimum-10-essential-skills.html

李彦宏Baidu CEO : Internet 3 Episodes : PC->Mobile ->AI

李彦宏剑桥大学演讲

https://m.toutiaoimg.cn/a6805832038692684300/?app=news_article&is_hit_share_recommend=0&tt_from=android_share&utm_medium=toutiao_android&utm_campaign=client_share

李彦宏 Baidu CEO Cambridge Speech 剑桥大学演讲
《3 waves of Internet》:
1) PC- based (1997-)

• Search Webpages
• 6-month software update cycle

2) Mobile-based (2010 -)

• “APP” is born
• Eco-System : eg. Apple Appstore, Google PlayStore
• O2O (Online to Offline) : Same day Hotel booking/Restaurant /…
• SW Update everyday few times

3) AI-based (2017 – now)

• Voice recognition sans keyboard input
• Image recognition (eg. Customer ePayment :McDonald’s )
• Natural language Pattern NLP (Salesman Virtual Assistant)

How much Math you need in Data Science?

https://www.kdnuggets.com/2020/06/math-data-science.html

Time Series Forecasting with Python

“Hands-on Time Series Forecasting with Python” by Idil Ismiguzel https://link.medium.com/KulDiXl816

Functional Programing in Data Science Projects

“Functional Programing in Data Science Projects” by Nathanael Weill https://link.medium.com/UiysKbFl16

MATH en supérieure -les applications-

The French Math distinguishes the Correspondence (对应):

in two definitions : Mapping (Application) & Function = Maximum 1 arrow from E to F.

1. Mapping (L’Application 映射)

• Function (La Fonction 函数): ONLY ONE image in F

Counter Examples:

Free Online Math in Data Science

https://towardsdatascience.com/4-free-maths-courses-to-do-in-quarantine-and-level-up-your-data-science-skills-f815daca56f7

How can maths fight a pandemic?

Key points:

1. SIR Model: used since 1910 for flu pandemic
2. SEIR Model: “E” for “Exposed” people but asymtomic for COVID19.
3. Social Distancing : 60% reduction effect on Ro (Reproduction number: 1.4 ~ 2.5)

https://plus.maths.org/content/how-can-maths-fight-pandemic

…

The Math Behind Social Distancing

https://www.visualcapitalist.com/the-math-behind-social-distancing/

Speak Math, not Code

Coding is about algorithm which is Math. The AI, Machine Learning, deep learning, Big Data, 5G Polar Codes, etc are all about Maths.

https://m.phys.org/news/2020-03-math-code.html

MIT New Course (Prof Gilbert Strang) : Linear Algebra and Learning From Data

https://m.toutiaocdn.com/group/6643741079306764814/?app=news_article_lite&timestamp=1573147247&req_id=201911080120460100140261091245FA0A&group_id=6643741079306764814
…

[MIT OCW Online Course Videos]

https://ocw.mit.edu/courses/mathematics/18-065-matrix-methods-in-data-analysis-signal-processing-and-machine-learning-spring-2018/video-lectures/index.htm

[Full Video]

https://m.toutiaoimg.com/group/6735569795619488264/?app=news_article_lite&timestamp=1573149133&req_id=201911080152120100140470322F644EBF&group_id=6735569795619488264

丘城桐：基础数学和AI, Big Data

AI and Big Data are Twins, their Mother is Math.

“AI 3.0“ today, although impressive in “DeepLearning“, is still using “primitive” high school Math, namely:

• Statistics,
• Probability (Bayesian) ,
• Calculus (Gradient Descent)

AI has not taken advantage of the power of post-Modern Math invented since WW II, esp. IT related, ie :

• Category Theory (Functional Programming),
• Algebraic Topology : Homology (Big Data Analytics)
• Homotopy Type Theory ‘HoTT’ (Machine Proof Math Theorems) .

That is the argument of the Harvard Math Dean Prof ST Yau 丘城桐 (First Chinese Fields Medalist), who predicts the future “AI 4.0“ can be smarter and more powerful.

https://www.toutiao.com/group/6751615620304863755/?app=news_article_lite&timestamp=1572193294&req_id=2019102800213401000804710406682570&group_id=6751615620304863755

… Current AI deals with Big Data:

1. Purely Statistical approach and experience-oriented, not from Big Data’s inherent Mathematical structures (eg. Homology or Homotopy).
2. The Data analytical result is environment specific, lacks portability to other environments.

…

3. Lack effective Algorithms, esp. Algebraic Topology computes Homology or Co-homology using Linear Algebra (Matrices).

4. Limited by Hardware Speed (eg. GPU), reduced to layered-structure problem solving approach. It is a simple math analysis, not the REAL Boltzmann Machine which finds the most Optimum solution.

Notes:

AI 1.0 : 1950s by Alan Turing, MIT John McCarthy (coined the term “AI”, Lisp Language inventor).

AI 2.0 : 1970s/80s. “Rule-Based Expert Systems” using Fuzzy Logic.

[AI Winter : 1990s / 2000s. Failed ambitious Japanese “5th Generation Computer” based on Prolog-based “Predicate” Logic]

AI 3.0 : 2010s – now. “DeepLearning” by Prof Geoffry Hinton using primitive Math (Statistics, Probability, Calculus Gradient Descent)

AI 4.0 : Future. Using “Propositional Type” Logic, Topology (Homology, Homotopy) , Linear Algebra, Category.

Mathematics behind Machine Learning – The Concepts you Need to Know

https://www.analyticsvidhya.com/blog/2019/10/mathematics-behind-machine-learning/

Data Science & Machine Learning (AI is a sub-discipline) overlap but not the same:

Data Science Math

https://m.toutiaocdn.com/group/6711910731773641223/?app=news_article_lite&timestamp=1569484366&req_id=201909261552450100140470722556973C&group_id=6711910731773641223

Statistical Significance Explained

“Statistical Significance Explained” by Will Koehrsen

https://towardsdatascience.com/statistical-significance-hypothesis-testing-the-normal-curve-and-p-values-93274fa32687

Normal Distribution

Convert to Z-score

p-Value vs Alpha (0.05% = noise)

A Programmer’s Regret: Neglecting Math at University – Adenoid Adventures

Advanced Programming needs Advanced Math: eg.

Video Game Animation: Verlet Integration

AI: Stats, Probability, Calculus, Linear Algebra

Search Engine : PageRank: Linear Algebra

Abstraction in Program “Polymorphism” : Monoid, Category, Functor, Monad

Program “Proof” : Propositions as Types, HoTT

https://awalterschulze.github.io/blog/post/neglecting-math-at-university/

Abstraction: Monoid, Category

Category

Why do Neural Networks Need an Activation Function? | Data Stuff

http://www.datastuff.tech/machine-learning/why-do-neural-networks-need-an-activation-function/

Activation function: Non-Linear Function

..

What if no Activation function : Affine Transformation

AI with Advanced Math helps in discovering new drugs

https://theconversation.com/i-build-mathematical-programs-that-could-discover-the-drugs-of-the-future-110689?from=timeline

Advanced Mathematical Methods with AI is a powerful tool:

• Algebraic Topology (Persistent Homology)
• Differential Geometry
• Graph Theory

https://sinews.siam.org/Details-Page/mathematical-molecular-bioscience-and-biophysics-1

Learn NUMPY in 5 minutes – BEST Python Library!