Mathematical Foundations of Machine Learning: Principles & Practical Implementations

Module 1: Modeling, Black‑Box, and White‑Box Modeling Techniques (2 hours)

This module introduces the fundamental concept of modeling. It builds a rigorous distinction between white‑box, gray‑box, and black‑box modeling techniques with practical examples. It sets the stage for all subsequent modules by clarifying how different modeling paradigms relate to:

• Mathematical transparency
• Interpretability
• Model risk
• Practical implementation choices

Module 2: Types of Data and Problem Formulations (1 hour)

This module teaches how to correctly identify:

• The type of ML problem (classification vs regression).
• The type of input features.
• How these choices affect model selection, encoding, and evaluation.

Module 3: Optimization and Statistical Foundations of Learning (2 hours)

As Module 1 introduces the modeling process, and Module 2 illustrates the type of data, it is time in Module 3 to learn how to fit the model to data. Without optimization, modeling is just theory. This Module 3 introduces: cost function, loss function, empirical risk minimization, gradient decent, statistical interpretation of regression, regularization as constrained optimization.

Module 4: Types of Machine Learning Models: Mathematical Foundations and MATLAB Implementation (3 hours)

It is time now to go deeper into Machine Learning models and learn how these models work and how to implement them on real data using MATLAB. This Module 4 illustrates why the MATLAB, and it introduces the most common and effective ML models that can be used for a wide range of problems and data types and the corresponding MATLAB functions:

• Linear/logistic regression models (MATLAB functions fitlm,fitclinear,lasso,ridge).
• Support vector machine (SVM) (fitcsvm,fitrsvm)
• Artificial neural networks (ANNs) (patternet,fitnet).
• k-Nearest neighbors (kNNs) (fitcknn,fitrknn).
• Decision trees and tree-based models (fitctree,fitrtree)
•  Ensemble Methods (fitcensemble,fitrensemble).
• Gaussian Processes (fitrgp).

Module 5: Machine Learning Model Validation

From ML model designer to responsible ML user, this module introduces how to ensure that the model is reliable, stable, and suitable for deployment. It introduces the mathematical foundations of model validation, the statistical tools used to measure generalization, and the practical MATLAB techniques required to perform robust evaluation. It includes:

• Local vs. global optimum
• k-fold cross-validation.
• Model-accuracy decay.
• Model uncertainty.
• Residual analysis.