bookshelf

study-goals

5 min read

🧠 Week 1: Core Math for AI

🔍 Topics:

  • Linear Algebra
  • Calculus (gradients, chain rule, partials)
  • Probability & Statistics

📚 Resources:

🛠️ Build:

  • Matrix & vector operations in Rust
  • Gradient calculator (for functions like f(x) = x^2 + 3x + 2)
  • Implement gradient_descent.rs

🤖 Week 2: Core Machine Learning Algorithms (No Libs)

🔍 Topics:

  • Linear Regression
  • Logistic Regression
  • K-Nearest Neighbors
  • Naive Bayes
  • Decision Trees

📚 Resources:

🛠️ Build:

  • Implement all models in Rust
  • Train/test on toy datasets (manually create data or use CSVs)
  • Create a predict.rs that classifies inputs based on the above models

🧬 Week 3: Genetic Algorithms & Optimization

🔍 Topics:

  • Genetic Algorithms (GAs)
  • Hill Climbing
  • Simulated Annealing

📚 Resources:

🛠️ Build:

  • GA with binary and float chromosome support
  • Fitness scoring & selection (roulette, tournament)
  • Build TestEvolver.rs: Generate optimized academic tests via GA

🧠 Week 4: Build Neural Networks from Scratch

🔍 Topics:

  • Perceptrons & Dense Layers
  • Forward Pass
  • Activation Functions (ReLU, Sigmoid, Softmax)
  • Backpropagation
  • Stochastic Gradient Descent

📚 Resources:

🛠️ Build:

  • Train XOR from scratch
  • Add support for multi-layer models
  • CLI viewer for weights & accuracy

🧬 Week 5: Neuroevolution (GAs + NNs)

🔍 Topics:

  • Evolving Neural Net Weights
  • Structure Mutation
  • Neuroevolution of Augmenting Topologies (NEAT-lite)

📚 Resources:

🛠️ Build:

  • Mutate & evolve neural net weights
  • Score NNs based on task performance
  • Use on Test Generator or Grid Path Solver

🛠️ Week 6: Build Your Own AI Framework (Micrograd-style)

🔍 Topics:

  • Tensor Struct
  • Layers, Model, Loss Functions
  • Automatic Differentiation
  • Optimizers (SGD, Adam)

📚 Resources:

🛠️ Build:

  • tensor.rs: tensors + ops + autodiff
  • model.rs: stackable layers
  • train.rs: end-to-end training with logs
  • Train simple classifiers from scratch

🧪 Week 7: Final Projects & Architecture

🎯 Project Ideas:

  • AI-Powered Academic Test Generator (GA + NN)
  • Self-learning Quiz Tutor
  • Grid Game Solver with Evolved Brain

🛠️ Deliverables:

  • Command-line or WebAssembly Rust app
  • Log fitness/loss over

1. Linear Algebra

Linear algebra is the backbone of most AI and ML algorithms, especially for working with high-dimensional data and neural networks.

  • Vectors and Matrices: Operations, norms, dot products, cross products.
  • Matrix Operations: Multiplication, inversion, transposition, and determinants.
  • Eigenvalues and Eigenvectors: Understanding dimensionality reduction techniques like PCA (Principal Component Analysis).
  • Singular Value Decomposition (SVD).
  • Tensor Operations: Multi-dimensional arrays, foundational for deep learning frameworks.

2. Calculus

Calculus is essential for understanding optimization algorithms and backpropagation in neural networks.

  • Differentiation: Partial derivatives, gradients, and Jacobians.
  • Integration: Probability distributions and expected values.
  • Differential Equations: Solving all types of differential equations.
  • Vector Calculus: Divergence, curl, and gradient.
  • Chain Rule: Critical for backpropagation in neural networks.
  • Optimization: Gradient descent, Lagrange multipliers.

3. Probability and Statistics

These are crucial for understanding the randomness and uncertainty in data and models.

  • Probability Basics: Independent events, conditional probability, Bayes’ theorem.
  • Distributions: Normal, Bernoulli, Binomial, Poisson, and others.
  • Expectation and Variance: Mean, variance, covariance, correlation.
  • Bayesian Inference: Prior, likelihood, posterior.
  • Hypothesis Testing: p-values, confidence intervals, t-tests.
  • Markov Chains: Stochastic processes and transitions.
  • Monte Carlo Methods: Sampling techniques.

4. Optimization

Optimization techniques are used to minimize or maximize objective functions in ML models.

  • Convex Optimization: Convex sets, convex functions, and optimization principles.
  • Gradient Descent: Variants like stochastic gradient descent (SGD), momentum, RMSProp, Adam.
  • Constrained Optimization: Linear and nonlinear constraints.

5. Discrete Mathematics

Discrete mathematics forms the basis of algorithm design and combinatorics in AI.

  • Set Theory: Union, intersection, and Cartesian products.
  • Graph Theory: Nodes, edges, shortest path algorithms, PageRank.
  • Combinatorics: Permutations, combinations, and counting problems.
  • Logic: Propositional and predicate logic.

6. Numerical Methods

These methods help solve mathematical problems computationally, especially for large datasets.

  • Root-Finding Algorithms: Newton-Raphson method, bisection method.
  • Numerical Linear Algebra: LU decomposition, QR decomposition.
  • Interpolation and Approximation: Polynomial interpolation, splines.
  • Numerical Optimization: Line search, trust region methods.

7. Information Theory

Information theory concepts are foundational for understanding entropy and communication in ML.

  • Entropy and Information Gain: Shannon entropy, mutual information.
  • KL Divergence: Comparing distributions.
  • Cross-Entropy Loss: Widely used in classification tasks.

8. Transformations

Transformations are often used in data preprocessing and signal analysis.

  • Fourier Transform: Frequency domain analysis.
  • Wavelet Transform: Signal processing applications.
  • Laplace Transform: Systems analysis.

9. Algorithms and Computational Complexity

Understanding the efficiency and scalability of algorithms is key for implementing ML models.

  • Big O Notation: Time and space complexity.
  • Dynamic Programming: Optimization problems.
  • Divide and Conquer Algorithms: Fast matrix multiplication, recursive methods.

10. Geometry

Geometry plays a role in understanding data structures and visualization.

  • Distance Metrics: Euclidean, Manhattan, cosine similarity.
  • Manifolds: High-dimensional data representation.
  • Convex Hulls: Applications in clustering and computational geometry.

Recommended Pathway

  1. Start with Linear Algebra and Calculus: These are foundational.
  2. Learn Probability and Statistics: For understanding uncertainty in AI models.
  3. Explore Optimization: Crucial for training models.
  4. Study Discrete Math and Algorithms: For structuring and implementing models.
  5. Integrate Numerical Methods and Transformations: For computational efficiency.
  6. Understand Information Theory: For advanced AI applications.