Machine Learning

Catalog

• Bayesian

  • All bayesian inference methods
  • Bayes by backprop - naive variational inference
  • Difference between MCMC and VI(SVI)
  • Laplace Approximation
  • Markov Chain Monte Carlo(MCMC)
  • Prior Uncertainty - A Key Hurdle in Advancing Bayesian Machine Learning
  • reparameter trick
  • Stochastic Variational Inference
  • What is uncertainty in neural networks
  • Why we are assuming all weight distributions are Gaussian

• CV

  • DETR
    Generally, DETR
  • JEPA
    I-JEPA and ViT
  • ViT

• Diffusers

  • How does diffusion model work
  • What does the U-net do in diffusion models?

• EBM

  • How to build EBMs

• Mathematics

  • KL and JS divergence
  • KL Divergence in Variational Inference.
  • Norm 范数
  • Regularization for Deep Learning
  • SVD

• Papers Read

  • Bayesian LSTM
  • Global sparse momentum SGD
  • How to write literature review

• Reinforcement Learning

  • Actor Critic
    How PPO improved on mode collapsingMode Collapse
  • RL的主要结构种类

• Resources

• Trick Layers

  • Attention
    mamba算法multi head attentionself-attention 自注意力self-attention详细计算方法单头 single head attention单头attention back propagation
  • Transformers
    Transformer it self
  • Tricks
    layer normalization - batch normalizationRMSNorm为什么有效旋转位置编码RoPE详解知识蒸馏Knowledge Distillation