GwanSiu Blog

Keep Learning, Shaping the Future

Support Vector Machine

1. Support Vector Machine Let’s consider two classes classification problem, and there are two point sets, each of which is $N$ point. One set of points lie in a hyperplane $H_{1}:\boldsymbol{x}{i...

Maximum Likelihood and KL Divergence

1. Maximum Likelihood In this session, I will prove that maximum likelihood is equivalent to minimize KL divergence between true distribution $p_{data}$ and model distribution $p_{\theta}$. sup...

今夜,我们都变成了曾经我们眼里的“坏孩子”

这是一个关于买烟,抽烟的故事;这是一个关于“好孩子”与“坏孩子”的话题讨论;这还是一个关于中国式“乖孩子”成长的思考。 临近冬至,香港的夜晚还是像平日一般清爽宜人。大家好,我是Siu, 走在我旁边的两个人是我的两个好朋友维克多和billy。维克多今天完成了她本科生涯的毕业设计以及一系列deadlines, 今晚她想好好地放松一下那条紧绷的神经,billy这几天被莫名的烦恼所困扰,今晚也正好...

Random Forest

1. Random Forest Decision tree is a classical algorithm in machine learning area, but decision tree suffer from the drawback of low biase and high variance(overfitting). Random forest is a boostin...

Kalman Filter

1. Introduction Kalman filter and its variants, i.e., extended Kalman fiter, are one of classical algorithm and have been regarded as the optimal solution in tracking and data prediction task. Kal...

Hierarchical Clustering

1. Introducation to Hierarchical Clustering Clustering algorithm is a process to group similar object together. From the view of ininput type, clustering algorithm can be divied into two categorie...

Hidden Markov Model

1. Introduction to Hidden Markov Model In the previous post, naive bayes and gaussian mixture model have been discussed. In naive bayes, we assume all observed data come from one possible latent d...

Latent Dirichlet Allocation(LDA)

1. Introduction to Latent Dirichlet Allocation(LDA) Latent Dirichlet Allocation(LDA) is widely used topic model in NLP area, which is also the simplest topic model. LDA is a generative model, and ...

Optimal Transport Problem and Wasserstein Distance

1. Introduction to Optimal Transport Problem Optimal transport problem is a classical problem in mathematics area. Recently, many researchers in machine learning community pay more attention to op...

Expectation Maximization

1. Introduction In this article, I will introduce Expectation Maximization(EM) algorithm. EM algorithm is introduced as early as 1950 by Ceppellini etal, and then it widely applied for parameters ...