GwanSiu Blog

Keep Learning, Shaping the Future

Kalman Filter

1. Introduction Kalman filter and its variants, i.e., extended Kalman fiter, are one of classical algorithm and have been regarded as the optimal solution in tracking and data prediction task. Kal...

Hierarchical Clustering

1. Introducation to Hierarchical Clustering Clustering algorithm is a process to group similar object together. From the view of ininput type, clustering algorithm can be divied into two categorie...

Hidden Markov Model

1. Introduction to Hidden Markov Model In the previous post, naive bayes and gaussian mixture model have been discussed. In naive bayes, we assume all observed data come from one possible latent d...

Latent Dirichlet Allocation(LDA)

1. Introduction to Latent Dirichlet Allocation(LDA) Latent Dirichlet Allocation(LDA) is widely used topic model in NLP area, which is also the simplest topic model. LDA is a generative model, and ...

Optimal Transport Problem and Wasserstein Distance

1. Introduction to Optimal Transport Problem Optimal transport problem is a classical problem in mathematics area. Recently, many researchers in machine learning community pay more attention to op...

Expectation Maximization

1. Introduction In this article, I will introduce Expectation Maximization(EM) algorithm. EM algorithm is introduced as early as 1950 by Ceppellini etal, and then it widely applied for parameters ...

Dimension Reduction:PCA, FA, and ICA

1. Introduction Dimension reduction algorithms are one of hot topic in machine learning and representation learning, and some of them, such as principle component analysis (PCA), factor analysis (...

Fater Iterative Shrinking Threshold Algorithm

1. Introduction: Proximal Mapping The definition of proximal mapping of convex function $h$ is: \[\begin{equation} \textbf{prox}_{h}(x)=\arg\min_{\mu}(h(\mu)+\frac{1}{2}\Arrowvert\mu-x\Arrowvert^...

KNN Algorithms

1. Introduction Neareset neighborhood algorithm is one kind of instance-learning algorithm, which has been attracted attention by researchers because the error-rate of 1-nearest-neighbor classific...

数学分析笔记系列(1)-实数的完备性

九月,便开始了PhD的征途,在我面前的将是知识的星辰大海。我从事的是low-level image processing以及machine learning领域,强有力的数学工具将有助于问题的思考以及让我对繁杂数学符号少一些恐惧。九月中旬,我开始旁听香港城市大学Ciarlet教授的泛函分析课程,弱鸡的实数分析让我很难follow他的进度。因此,我才打算在科研空闲之余重新捡起数学分析相关...