GwanSiu Blog

Keep Learning, Shaping the Future

Expectation Maximization

1. Introduction In this article, I will introduce Expectation Maximization(EM) algorithm. EM algorithm is introduced as early as 1950 by Ceppellini etal, and then it widely applied for parameters ...

Dimension Reduction:PCA, FA, and ICA

1. Introduction Dimension reduction algorithms are one of hot topic in machine learning and representation learning, and some of them, such as principle component analysis (PCA), factor analysis (...

Fater Iterative Shrinking Threshold Algorithm

1. Introduction: Proximal Mapping The definition of proximal mapping of convex function $h$ is: 2. Faster Iterative Shrinking Threshold Algorithm The basic idea of the iterative shrinkage algo...

KNN Algorithms

1. Introduction Neareset neighborhood algorithm is one kind of instance-learning algorithm, which has been attracted attention by researchers because the error-rate of 1-nearest-neighbor classific...

数学分析笔记系列(1)-实数的完备性

九月,便开始了PhD的征途,在我面前的将是知识的星辰大海。我从事的是low-level image processing以及machine learning领域,强有力的数学工具将有助于问题的思考以及让我对繁杂数学符号少一些恐惧。九月中旬,我开始旁听香港城市大学Ciarlet教授的泛函分析课程,弱鸡的实数分析让我很难follow他的进度。因此,我才打算在科研空闲之余重新捡起数学分析相关...

Principle of Microeconomics: Introduction

1. Introduction Economics is social science that examines how people choose among the alternative availiable to them, the foundamental issue of which is scarcity. Economics is quasi-science, becau...

Naive Bayes

1. Introduction Naive Bayes is a classical supervised learning method, which is simple but effective. Usually, given a dataset $X={x_{1},…,x_{N}}$ and its label $Y={y_{1},…,y_{K}}$, where $N$ deno...

Logistics Regression

1. Logistic Regression Data: Inputs are continuous vectors of length $K$. Outputs are discrete. Model: Logistic function applied to dot product of parameters with input vector. Learning: Fin...

Linear Regression

1. Linear Regression Algorithm 1.1 Linear Regression Originally, linear regression is a linear combination of input variables, the goal of linear regression is find a polynomial funciton to appro...

Naive Bayes and Logistics Regression

1. Naive Bayes Algorithm First, we see the whole picture of Naive Bayes. Naive bayes algorithm is supervised learning algorithm, it has two formulations for different inputs, such as discrete-valu...