GwanSiu Blog

Keep Learning, Shaping the Future

Logistics Regression

1. Logistic Regression Data: Inputs are continuous vectors of length $K$. Outputs are discrete. Model: Logistic function applied to dot product of parameters with input vector. Learning: Fin...

Linear Regression

1. Linear Regression Algorithm 1.1 Linear Regression and Basis Function Originally, Linear Regression is a linear combination of input variables, the goal of linear regression is find a polynomia...

Naive Bayes and Logistics Regression

1. Naive Bayes Algorithm First, we see the whole picture of Naive Bayes. Naive bayes algorithm is supervised learning algorithm, it has two formulations for different inputs, such as discrete-valu...

浅谈中国式的“孝文化”

“仁义礼智信,忠孝廉耻勇” 是中华流传几千年的传统普世价值观。其中,“孝”字,在经过千年洗礼之后,便深深植入中国广大老百姓的骨髓,扎根于生活的方方面面。现代社会价值体系中,“孝”也当仁不让地成为一种政治正确,被人说“不孝”,那便是一种极大的罪过。 在成长过程中,我印象中,有许多人,其中包括亲戚朋友,邻里街坊,都会时不时地当着父母的面夸赞我,然后转身便我说:“xx, 你爸妈辛辛苦苦赚钱供你读...

The completement of real number

本节主要讲述实数的完备性 1. 实数完备性主要体现在两个等价的叙述: 1.1 完备性(1) 假设实数列${a_{n}}$具有如下性质: ${a_{n}}$ 单调递增,则:$a_{n}\leq a_{n+1}\forall n.$ ${a_{n}}$上方有界,即:$\exists k$ 使得$a_{n}\leq k\forall n$. 则$\displaystyl...

Analysis: Interger and Rational Number

1. Integers 1.1 The definition of intergers (Definition-integer): An integer is an expression of the form $a-b$, where $a$ and $b$ are natural numbers. Two integers are considered to be equal, $a...

Note(7)--Uniform Laws, empirical process theory and VC dimension

1. Uniform Convergence and Empirical Risk Minimization Empirical risk minimization is core topic in machine learning. Let’s take a binary classification as example. Given a training set ${(X_{1},...

Note(8)-- Sufficiency Statistics

1.Sufficient Statistic and Factorization Theorem 1.2 The Definition of Sufficient Statistic Sufficiency Statistics: A statistics $T(X_{1},…,X_{n})$ is said to be sufficient for the parameter $\th...

Variational Inference

1. What’s variational inference? Variational inference is an approximate technique, which is widely used in inference problem in modern statistic. As we all know, inferene problem is especially im...

Sampling-Inverse sampling, Reject-accept sampling and MCMC

In this article, I will discuss some sampling methods: Inverse Samling, Reject-Accept Sampling, Important Sampling, Markov Chain Monte Carol(MCMC)-MH Algorithm and Gibbs Sampling. Why do we ne...