Machine Learning Garden

Machine Learning

References

Tags: concept

Sources:

Related notes:

Deep Learning

Reinforcement Learning

Curriculum Learning

Updates:

Notes {{word-count}}

Summary:

Key points:

What is Machine Learning?

In Machine Learning, instead of defining the input -> output relationship by hand, we learn a program that acquires the relationship from data.

Tom Mitchell's definition

A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure $P$ , if its performance at tasks in $T$ , as measured by $P$ , improves with experience $E$ .

The Machine Learning method for solving any problem ever.

Define your model class.

Model class refers to the the set of possible programs。

Define your Loss Function.

Loss Function measures if one model in the model class is better than another.

Define your Optimizer.

The optimizer searches the model class to find the model that minimizes the Loss Function.

Run it on a GPU.

Marr's levels of analysis

What is learning?

We want to learn

p_\theta (y \mid x)

, and it is a model which approximates the true

p(y \mid x)

A good model should make the data look probable.

We choose

\theta

such that

p(\mathcal{D})=\prod_{i} p\left(x_{i}\right) p_{\theta}\left(y_{i} \mid x_{i}\right)

is maximized.

However, one numerical problem here is that we are multiply together many numbers less than one.

To solve the problem, we can use

\log

to convert multiplication into addition.

\log p(\mathcal{D})=\sum_{i} \log p\left(x_{i}\right)+\log p_{\theta}\left(y_{i} \mid x_{i}\right) =\sum_{i} \log p_{\theta}\left(y_{i} \mid x_{i}\right)+\text { const }

\theta^{\star} \leftarrow \arg \max _{\theta} \sum_{i} \log p_{\theta}\left(y_{i} \mid x_{i}\right)

This is called the Maximum Likelihood Estimation (MLE), and it can be formulated as a Negative Log-Likelihood (NLL) problem.

Referenced in

Inductive Bias

Machine Learning

In Machine Learning, instead of defining the input -> output relationship by hand, we learn a program that acquires the relationship from data.

Deep Learning

Deep Learning is Machine Learning with multiple layers of learned representations.

The Machine Learning method for solving any problem ever.

Machine Learning Concepts

Machine Learning basics

Deep Learning

Machine Learning

Neural Network

Machine Learning

Machine Learning Concepts

Summary: This is a knowledge base of Machine Learning concepts.

Natural Language Processing

Machine Learning

Machine Learning

Referenced in

Inductive Bias

Machine Learning

Deep Learning

Generative Model

Machine Learning Concepts

Negative Log-Likelihood (NLL)

Discriminative Model

Maximum Likelihood Estimation (MLE)

Supervised Learning

Machine Learning

Machine Learning Concepts

Deep Learning

Neural Network

Machine Learning Concepts

Natural Language Processing