Understanding Machine Learning Through Examples

A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.

Basic Concepts

In recent years, artificial intelligence has developed rapidly, and we often hear terms such as artificial intelligence, machine learning, deep learning, and reinforcement learning. In fact, they are closely related to one another.

Artificial Intelligence

Artificial Intelligence (AI) is a broad field, and it can be defined from two aspects: “capability” and “discipline”. From the perspective of capability, artificial intelligence refers to intelligence realized on machines by artificial means; from the perspective of discipline, artificial intelligence is a discipline that studies how to construct intelligent machines or intelligent systems so that they can simulate, extend, and expand human intelligence.

Machine Learning

Machine Learning (ML) is a subset of AI, focusing on developing algorithms and statistical models. Machine learning refers to algorithms that enable computers to use data to improve their performance on a task without being explicitly programmed. Machine learning can in turn be divided into supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and so on.

Deep Learning

Deep Learning (DL) is a subfield of machine learning. It uses neural network structures similar to the human brain—especially deep neural networks with multiple layers—to learn complex patterns and representations.

Reinforcement Learning

Reinforcement Learning (RL) is a branch of machine learning. It focuses on how to enable an agent to learn the best behavior or policy through interaction with the environment, so as to maximize some cumulative reward.

Deep Reinforcement Learning

Deep Reinforcement Learning (DRL) is the combination of deep learning and reinforcement learning. It uses deep neural networks as the agent’s brain to handle high-dimensional input data and learn complex policies.

Their relationships can be roughly summarized in the figure below:

Course Basics

Machine Learning

The most important thing in machine learning is to find a function—to give the machine the ability to find a function.

Usually, functions are very complex, and it is difficult for a person to write them out themselves; if they are found by the machine on its own, there are many advantages.

The functions that machine learning looks for are usually different; depending on actual requirements, they can be divided into regression, classification, structured learning, and so on.

Regression

Suppose the function to be found outputs a numerical value, or a scalar. This kind of machine learning task is called regression. (Predicting a future value based on historical data and metrics)

Among regression problems, a fairly classic one is house price prediction; this problem is used as an example in many tutorials and can be practiced on Kaggle.

Classification

A classification task is to let the machine do multiple-choice questions: the output of the function is one option selected from the preconfigured options. Usually, determining the option involves probability calculations—judging the most likely option according to probability. Common classification algorithms all have traces of probability; personally, I feel that the development of artificial intelligence technology is indeed inseparable from the support of mathematics.

Among classification problems, a fairly common one is the spam detection example: judging whether an email is spam is a binary classification problem. In addition, there is also multi-class classification; AlphaGo playing Go is about finding the most correct option from the board, and there is also animal recognition (cat, dog, bird…).

Structured Learning

In this case, machine learning is not just about making a choice or outputting a number, but about producing a structured object, such as drawing a picture or writing an article… In recent years, it has gradually gained more applications. I once saw an interesting figure in another study group:

Structured learning still needs to be explored by us.

Case Study

The existing data is past information from YouTube, and what we want to obtain is how many views there may be tomorrow Implement this process through a function

The process by which machine learning finds a function

1. Write out a function with unknown parameters (this function is set based on the knowledge we have accumulated; it is only a guess), and this function can predict the number of views in the future

Here $y$ represents the number of views to be predicted, and $x_1$ represents the number of viewers on the previous day

Both $w$ and $b$ represent unknown parameters, and both need to be learned by the machine from the data

2. Define the loss function

The loss function is: the average of the sum of the errors of all the data under the current combination of parameters (my personal understanding). The formula is as follows:

$L= \frac{1}{N} \sum_n^{i=1} e_n$

Here $e$ represents the error

We can use the absolute value of the difference between the predicted value and the target value: $e = \vert y - \hat{y} \vert$

We can also use the square of the difference: $e=(y-\hat{y})^2$

When the absolute value is used, it is called MAE (Mean Absolute Error)

When the square is used, it is called MSE (Mean Squared Error)

Besides the two methods above, there is another way to calculate the loss, called “cross-entropy”.

The calculated loss function can be presented visually, for example as a heat map, a contour plot… Below is the example used in the course:

3. Optimization

The purpose of optimization is to make the value of the loss function as small as possible, as in the following formula:

We need to find the combination of unknown parameters that makes the value of the loss function the smallest.

A commonly used method is gradient descent. The process of optimizing parameters through gradient descent is shown in the figure:

There are several very important concepts here:

Slope: a slope greater than 0 means the loss is rising, a slope less than 0 means the loss is falling, and a slope equal to 0 means an extremum has been reached.

Learning rate: $\eta$ can be understood as the size of a step; the higher $\eta$ is, the larger the step, and vice versa. $\eta$ can be determined by experience.

Hyperparameter: a parameter that needs to be set in advance by humans during the machine learning process.

Another important point is the update of $w$:

As $w$ is continuously updated, under ideal conditions the slope of the loss will become 0, as shown below:

When $w=w^T$, the slope is 0, but at this time the loss is not the lowest

local minima: local minimum

global minima: global minimum

In the figure above, there is an interesting phenomenon: the loss has negative values. Why can the loss be negative?

Because the loss can be defined by ourselves. Calculating it with the absolute value and square above will definitely give a positive value, but if we define it in a different way, it can be negative.

4. Finally

Finding a suitable model requires observing the data, understanding the data, and conducting experiments, and then updating the model based on the results.