Structure and Principles of Recurrent Neural Networks (RNN)

An RNN (Recurrent Neural Network) is a type of neural network that processes data while maintaining the sequence of inputs.

In this lesson, we will explore how an RNN is constructed and the principles behind how it remembers order and processes information.

An RNN is composed in a repeating form of neural network cells with the same structure.

Each cell receives the current input along with the previous time step’s output, processes them, and passes the result to the next time step.

Copy
Input X1 → [RNN Cell] → Output H1 →
Input X2 → [RNN Cell] → Output H2 →
Input X3 → [RNN Cell] → Output H3 → ...

The output at each time step utilizes the output (H) from the previous step, remembering it.

An RNN repeats the same computational structure along the time axis, updating information at each step.

For example, when processing a sentence, inputting words sequentially from the first word to the last, the information from earlier words can influence how later words are processed.

Copy
[t=1] → [t=2] → [t=3] → ...
 X1      X2      X3
  |       |       |
  ↓       ↓       ↓
 H1 →    H2 →    H3

Thanks to this structure, sequential data such as words, speech, and numbers can be naturally understood and predicted.

An RNN cell consists of three main components:

• Input (X_t): The input data at the current time step.

• Hidden State (H_t): The internal state that stores and summarizes past information.

• Output (Y_t): The final output or prediction at the current time step.

An RNN calculates a new hidden state (H_t) by using the previous hidden state (H_t-1) along with the current input (X_t).

This hidden state is reused as memorized information when moving on to the next input.

In the next lesson, we will discuss the Long-term Dependency Problem, which is a representative issue in RNNs.