Skip to content

Commit

Permalink
enrich(rnn): add 6 backslashes for a Mathjax linebreak
Browse files Browse the repository at this point in the history
  • Loading branch information
GaoangLiu committed Nov 8, 2023
1 parent 6aadf27 commit a0c0493
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 19 deletions.
21 changes: 12 additions & 9 deletions _drafts/2023/rnn.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,18 @@ RNN 引入了循环连接的思想,允许网络保持一个内部状态,以

RNN 有多种结构,比如 1 to 1, 1 to n, n to 1, n to n, n to m 等等,这里我们以 n to n 为例,介绍 RNN 的结构及其计算过程。这种情况下,输入输出长度相同,对应 feed forward 公式:

$\begin{aligned}
a^{(t)} &= b + Wh^{(t-1)} + Ux^{(t)} \\
h^{(t)} &= \tanh(a^{(t)})\\
o^{(t)} &= c + Vh^{(t)} \\
y^{(t)} &= \text{softmax}(o^{(t)})
\end{aligned}$

其中 $b,c$ 是偏置项,$W,U,V$ 是权重矩阵,$h^{(t)}$ 是隐藏层状态,$y^{(t)}$ 是输出层状态,$x^{(t)}$ 是输入层状态。

$$\begin{aligned}
a^{(t)} &= b + Wh^{(t-1)} + Ux^{(t)} \\\\\\
h^{(t)} &= \tanh(a^{(t)})\\\\\\
o^{(t)} &= c + Vh^{(t)} \\\\\\
\hat{y}^{(t)} &= \text{softmax}(o^{(t)})
\end{aligned}$$

其中 $b,c$ 是偏置项,$W,U,V$ 是权重矩阵,$h^{(t)}$ 是隐藏层状态,$\hat{y}^{(t)}$ 是输出层状态,$x^{(t)}$ 是输入层状态。在给定 $x=(x^{(1)}, ..., x^{(T_x)}), y=(y^{(1)}, ..., y^{(T_y)})$ 的情况下,RNN 的损失函数为:

$$
\mathcal{L} = -\sum_{t} \log p_\text{model} (y^{(t)} | x^{(1)}, ..., x^{(t)})
$$


# 实现
Expand Down
21 changes: 12 additions & 9 deletions _posts/2022/2022-11-06-Recurrent-Neural-Network.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,18 @@ RNN 引入了循环连接的思想,允许网络保持一个内部状态,以

RNN 有多种结构,比如 1 to 1, 1 to n, n to 1, n to n, n to m 等等,这里我们以 n to n 为例,介绍 RNN 的结构及其计算过程。这种情况下,输入输出长度相同,对应 feed forward 公式:

$\begin{aligned}
a^{(t)} &= b + Wh^{(t-1)} + Ux^{(t)} \\
h^{(t)} &= \tanh(a^{(t)})\\
o^{(t)} &= c + Vh^{(t)} \\
y^{(t)} &= \text{softmax}(o^{(t)})
\end{aligned}$

其中 $$b,c$$ 是偏置项,$$W,U,V$$ 是权重矩阵,$$h^{(t)}$$ 是隐藏层状态,$$y^{(t)}$$ 是输出层状态,$$x^{(t)}$$ 是输入层状态。

$$\begin{aligned}
a^{(t)} &= b + Wh^{(t-1)} + Ux^{(t)} \\\\\\
h^{(t)} &= \tanh(a^{(t)})\\\\\\
o^{(t)} &= c + Vh^{(t)} \\\\\\
\hat{y}^{(t)} &= \text{softmax}(o^{(t)})
\end{aligned}$$

其中 $$b,c$$ 是偏置项,$$W,U,V$$ 是权重矩阵,$$h^{(t)}$$ 是隐藏层状态,$$\hat{y}^{(t)}$$ 是输出层状态,$$x^{(t)}$$ 是输入层状态。在给定 $$x=(x^{(1)}, ..., x^{(T_x)}), y=(y^{(1)}, ..., y^{(T_y)})$$ 的情况下,RNN 的损失函数为:

$$
\mathcal{L} = -\sum_{t} \log p_\text{model} (y^{(t)} | x^{(1)}, ..., x^{(t)})
$$


# 实现
Expand Down
2 changes: 1 addition & 1 deletion assets/progress.json
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,6 @@
],
[
"2023-11-08",
143
156
]
]

0 comments on commit a0c0493

Please sign in to comment.