tensorflow+RNN的使用讲解.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"#006600\"> （1）tensorflow  常用shape 变换函数用例</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#### <font color=\"#0000dd\"> tensorflow  常用shape 变换函数用例</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "out1:  [[ 5.  6.]\n",
      " [11. 12.]]\n",
      "out2:  [[ 5.  6.]\n",
      " [11. 12.]]\n",
      "out3:  [array([[1., 2.],\n",
      "       [7., 8.]], dtype=float32), array([[ 3.,  4.],\n",
      "       [ 9., 10.]], dtype=float32), array([[ 5.,  6.],\n",
      "       [11., 12.]], dtype=float32)]\n",
      "out4:  [[[ 1.  2.]\n",
      "  [ 3.  4.]]\n",
      "\n",
      " [[ 5.  6.]\n",
      "  [ 7.  8.]]\n",
      "\n",
      " [[ 9. 10.]\n",
      "  [11. 12.]]]\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "a = [[[1.,2.],[3.,4.],[5.,6.]],[[7.,8.],[9.,10.],[11.,12.]]]\n",
    "\n",
    "a = tf.convert_to_tensor(a)\n",
    "#a = np.array(a)\n",
    "\n",
    "out1 = tf.transpose(a,[1,0,2])[-1]\n",
    "out2 = a[:,-1,:] #仅仅支持tensor 取值，不支持list, array操作\n",
    "out3 = tf.unstack(a, 3, 1)\n",
    "out4 = tf.reshape(a, [3,2,2])\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    print('out1: ',sess.run(out1))\n",
    "    print('out2: ',sess.run(out2))\n",
    "    print('out3: ',sess.run(out3))\n",
    "    print('out4: ',sess.run(out4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* <font face=\"微软雅黑\">可以将a想象成一个batch 中的两句话，每句话长度为3（3个词语，即，time sequence）, 每个词语用长度为2 的词向量表示，即输入形式为：<br>\n",
    "   [batchsize:2, sequencelength:3, inputlength:2]\n",
    "   tensorflow 中 lstm model 包含static rnn 和dynamic rnn 两种实现方式，分别对应两种不同的输入结果:<br>\n",
    "  static rnn 输入为[sequencelength,batchsize,inputlength], dynamic rnn 则直接输入[batchsize,sequencelength,inputlength]<br><br>\n",
    "* 对于rnn 计算输出的结果shape 不变，而我们则需要取出最后一个时刻输出单元的结果，对于static rnn 我们则直接取出output list 中最后一个序列即可，而对于dynamic rnn 我们则需要首先将其输出的shape 转成static rnn 输入的shape 形式即可，所以二者在输入shape 上面有较大区别，但我们最终取的结果均是output[-1]<br> </font>\n",
    "\n",
    "<font color=\"#dd0000\"> 上面的例子给我们介绍了从[batchsize,sequencelength,inputlength] 转换到 [sequencelength, batchsize,inputlength]的方法。从例子中我们可以看到transpose、unstack都可以实现数据的转换，通过tensor数据取值，我们可以获取output[-1], 但是通过reshape 我们咱不能实现数据的转换</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"#006600\"> （2）tensorflow  static rnn 与 dynamic rnn 用法示例</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1\n",
      "dynamic outputs:  [[ 0.12574106 -0.05531842  0.10649382 -0.11759173  0.0105759  -0.18829712\n",
      "  -0.12043086 -0.1059809   0.12805904 -0.01038941]\n",
      " [ 0.09083233 -0.17109962  0.09970291 -0.1301236  -0.04008966 -0.1547585\n",
      "  -0.02269071  0.08539578  0.1386207   0.04306188]]\n",
      "static outputs:  [[ 0.12574106 -0.05531842  0.10649382 -0.11759173  0.0105759  -0.18829712\n",
      "  -0.12043086 -0.1059809   0.12805904 -0.01038941]\n",
      " [ 0.09083233 -0.17109962  0.09970291 -0.1301236  -0.04008966 -0.1547585\n",
      "  -0.02269071  0.08539578  0.1386207   0.04306188]]\n",
      "\n",
      "\n",
      "epoch 2\n",
      "dynamic outputs:  [[-0.04772327 -0.01414591 -0.00145192 -0.0350657  -0.03961538 -0.02965488\n",
      "  -0.03012469  0.02317428  0.16136736  0.02352579]\n",
      " [-0.00790418  0.03858471 -0.00764068 -0.00335012 -0.01072326  0.05396807\n",
      "   0.08786124 -0.01190581 -0.03911839 -0.09422512]]\n",
      "static outputs:  [[-0.04772327 -0.01414591 -0.00145192 -0.0350657  -0.03961538 -0.02965488\n",
      "  -0.03012469  0.02317428  0.16136736  0.02352579]\n",
      " [-0.00790418  0.03858471 -0.00764068 -0.00335012 -0.01072326  0.05396807\n",
      "   0.08786124 -0.01190581 -0.03911839 -0.09422512]]\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.contrib import rnn\n",
    "\n",
    "num_hidden = 10\n",
    "vocab_size = 15\n",
    "embedding_size = 5\n",
    "sequence_length = 3\n",
    "\n",
    "#假设data 为两个batch ，每个batch 2句话，每句话3个词语，通过embeding取出每个词语index 对应的初始化词向量\n",
    "data = [[1.,2.,3.],[4.,5.,6.]],[[7.,8.,9.],[10.,11.,12.]]\n",
    "\n",
    "\n",
    "# dynamic rnn\n",
    "input_x = tf.placeholder(tf.int32, [None, sequence_length])\n",
    "\n",
    "#with tf.variable_scope('W',reuse=tf.AUTO_REUSE):\n",
    "W = tf.Variable(tf.random_uniform([vocab_size, embedding_size], -1.0, 1.0), name=\"W\")\n",
    "embedded_chars = tf.nn.embedding_lookup(W, input_x)\n",
    "\n",
    "lstm_cell_ = rnn.BasicLSTMCell(num_hidden, forget_bias=1.0)\n",
    "\n",
    "#动态rnn 函数\n",
    "with tf.variable_scope('dynamic_rnn',reuse=tf.AUTO_REUSE):\n",
    "    dynamic_outputs, dynamic_states = tf.nn.dynamic_rnn(lstm_cell_, embedded_chars, dtype=tf.float32)\n",
    "dy_output = dynamic_outputs[:, -1, :]\n",
    "\n",
    "\n",
    "#静态rnn 函数\n",
    "static_input = tf.unstack(embedded_chars, sequence_length, 1)\n",
    "\n",
    "with tf.variable_scope('static_rnn',reuse=tf.AUTO_REUSE):\n",
    "    static_outputs, static_states = tf.nn.static_rnn(lstm_cell_, static_input, dtype=tf.float32)\n",
    "sta_output = static_outputs[-1]\n",
    "\n",
    "init = tf.initialize_all_variables()\n",
    "with tf.Session() as sess:\n",
    "    sess.run(init)\n",
    "    for idx, batch in enumerate(data, 1):\n",
    "        feed_dict = {input_x: batch}\n",
    "        print('epoch {}'.format(idx))\n",
    "        print('dynamic outputs: ', sess.run(dy_output,feed_dict))\n",
    "        print('static outputs: ', sess.run(sta_output,feed_dict))\n",
    "        print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* <font face=\"微软雅黑\">上面的例子详细介绍了dynamic rnn 与 static rnn 的区别，从结果可以看到二者的结果完全一致</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}