Skip to content

Commit

Permalink
Bump version and add more docs
Browse files Browse the repository at this point in the history
  • Loading branch information
ixaxaar committed Nov 1, 2017
1 parent f8fc2f5 commit f18e591
Show file tree
Hide file tree
Showing 7 changed files with 12 additions and 1 deletion.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ This is an implementation of [Differentiable Neural Computers](http://people.ids
pip install dnc
```

## Architecure

<img src="./docs/dnc.png" height="600" />

## Usage

**Parameters**:
Expand All @@ -20,6 +24,7 @@ pip install dnc
| hidden_size | None | Size of hidden units |
| rnn_type | 'lstm' | Type of recurrent cells used in the controller |
| num_layers | 1 | Number of layers of recurrent units in the controller |
| num_hidden_layers | 2 | Number of hidden layers per layer of the controller |
| bias | True | Bias |
| batch_first | True | Whether data is fed batch first |
| dropout | 0 | Dropout between layers in the controller |
Expand Down Expand Up @@ -91,3 +96,8 @@ The visdom dashboard shows memory as a heatmap for batch 0 every `-summarize_fre
1. DNCs converge with Adam and RMSProp learning rules, SGD generally causes them to diverge.
2. Using a large batch size (> 100, recommended 1000) prevents gradients from becoming `NaN`.

Repos referred to for creation of this repo:
https://github.com/deepmind/dnc
https://github.com/ypxie/pytorch-NeuCom
https://github.com/jingweiz/pytorch-dnc

Binary file added docs/dnc.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/dnc.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<mxfile userAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36" version="7.6.7" editor="www.draw.io" type="device"><diagram id="07843baa-5123-7ce6-3a0d-b229d29ab99d" name="Page-1">7T1Ze6LKtr8mj2d/MhY8MghomByiwMv5BAoEFZQZfv2tUtOddLLP6Xt2x6TTsdMKi1UFtea1qoA7Sjp0arE5bo08hPs7chR2d5R8R5IUyVHoB0P6C4QYcdwFEhdJeIGNvgMWyQAfEa/QOglh+QyxyvN9lRyvQOICDPIsg0H1DLYpirx9jhbl+/AZ4LiJ4bPeMWARbPbwBdo6CavtBcqR4Dtcg0m8fTwzwfKXI/4m2MVFXmfX892RVHT+XA4fNo99Xc9bbjdh3j4BUeM7SiryvLpsHToJ7jFxn5NN+Zuj3667gFn1Mw2Y63U0m30NHy/5fGFV/0iMcFNuIcYf3VHitjrs0SaBNtG1HzHKoYuxGPy1aUvqLxiQ/w7yw7GuIPrNqk2SweLfaFxilOz3Ur7Pi3OvlMJwDEUjOGocJuhyH49leYZ6FVEnSfB4pqrId/BJ49H5gztFp7iKD8Fe95/gEQL+h+DXUcKigt3fkor4xgAk2TA/wKroEcq1wb8Ay1zaXKWaBexlv/0uIuToStDtM/G4AjdXsYy/df6dNWjjyp3XOUXzX5z6aU6BHzjF3pRT7Ben/ledokc35RTzxan/Vaco/pac+uYvvzMGhshlX3fzotrmcZ5t9uPvUDGoi+bMOkzFs1P+xsin9H3CVNgllYNR/mKue+6TIzYsEnTlsLh2gkhX9E/Q8a779NiPDb7JEu4xhVXVX7m3qascgb4PQ8/z49+xnxckICrfjjwGKMz/WyAwBf+zOCCC53URXLGuDK82RQwfsfjXpaaA+02VNM+7/ycCQFEvVHVdJBW8qVgQT4Ri9BfJ/Fe5eCYVlwafUAb+xnL8chn48qs/b625H6w1d0trTb1kzJta62dqyTzTwL/AH2GaH7n2VC/ZW5lm8EIvkboVEOe78JBfO3tH1/0f3PQnZPvNzDHxE7WDR7ubHM4VF3FTHi/1mijpMN3F6wE53FSbO0q47JLKMYvvSClZida8Hd2rcS6gj7l42I4fYrQVG+jLGEuCi+HJaeH5eEPUTGmxmk0kIZ5EwnaXnIH70Xy1HT2Q/CHUwm1weBA8kmn89UMVkGbvrVd1QG6b0OLkZqBRgx4sHubiSksCEHpiMDPFsbAUNXedq/dVNEuCmZGXnWQ3iXZ8mOb3VmSu0CVHxTByyHA0GERkCi0avUJre0/yumEoeyYUOeGOFHVbo2aBEGv1VJhJk3FrCGMpUAVJtGRBFDIxFmI57tBldy4aKV3O0EAXnw13f9+6wmTMPSIIljwWBE1BDJMThCB0k50oTOKl0Aqz6RmX/ry4aqed6bP9UDx6D9xyFWOi0IgoAm0glGA9EQRLRwSUx8lsLCSi8a27ZfxIwC/cL9wv3D8G1zxMhJn80WzX7XGniwr7F/t38HFviiu76dl3th+PR1+4X7jvjauLHrar8Hew7V+4N8CVWm2J7er+N7Dtb4tbKjusR8xvo8tvgEvp7R2p2FRDSTM5nmaNpbOirEJRWE6pApdQhPFeWe4W9ewgSb+oQE3xzwvUgH9RnyYo5mV9mhyRv6BiSb+oXM3hJnzHojV4VrUmnpUpnxz8bFVr4h2r1txX1foDsZ0gb1W1Jt6tah1x6EtnLlVrdKHiTBqh719WuRbH9PfKdWr3pEPNgqgDIKOG03K5VMPkgK7yJLvHSinmymK9e3g4oSsQ1bEG2aCuI7pkTYQSU3UNDL8ip8r8Ptlv9mo7F8fL02LWjfq5Oc0OEBLtuPRnZrhlrAyTv4Y+DxPQoP4iOaMyG/IeAmelhr6pqdV48WFJCsmpAQxB8BTNNLpqeyNai83joK0N1oE8F3gxopLKOpgwppDasTyboMGNJOMzQGSD9Q9NI69A2/voICKNE9UDw7an6IGG7ThiLDuyJsPe1holLMBwWGpq4O14oQf3dDR2Ow4OK74PIjy/gPy4jU8hCinmmjCbfEH/IVQc1C3HDDzbJs66LPuw3CDzIWakFy2pTd0ByyLVtWWUiz4Loo0Z95BMDw65FUtARZS8IsOGNJbYzIjAzgrpyDg947BFU9F0pzfYUIgzFdsrdPYwUqmocbCcwNLbnaEiXUVqGDpHfjdU9ZiS9XDXzJnUMQgYOCZu334cif4YkMHKODg98O68KYAw9kneZPaFWnBEz8Nu0HnYrlSi0KITsJ0SiFsegIqj9lGtrpGpURreBiDgUQiopMQ8Ivn5NKVCswubMbKH4sDZDfUAtzhOLpWwQoGp4tv6esIBdHRuCBU54wsx4kHTS0cUX5u99mEk+nNAxQE2hR4NJrKcHsTeltGxdrLDHLQTf03OLJKPpIjqEu3E74qqsfN6Fqa6bjq+aoTuEGZ0YFe86fZpjZobTqYMmPW7DyXJHwUiG5uIWiikh6zZnD0URLxm68ib85uL6aIYkTSYgWC7LLJCI/A6Pj7RxiTsRth3RzoVs4ImLGnkzw04sQVZS+RW6P6I/U2k4fn/aI4J5ZDxPVWBnQwCZ0tpQcXwKEDXVcPDsRfn21NtyJy1gbJw0YHWfeg01Lbhc6/SdUPjJto6WpHw2LFhhzAUFre29Rz7l7qU2gVJwyzmh5hr+JHapsgiKbxgSPbSbsXJ2OeEhYjdjTCO582fC2GaRbcELQM54+BkDcdb7ZQkjpnMp5aReURma3DJAK2sQTa3ms4yvT22PFBcA9UyNqxDLqoa2DMdRbiK2pRDWNKncwiH/0q9lDyU0BwCpC8Mk9rYlQhwGLOYHwSUBo1c1GQJGNqjhyFN6Bk18QfX7Si9wS6dC/NIUPWx1opJJ3KCW87+gP2phZIBatTwBJ9LJoA7rufTFfT8RithD6ytwKJ93tX8QN5C5BhEO2RRXOTXw6hPG8rDmYfbafcMjrckbY38gUJ2GdrxgG376cB3XAjJAivFIlyDVt5KIcJVONunyfsIZjKVdtXQfwwv9zmgyFdnIot99EZdbgbdyhqpO/qeMGxIzJpqqHF4RBRsNVgThnB6KGmgKThS5S7qxHR83voWNp+HEnOrx3qLNmpgMaRnk6YPs/IBWOigyOugJeUy4tHx6ZZhO9SJYoX+VcsmVJOhaMAuyAdgZtQ0dd6fQp8Sqpks5hCbkvOSLBdnTSyWXOjtkQ/cjgmeqTOnq/cj22lUb0/ztlzXAzCdVl15aR12SOkJyG3MzAHMBPdlUoVkNj4lk2bFhM28YJqDRnWMTflZgMUBgCii2tpReMtgoTCP9tAKOLkJan44tBU3EwSbs7kZFFR7rMVi2om04H7ufckdDUFk40h58K2yDUt/OuLdsa2T5opxsgIcHC1rYOMZgzc4R5RqznHWKcMQV79EmyMOGZmifBYRGI5AxvQor7J1astAayjs8lzkUVrZTeZRFGlusGxqZmZFg7uKIhPEJsNFWrk513AQos4Fx0AW5CD5cBL7m0JFlLeIzRFHmFpBbg9EMMzZtozsUnD6WUWwrM14+2xJpBYR6apcsnLVq8xyUGGphZZDEZ2HY/RmPV4esT3e8/ncBJsdvp/mbDlHAfLDc2RQRTJlhxR1sG1VDjhu1697QHSN7X4Qab/h/jRdNl6YLflUhSNsnZxS0NaHAHmjDb/imJQMVY9qUi8GnMjdz3CWKe5xWJ/v0eb6gHOpWZFhP6VZEXDqE9S4ZaFmJ2DFFA7o1RwWOE7EbtC3KZEqEBfDiR/vcc7VNu4YHZgKM+wjReuDZT032Vcz7L6L0OcALCe4YhYhWhwUcm6R4Yo56CieT3O677bx3TlRdcoGx41mn6VEavQXm4T+qp52mpEdS7DTKtg5jTSAaHnUsVfJgED6a5sv8LmUqDi4iHMsmyoUL2hpm9A4BchCmwBihsu4QTOoSpTIWCp2hgTYGm6sDyW7N/M8OFhjg6vvnrYcdMi5TQtaDYwECz6zstN6ymQDn+KaQzPvdHRsYYtpVfSUYy9lxIXeufDoAfPBsgdSXOHUVyMn/LBn2zxaJ4AZR5wV+wM9yOfqHpaKlbfoM3s12taCkxmeh7KKxOf4PzGz0rGFAJJizgVWcaX6RvP41OM91f99Iv9XTOK+vCXwayL/PWZ0yVdmdG91azBNviIEJboMNI7uiAgNs+A97xP+TxLxCe4IfpX3o3/I+3NToSg2/ROEY55kVfmkZxsDnhgjgvxhUdF1bl/52QYEN/pB8i7X8F0Ovw3m5xYbvBTNWy02CM6LDXBR7LzcQNar46W08uuWG8jz78sNEr44+M4siLZVwDNGxwCgpaWgeBNpsktnwhSfWorjWYJC2cWDEnLiKhDGEz9HqY34MFaUmfvwIMCpvBPihdnP55Nd0GmI3+lADMPJjccugWuq90ul0OXeNnWc/4wagGMxCCMH50TpJWwYaXJJd0NKWkKGdmW+5wK9Ytv90vFxnbDKUtnk4Ny11+oJh9q6wp/MPOsDVZjSC262EBZyK07EqS1M0vG4FZLfFcba94yNUwsalsZDivSfFLXThUxMZhXeCNEp1VQe5qtousBxF6k2FEM0jXMqMZ2yZejoTFpwbgOcaTWk8pi0Agb6rSBAoXj3Ed6EipoO7VXYHCzbsgK261eZqlqGazHEwWICsCQ8+2DlwZ5/WB7Tg03MmgNZ53xpKgAssRayhnaSV3ZgF+3gF6ADVu5xTJdqYs2LS5ftt9s0xfaEp8OrhK72FzaxCVtmS9uGCYDFdNQR/DoZZHrhN0JJ1n0ucBpv28I4Hu84ITFmS0HSOzSEyWeC1ceh0diSGmyN3uTdwLvJdCg0By59rPimRrDxfgDOoqE2NDTMvYUV3XVJZsAV5/vRmZbi1KjSRqfrpYZNaN1essFRErloayCctaSjrF3scOEZ55V2E4KZj1NCyU4jn1svibjILJxF5ko03fL4pk04/kO0wNY7XPnrKWJeH6C9POKJK3t7JJjQXlvQpbsOESjJq2aF73Xe4BIuWGunPjR4KGd2wbBtunBCIkCa1Eyb8EGfptVaZlI5YmjkDXaZMF4m0kyaQ2EqTKefESbPGeg5dsGzbTejdII0OdIqVhyyw0iel0wdDqmuQsvQdbZHkrxmRpHD0PcrJPOLY3qsUn42wzI63tiygJQfTtApFFrYCbssFmbJIhDm1meDiSOIV7wpdlQwjH0oAqTlLVZa89TyZM3XFjGzDrVPDDVVSCslakCk4vmjk5LuqKbiSrehCJJ33cJngYb74hVAqTbacIG9tm0zLrpoZNkeA1UGNvsd725OBV7fwwx+4KkpR6ir9kHIYvDu1vBtYXIe1NwSsAAaxnrJkScuimyD3WbcQUu5vqRO0srCixEU5yRFdhmuM7V2CmFuBF1XZNNkwWIz0eDln0VFt92qtLPtyG6ak0cskaMMAtrfQ7SxRRGYVIt4ogH/r/PpA0M21ArR3ivSGEfj4kmlRtgm+0HOVKk85SthISyXrTTpREOYNGN0+TGHYIvPBNN3fJ3yD/6xiHkUetErHLi5rNiBbB1QgBzImrIybH+J05EzS+9k6gGVta1vFeZo6KjaADTYhs36PgnoLs27qjrS4QFFb/UlrBg45DGXdpRNowWYBZA/IPvCUMWpdFLZtgIXe0gWML5QCgcnRsZsAQVRmk5bcUzHSEk/E0ybM3SkQzYmlo4yilDykCF5X2pG4DLUocxKRau1YstBLKFUWmaFZFpRxcarAUTm+pJZKcCP1vfWArs9rrOaIkSy78PG2QIcS5hJtsLT1S6TDpqlMuuC2eYlz4k8OCcxIYodsyJzCMZDeQnEa4cabixuU99eIawBNEJO1kmLTP5i8iGo9sawEVw5GbUvkNFGhKGGQd46KZOWa5mI84qsBxjVEc5YiO39HmpYQ84ZCx9YYXYc+OxIoNQFjCi82KCiUlmh3JDCRej8Eg1mfrkK0O/Z4+Io0QEUMcJ2v9H0S840Ge0BrFejs/Y14zPaasDtI2FZ7MT4PpQ+juV+o9i7Gaqa4d1mrd4XA8EG7uATi/Jg4eS67ZDsijsfMFjIlfW4Xm5YggS1ny1cw9+wZlqHTDR1ZIpnMScd3KrBjWyXNfQynFqmEWw6WIo7h8LznA8o3+FEpQY1RXAKdu/UkXRtu/ow0cgbwqbhxTzzbrYisdTR9RiEHKTXPpFkNTZIfYGfNyTOtQzg3AfZDaY4oghcgshXD9gSReRaJwWZ0piOD87CX8iaHKaOySN6mxXfDA1Tao5D5Ejai5TauRRlr1Svuy5Bl1Q+oSmLLozq40XQbxGRg+LUR0awYk0OLjxi0dBBpJU8yayXTHoC6wjXUOYEH2kStjEOgLm4Y8hC5LoORipBzG2UEQ2Bel3ZD2TaHpaZuqo4FICO4EHkWc5/yM65UCRmfJPtYbTK8zbMUk3FTAKDE8ZZpio1n8VazM7FD5H1vS1MSnrsLnlqHzo+kyZphKhJXWjYYKrUiDw4GnHx4td8FU3mOGRvWBQknm3ysWqa4ojzSgpn/NFJcswqPHahI4NhiwOkElt+xqUAjVxIyVYZ8r5WEPa1hnquw6yJbRwScdBHbW1+QJwdE2SjpZIHbWbCVbv1R7DCbx3zcyfsGfnmEj/y9doLHKdBvjYtclzw6wuFGwiAiyE0b6E4qKPsgqDbYbSxh/k2TS27Q/EQdzAqqmn0exxLNdqFk5v9gNeSpOw3nQjzJjtcq7Ucjvs7IqKsyEPREzpPR1AWyrqcTT+CpSdMaevjSOwb1VZ0ToQJh4U8Meq59HCer108rKz5PSO5kwmeLPhVU7bEs0kPlnj5bFCC5F6ZsiV+xf2X9Ae+sfZTzs1S73eTNfHyqc2T7FhXL0TgO08x8dptUsHFcXMeQltsjs95+4whP/ns3ecP9JUoQZRv/6z70auTjc/07vGFHE/17vHtBf/oAerMe6vdKw/K/t01i3+pWSR9I9Ui+RszlPxTOcrdiKPUy7dRfD3i/oMIwc0eT0HdWq3BP33TxSfk9u3io6+XOv3PrzRgmZu+gOblI6OsuvoAoexYYEWWvTUvfghlefKVFPLNQtm/W/SLBhVUeVG+N0v48+fGLGGY5+rBv/LGjzdjCXXj1zPd/e362h/ikc+64PqxPPMeST358slpEhpcke/3iLxIuJIwhNlZGjavRK83VkYWCCKv3FgZ+fdURvrlezmM159oh0ZYPad3Actk2PhnBEzM65JrhM2IdwyuQGL9KK9kxLv7JM7Q9h5GuCtMtiTY7IUruMK6I5aIv0kWL8+K9C/6xsygWPIZMwjqJTPAK7wgf0HYQL98N0521klk/M4DeKIu+00PX3FefwaTAHgeUTCvMIng34pLLy3aF5dej8Gfcwnw4M24hBePfXsP8uXei+9vm6bG/wc=</diagram></mxfile>
Binary file added docs/dnc0.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/dnc1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/dnc2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
setup(
name='dnc',

version='0.0.4',
version='0.0.5',

description='Differentiable Neural Computer, for Pytorch',
long_description=long_description,
Expand Down

0 comments on commit f18e591

Please sign in to comment.