UMass-Hackathon-2017.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# UMass Med DNA Hackathon\n",
    "## Chosen Project: Mystalign\n",
    "\n",
    "\n",
    "## Problems to be tackled\n",
    "- User-friendy tools to analyze their data\n",
    "- Sometimes they don't know how to formulate the questions they want to answer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import pandas as pd\n",
    "from Bio import SeqIO\n",
    "\n",
    "dna_df = pd.read_csv('data/mystalign.txt', sep='\\t', usecols=range(4), header=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dna_list = list(SeqIO.parse(\"data/c_elegans.PRJNA275000.WS260.genomic.fa\", \"fasta\"))\n",
    "record_dict = SeqIO.to_dict(SeqIO.parse(\"data/c_elegans.PRJNA275000.WS260.genomic.fa\", \"fasta\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>seq</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ATTCATCATTTGCCCCGCACAG:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ATTCATCATTTGCCCCGCACAG</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACACAGATGCTCTATTGGATTCA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ACACAGATGCTCTATTGGATTCA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GCTGGTCAGAAGGCTCTTGGGT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCTGGTCAGAAGGCTCTTGGGT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GTATATATATGTGCGCGAGAACA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTATATATATGTGCGCGAGAACA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GCAGATTACTTGGAGTCAACCT:8</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCAGATTACTTGGAGTCAACCT</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GACGAGTTGATTTAATGTAGNN:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GACGAGTTGATTTAATGTAGNN</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GTGGTTTTTATGTGGATCTGATTT:5</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTGGTTTTTATGTGGATCTGATTT</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GCGCTTCTCGTAATCTCTTGATTTT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCGCTTCTCGTAATCTCTTGATTTT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GAAAATGTATTGCGGATTGTTAT:2</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GAAAATGTATTGCGGATTGTTAT</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>GTTGTCTTCAAGATGTTTGTT:29</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTTGTCTTCAAGATGTTTGTT</td>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>ATCATTCATTTGGGAACTTGGTAA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ATCATTCATTTGGGAACTTGGTAA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>GATGATCAAGACATTTTGTTTGAA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GATGATCAAGACATTTTGTTTGAA</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>GTGGTGTTGTTATGATTATCAG:9</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTGGTGTTGTTATGATTATCAG</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>CGACGATCGTTCTGGATGTTGTAAGCT:3</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>CGACGATCGTTCTGGATGTTGTAAGCT</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>TACCTGATTGATTGAGCG:5</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>TACCTGATTGATTGAGCG</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>TGTGACCTTTTTGT:2</td>\n",
       "      <td>272</td>\n",
       "      <td>R09E12.2|srbc-58|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>TGTGACCTTTTTGT</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>GGATTGGGAGGTGAATAGTTAT:7</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GGATTGGGAGGTGAATAGTTAT</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>GATATCGCCGTCTTCACTGAATTT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GATATCGCCGTCTTCACTGAATTT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>GTATTGTTTCCGATATTTGACT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTATTGTTTCCGATATTTGACT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>GTGTTAGGTCGGTGGTTGCA:4</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTGTTAGGTCGGTGGTTGCA</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>GATCCGATTCCTCTGATGCG:3</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GATCCGATTCCTCTGATGCG</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>TCGTATGGATTGCAACCGTTTTC:3</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>TCGTATGGATTGCAACCGTTTTC</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>GAAAAAAAACCGGGAACACTTTTG:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GAAAAAAAACCGGGAACACTTTTG</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>GATAGTACGCTCTATTGGACTTT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GATAGTACGCTCTATTGGACTTT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>TCAATCACAACGACATCGTCCTT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>TCAATCACAACGACATCGTCCTT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>AGATTGGGCTCCTATCCTGGAA:2</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>AGATTGGGCTCCTATCCTGGAA</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>GTTTCTCTGCCGATATCTTTCA:2</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTTTCTCTGCCGATATCTTTCA</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>CACTGTTGGTTCCTCTATCTCGTG:4</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>CACTGTTGGTTCCTCTATCTCGTG</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>ATTTGATGTGAACAATTTGTGA:6</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ATTTGATGTGAACAATTTGTGA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109025</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>F08B6.3|calu-2|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109026</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>M110.10|M110.10|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109027</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y51H4A.20|Y51H4A.20|pseudogene</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109028</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y55B1BR.4|magu-1|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109029</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>B0513.69|21ur-12444|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109030</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>C46F11.5|C46F11.5|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109031</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y73B6BR.27|21ur-11539|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109032</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>K08G2.8|srh-293|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109033</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>W04D2.9|W04D2.9|ncRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109034</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y67A10A.269|21ur-11794|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109035</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>M01G5.9|M01G5.9|ncRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109036</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>F36H12.43|21ur-6283|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109037</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>T27E7.16|21ur-5242|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109038</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>F47E1.14|F47E1.14|ncRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109039</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>transcript|F07E5.1|protein_coding|fbxb-6|F07E5.1</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109040</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>C04A11.8|C04A11.8|ncRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109041</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>F39H2.2|sig-7|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109042</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>Y6E2A.10|Y6E2A.10|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109043</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>C18C4.4|C18C4.4|pseudogene</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109044</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>F28F8.13|linc-108|lincRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109045</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>Y61A9LA.13|Y61A9LA.13|ncRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109046</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>R160.2|R160.2|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109047</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y73F8A.698|21ur-14179|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109048</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>Y51A2D.19|slo-1|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109049</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>B0513.87|21ur-7913|piRNA</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109050</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>T01G6.3|str-196|protein_coding</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109051</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>256</td>\n",
       "      <td>transcript|F18C12.3|protein_coding|F18C12.3|F1...</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109052</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>K12D9.13|srw-131|pseudogene</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109053</th>\n",
       "      <td>AAAAAAAAA:6</td>\n",
       "      <td>272</td>\n",
       "      <td>T08G5.14|T08G5.14|pseudogene</td>\n",
       "      <td>4</td>\n",
       "      <td>AAAAAAAAA</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1109054</th>\n",
       "      <td>GTACAACCTTGCAGGAGAGAATT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTACAACCTTGCAGGAGAGAATT</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1109055 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                         0    1  \\\n",
       "0                                 ATTCATCATTTGCCCCGCACAG:1    4   \n",
       "1                                ACACAGATGCTCTATTGGATTCA:1    4   \n",
       "2                                 GCTGGTCAGAAGGCTCTTGGGT:1    4   \n",
       "3                                GTATATATATGTGCGCGAGAACA:1    4   \n",
       "4                                 GCAGATTACTTGGAGTCAACCT:8    4   \n",
       "5                                 GACGAGTTGATTTAATGTAGNN:1    4   \n",
       "6                               GTGGTTTTTATGTGGATCTGATTT:5    4   \n",
       "7        GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...    4   \n",
       "8                              GCGCTTCTCGTAATCTCTTGATTTT:1    4   \n",
       "9                                GAAAATGTATTGCGGATTGTTAT:2    4   \n",
       "10                                GTTGTCTTCAAGATGTTTGTT:29    4   \n",
       "11                              ATCATTCATTTGGGAACTTGGTAA:1    4   \n",
       "12                              GATGATCAAGACATTTTGTTTGAA:1    4   \n",
       "13                                GTGGTGTTGTTATGATTATCAG:9    4   \n",
       "14                           CGACGATCGTTCTGGATGTTGTAAGCT:3    4   \n",
       "15                                    TACCTGATTGATTGAGCG:5    4   \n",
       "16                                        TGTGACCTTTTTGT:2  272   \n",
       "17                                GGATTGGGAGGTGAATAGTTAT:7    4   \n",
       "18                              GATATCGCCGTCTTCACTGAATTT:1    4   \n",
       "19                                GTATTGTTTCCGATATTTGACT:1    4   \n",
       "20                                  GTGTTAGGTCGGTGGTTGCA:4    4   \n",
       "21                                  GATCCGATTCCTCTGATGCG:3    4   \n",
       "22                               TCGTATGGATTGCAACCGTTTTC:3    4   \n",
       "23                              GAAAAAAAACCGGGAACACTTTTG:1    4   \n",
       "24                               GATAGTACGCTCTATTGGACTTT:1    4   \n",
       "25                               TCAATCACAACGACATCGTCCTT:1    4   \n",
       "26                                AGATTGGGCTCCTATCCTGGAA:2    4   \n",
       "27                                GTTTCTCTGCCGATATCTTTCA:2    4   \n",
       "28                              CACTGTTGGTTCCTCTATCTCGTG:4    4   \n",
       "29                                ATTTGATGTGAACAATTTGTGA:6    4   \n",
       "...                                                    ...  ...   \n",
       "1109025                                        AAAAAAAAA:6  256   \n",
       "1109026                                        AAAAAAAAA:6  272   \n",
       "1109027                                        AAAAAAAAA:6  256   \n",
       "1109028                                        AAAAAAAAA:6  256   \n",
       "1109029                                        AAAAAAAAA:6  272   \n",
       "1109030                                        AAAAAAAAA:6  256   \n",
       "1109031                                        AAAAAAAAA:6  256   \n",
       "1109032                                        AAAAAAAAA:6  256   \n",
       "1109033                                        AAAAAAAAA:6  272   \n",
       "1109034                                        AAAAAAAAA:6  256   \n",
       "1109035                                        AAAAAAAAA:6  256   \n",
       "1109036                                        AAAAAAAAA:6  256   \n",
       "1109037                                        AAAAAAAAA:6  256   \n",
       "1109038                                        AAAAAAAAA:6  256   \n",
       "1109039                                        AAAAAAAAA:6  256   \n",
       "1109040                                        AAAAAAAAA:6  256   \n",
       "1109041                                        AAAAAAAAA:6  256   \n",
       "1109042                                        AAAAAAAAA:6  272   \n",
       "1109043                                        AAAAAAAAA:6  256   \n",
       "1109044                                        AAAAAAAAA:6  272   \n",
       "1109045                                        AAAAAAAAA:6  272   \n",
       "1109046                                        AAAAAAAAA:6  256   \n",
       "1109047                                        AAAAAAAAA:6  256   \n",
       "1109048                                        AAAAAAAAA:6  256   \n",
       "1109049                                        AAAAAAAAA:6  272   \n",
       "1109050                                        AAAAAAAAA:6  272   \n",
       "1109051                                        AAAAAAAAA:6  256   \n",
       "1109052                                        AAAAAAAAA:6  272   \n",
       "1109053                                        AAAAAAAAA:6  272   \n",
       "1109054                          GTACAACCTTGCAGGAGAGAATT:1    4   \n",
       "\n",
       "                                                         2  3  \\\n",
       "0                                                        *  0   \n",
       "1                                                        *  0   \n",
       "2                                                        *  0   \n",
       "3                                                        *  0   \n",
       "4                                                        *  0   \n",
       "5                                                        *  0   \n",
       "6                                                        *  0   \n",
       "7                                                        *  0   \n",
       "8                                                        *  0   \n",
       "9                                                        *  0   \n",
       "10                                                       *  0   \n",
       "11                                                       *  0   \n",
       "12                                                       *  0   \n",
       "13                                                       *  0   \n",
       "14                                                       *  0   \n",
       "15                                                       *  0   \n",
       "16                         R09E12.2|srbc-58|protein_coding  4   \n",
       "17                                                       *  0   \n",
       "18                                                       *  0   \n",
       "19                                                       *  0   \n",
       "20                                                       *  0   \n",
       "21                                                       *  0   \n",
       "22                                                       *  0   \n",
       "23                                                       *  0   \n",
       "24                                                       *  0   \n",
       "25                                                       *  0   \n",
       "26                                                       *  0   \n",
       "27                                                       *  0   \n",
       "28                                                       *  0   \n",
       "29                                                       *  0   \n",
       "...                                                    ... ..   \n",
       "1109025                      F08B6.3|calu-2|protein_coding  4   \n",
       "1109026                     M110.10|M110.10|protein_coding  4   \n",
       "1109027                     Y51H4A.20|Y51H4A.20|pseudogene  4   \n",
       "1109028                    Y55B1BR.4|magu-1|protein_coding  4   \n",
       "1109029                          B0513.69|21ur-12444|piRNA  4   \n",
       "1109030                   C46F11.5|C46F11.5|protein_coding  4   \n",
       "1109031                        Y73B6BR.27|21ur-11539|piRNA  4   \n",
       "1109032                     K08G2.8|srh-293|protein_coding  4   \n",
       "1109033                              W04D2.9|W04D2.9|ncRNA  4   \n",
       "1109034                       Y67A10A.269|21ur-11794|piRNA  4   \n",
       "1109035                              M01G5.9|M01G5.9|ncRNA  4   \n",
       "1109036                          F36H12.43|21ur-6283|piRNA  4   \n",
       "1109037                           T27E7.16|21ur-5242|piRNA  4   \n",
       "1109038                            F47E1.14|F47E1.14|ncRNA  4   \n",
       "1109039   transcript|F07E5.1|protein_coding|fbxb-6|F07E5.1  4   \n",
       "1109040                            C04A11.8|C04A11.8|ncRNA  4   \n",
       "1109041                       F39H2.2|sig-7|protein_coding  4   \n",
       "1109042                   Y6E2A.10|Y6E2A.10|protein_coding  4   \n",
       "1109043                         C18C4.4|C18C4.4|pseudogene  4   \n",
       "1109044                          F28F8.13|linc-108|lincRNA  4   \n",
       "1109045                        Y61A9LA.13|Y61A9LA.13|ncRNA  4   \n",
       "1109046                       R160.2|R160.2|protein_coding  4   \n",
       "1109047                        Y73F8A.698|21ur-14179|piRNA  4   \n",
       "1109048                     Y51A2D.19|slo-1|protein_coding  4   \n",
       "1109049                           B0513.87|21ur-7913|piRNA  4   \n",
       "1109050                     T01G6.3|str-196|protein_coding  4   \n",
       "1109051  transcript|F18C12.3|protein_coding|F18C12.3|F1...  4   \n",
       "1109052                        K12D9.13|srw-131|pseudogene  4   \n",
       "1109053                       T08G5.14|T08G5.14|pseudogene  4   \n",
       "1109054                                                  *  0   \n",
       "\n",
       "                                                       seq count  \n",
       "0                                   ATTCATCATTTGCCCCGCACAG     1  \n",
       "1                                  ACACAGATGCTCTATTGGATTCA     1  \n",
       "2                                   GCTGGTCAGAAGGCTCTTGGGT     1  \n",
       "3                                  GTATATATATGTGCGCGAGAACA     1  \n",
       "4                                   GCAGATTACTTGGAGTCAACCT     8  \n",
       "5                                   GACGAGTTGATTTAATGTAGNN     1  \n",
       "6                                 GTGGTTTTTATGTGGATCTGATTT     5  \n",
       "7        GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...     1  \n",
       "8                                GCGCTTCTCGTAATCTCTTGATTTT     1  \n",
       "9                                  GAAAATGTATTGCGGATTGTTAT     2  \n",
       "10                                   GTTGTCTTCAAGATGTTTGTT    29  \n",
       "11                                ATCATTCATTTGGGAACTTGGTAA     1  \n",
       "12                                GATGATCAAGACATTTTGTTTGAA     1  \n",
       "13                                  GTGGTGTTGTTATGATTATCAG     9  \n",
       "14                             CGACGATCGTTCTGGATGTTGTAAGCT     3  \n",
       "15                                      TACCTGATTGATTGAGCG     5  \n",
       "16                                          TGTGACCTTTTTGT     2  \n",
       "17                                  GGATTGGGAGGTGAATAGTTAT     7  \n",
       "18                                GATATCGCCGTCTTCACTGAATTT     1  \n",
       "19                                  GTATTGTTTCCGATATTTGACT     1  \n",
       "20                                    GTGTTAGGTCGGTGGTTGCA     4  \n",
       "21                                    GATCCGATTCCTCTGATGCG     3  \n",
       "22                                 TCGTATGGATTGCAACCGTTTTC     3  \n",
       "23                                GAAAAAAAACCGGGAACACTTTTG     1  \n",
       "24                                 GATAGTACGCTCTATTGGACTTT     1  \n",
       "25                                 TCAATCACAACGACATCGTCCTT     1  \n",
       "26                                  AGATTGGGCTCCTATCCTGGAA     2  \n",
       "27                                  GTTTCTCTGCCGATATCTTTCA     2  \n",
       "28                                CACTGTTGGTTCCTCTATCTCGTG     4  \n",
       "29                                  ATTTGATGTGAACAATTTGTGA     6  \n",
       "...                                                    ...   ...  \n",
       "1109025                                          AAAAAAAAA     6  \n",
       "1109026                                          AAAAAAAAA     6  \n",
       "1109027                                          AAAAAAAAA     6  \n",
       "1109028                                          AAAAAAAAA     6  \n",
       "1109029                                          AAAAAAAAA     6  \n",
       "1109030                                          AAAAAAAAA     6  \n",
       "1109031                                          AAAAAAAAA     6  \n",
       "1109032                                          AAAAAAAAA     6  \n",
       "1109033                                          AAAAAAAAA     6  \n",
       "1109034                                          AAAAAAAAA     6  \n",
       "1109035                                          AAAAAAAAA     6  \n",
       "1109036                                          AAAAAAAAA     6  \n",
       "1109037                                          AAAAAAAAA     6  \n",
       "1109038                                          AAAAAAAAA     6  \n",
       "1109039                                          AAAAAAAAA     6  \n",
       "1109040                                          AAAAAAAAA     6  \n",
       "1109041                                          AAAAAAAAA     6  \n",
       "1109042                                          AAAAAAAAA     6  \n",
       "1109043                                          AAAAAAAAA     6  \n",
       "1109044                                          AAAAAAAAA     6  \n",
       "1109045                                          AAAAAAAAA     6  \n",
       "1109046                                          AAAAAAAAA     6  \n",
       "1109047                                          AAAAAAAAA     6  \n",
       "1109048                                          AAAAAAAAA     6  \n",
       "1109049                                          AAAAAAAAA     6  \n",
       "1109050                                          AAAAAAAAA     6  \n",
       "1109051                                          AAAAAAAAA     6  \n",
       "1109052                                          AAAAAAAAA     6  \n",
       "1109053                                          AAAAAAAAA     6  \n",
       "1109054                            GTACAACCTTGCAGGAGAGAATT     1  \n",
       "\n",
       "[1109055 rows x 6 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dna_df['seq'] = dna_df[0].map(lambda x : x.split(\":\")[0])\n",
    "dna_df['count'] = dna_df[0].map(lambda x : x.split(\":\")[1])\n",
    "dna_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                                     ATTCATCATTTGCCCCGCACAG\n",
       "1                                    ACACAGATGCTCTATTGGATTCA\n",
       "2                                     GCTGGTCAGAAGGCTCTTGGGT\n",
       "3                                    GTATATATATGTGCGCGAGAACA\n",
       "4                                     GCAGATTACTTGGAGTCAACCT\n",
       "5                                     GACGAGTTGATTTAATGTAGNN\n",
       "6                                   GTGGTTTTTATGTGGATCTGATTT\n",
       "7          GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...\n",
       "8                                  GCGCTTCTCGTAATCTCTTGATTTT\n",
       "9                                    GAAAATGTATTGCGGATTGTTAT\n",
       "10                                     GTTGTCTTCAAGATGTTTGTT\n",
       "11                                  ATCATTCATTTGGGAACTTGGTAA\n",
       "12                                  GATGATCAAGACATTTTGTTTGAA\n",
       "13                                    GTGGTGTTGTTATGATTATCAG\n",
       "14                               CGACGATCGTTCTGGATGTTGTAAGCT\n",
       "15                                        TACCTGATTGATTGAGCG\n",
       "16                                            TGTGACCTTTTTGT\n",
       "17                                    GGATTGGGAGGTGAATAGTTAT\n",
       "18                                  GATATCGCCGTCTTCACTGAATTT\n",
       "19                                    GTATTGTTTCCGATATTTGACT\n",
       "20                                      GTGTTAGGTCGGTGGTTGCA\n",
       "21                                      GATCCGATTCCTCTGATGCG\n",
       "22                                   TCGTATGGATTGCAACCGTTTTC\n",
       "23                                  GAAAAAAAACCGGGAACACTTTTG\n",
       "24                                   GATAGTACGCTCTATTGGACTTT\n",
       "25                                   TCAATCACAACGACATCGTCCTT\n",
       "26                                    AGATTGGGCTCCTATCCTGGAA\n",
       "27                                    GTTTCTCTGCCGATATCTTTCA\n",
       "28                                  CACTGTTGGTTCCTCTATCTCGTG\n",
       "29                                    ATTTGATGTGAACAATTTGTGA\n",
       "                                 ...                        \n",
       "1109025                                            AAAAAAAAA\n",
       "1109026                                            AAAAAAAAA\n",
       "1109027                                            AAAAAAAAA\n",
       "1109028                                            AAAAAAAAA\n",
       "1109029                                            AAAAAAAAA\n",
       "1109030                                            AAAAAAAAA\n",
       "1109031                                            AAAAAAAAA\n",
       "1109032                                            AAAAAAAAA\n",
       "1109033                                            AAAAAAAAA\n",
       "1109034                                            AAAAAAAAA\n",
       "1109035                                            AAAAAAAAA\n",
       "1109036                                            AAAAAAAAA\n",
       "1109037                                            AAAAAAAAA\n",
       "1109038                                            AAAAAAAAA\n",
       "1109039                                            AAAAAAAAA\n",
       "1109040                                            AAAAAAAAA\n",
       "1109041                                            AAAAAAAAA\n",
       "1109042                                            AAAAAAAAA\n",
       "1109043                                            AAAAAAAAA\n",
       "1109044                                            AAAAAAAAA\n",
       "1109045                                            AAAAAAAAA\n",
       "1109046                                            AAAAAAAAA\n",
       "1109047                                            AAAAAAAAA\n",
       "1109048                                            AAAAAAAAA\n",
       "1109049                                            AAAAAAAAA\n",
       "1109050                                            AAAAAAAAA\n",
       "1109051                                            AAAAAAAAA\n",
       "1109052                                            AAAAAAAAA\n",
       "1109053                                            AAAAAAAAA\n",
       "1109054                              GTACAACCTTGCAGGAGAGAATT\n",
       "Name: seq, dtype: object"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dna_df['seq']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dna_test = dna_df[dna_df['seq'].str.contains('TCATTTGCC')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def align(seq):\n",
    "    maxSequence = ''\n",
    "    foundList = []\n",
    "    \n",
    "    cutoff = 5\n",
    "    post = seq[5:]\n",
    "    term = seq[:5]\n",
    "\n",
    "    # For the real dataset, will replace this with each chromosome\n",
    "    full_dna = ''.join(record_dict['X'].seq[:10000])\n",
    "\n",
    "    last_position = -1\n",
    "    cont = True\n",
    "    maxScore = -999\n",
    "    while cont:\n",
    "        last_position = full_dna.find(term, last_position+1)\n",
    "        if last_position > 4:\n",
    "\n",
    "            fragment = full_dna[last_position-5:last_position+len(post)]\n",
    "            score = compare(seq,fragment)\n",
    "            if score > maxScore:\n",
    "                maxScore = score\n",
    "                maxSequence = fragment\n",
    "        else:\n",
    "            cont = False\n",
    "    return maxSequence+':'+ str(maxScore/float(len(seq)))\n",
    "    \n",
    "def compare(first, second):\n",
    "    if len(first) != len(second):\n",
    "        return -999\n",
    "    score = 0\n",
    "    for i in range(len(second)):\n",
    "        if first[i] == second[i]:\n",
    "            score +=1\n",
    "        else:\n",
    "            score -= 1\n",
    "    return score\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "temp = dna_df.iloc[:1000]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/ipykernel/__main__.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  if __name__ == '__main__':\n"
     ]
    }
   ],
   "source": [
    "temp['best'] = temp['seq'].map(align)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>seq</th>\n",
       "      <th>count</th>\n",
       "      <th>best</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ATTCATCATTTGCCCCGCACAG:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ATTCATCATTTGCCCCGCACAG</td>\n",
       "      <td>1</td>\n",
       "      <td>TTTGAATTCAGCTCGCCGAGAG:-0.272727272727</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ACACAGATGCTCTATTGGATTCA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>ACACAGATGCTCTATTGGATTCA</td>\n",
       "      <td>1</td>\n",
       "      <td>AAACAACACAATCATTTAAAAAT:-0.304347826087</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>GCTGGTCAGAAGGCTCTTGGGT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCTGGTCAGAAGGCTCTTGGGT</td>\n",
       "      <td>1</td>\n",
       "      <td>TGAACGCTGGAAATGTGTCTGT:-0.454545454545</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>GTATATATATGTGCGCGAGAACA:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTATATATATGTGCGCGAGAACA</td>\n",
       "      <td>1</td>\n",
       "      <td>AAAAAGTATAGTAATAGTTAAGT:-0.391304347826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>GCAGATTACTTGGAGTCAACCT:8</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCAGATTACTTGGAGTCAACCT</td>\n",
       "      <td>8</td>\n",
       "      <td>AGAATGCAGAAAAATGCAATGT:-0.363636363636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>GACGAGTTGATTTAATGTAGNN:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GACGAGTTGATTTAATGTAGNN</td>\n",
       "      <td>1</td>\n",
       "      <td>GCCAAGACGAAAAAATACAGTA:0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>GTGGTTTTTATGTGGATCTGATTT:5</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTGGTTTTTATGTGGATCTGATTT</td>\n",
       "      <td>5</td>\n",
       "      <td>AGTGTGTGGTTACAATGCAGTTAA:-0.416666666667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...</td>\n",
       "      <td>1</td>\n",
       "      <td>CATCCGTATATTTTTAGCAATTGAGAGCTCAAATTATTCAGCCTTT...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>GCGCTTCTCGTAATCTCTTGATTTT:1</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GCGCTTCTCGTAATCTCTTGATTTT</td>\n",
       "      <td>1</td>\n",
       "      <td>:-39.96</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>GAAAATGTATTGCGGATTGTTAT:2</td>\n",
       "      <td>4</td>\n",
       "      <td>*</td>\n",
       "      <td>0</td>\n",
       "      <td>GAAAATGTATTGCGGATTGTTAT</td>\n",
       "      <td>2</td>\n",
       "      <td>CAGGAGAAAATTCCGATTTTCTA:-0.130434782609</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   0  1  2  3  \\\n",
       "0                           ATTCATCATTTGCCCCGCACAG:1  4  *  0   \n",
       "1                          ACACAGATGCTCTATTGGATTCA:1  4  *  0   \n",
       "2                           GCTGGTCAGAAGGCTCTTGGGT:1  4  *  0   \n",
       "3                          GTATATATATGTGCGCGAGAACA:1  4  *  0   \n",
       "4                           GCAGATTACTTGGAGTCAACCT:8  4  *  0   \n",
       "5                           GACGAGTTGATTTAATGTAGNN:1  4  *  0   \n",
       "6                         GTGGTTTTTATGTGGATCTGATTT:5  4  *  0   \n",
       "7  GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...  4  *  0   \n",
       "8                        GCGCTTCTCGTAATCTCTTGATTTT:1  4  *  0   \n",
       "9                          GAAAATGTATTGCGGATTGTTAT:2  4  *  0   \n",
       "\n",
       "                                                 seq count  \\\n",
       "0                             ATTCATCATTTGCCCCGCACAG     1   \n",
       "1                            ACACAGATGCTCTATTGGATTCA     1   \n",
       "2                             GCTGGTCAGAAGGCTCTTGGGT     1   \n",
       "3                            GTATATATATGTGCGCGAGAACA     1   \n",
       "4                             GCAGATTACTTGGAGTCAACCT     8   \n",
       "5                             GACGAGTTGATTTAATGTAGNN     1   \n",
       "6                           GTGGTTTTTATGTGGATCTGATTT     5   \n",
       "7  GTATATCTAGTTTGTCGTTTAATTCCCCTTGAAATTTTTTTGGGGG...     1   \n",
       "8                          GCGCTTCTCGTAATCTCTTGATTTT     1   \n",
       "9                            GAAAATGTATTGCGGATTGTTAT     2   \n",
       "\n",
       "                                                best  \n",
       "0             TTTGAATTCAGCTCGCCGAGAG:-0.272727272727  \n",
       "1            AAACAACACAATCATTTAAAAAT:-0.304347826087  \n",
       "2             TGAACGCTGGAAATGTGTCTGT:-0.454545454545  \n",
       "3            AAAAAGTATAGTAATAGTTAAGT:-0.391304347826  \n",
       "4             AGAATGCAGAAAAATGCAATGT:-0.363636363636  \n",
       "5                         GCCAAGACGAAAAAATACAGTA:0.0  \n",
       "6           AGTGTGTGGTTACAATGCAGTTAA:-0.416666666667  \n",
       "7  CATCCGTATATTTTTAGCAATTGAGAGCTCAAATTATTCAGCCTTT...  \n",
       "8                                            :-39.96  \n",
       "9            CAGGAGAAAATTCCGATTTTCTA:-0.130434782609  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/ipykernel/__main__.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  if __name__ == '__main__':\n",
      "/usr/local/lib/python2.7/dist-packages/ipykernel/__main__.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  from ipykernel import kernelapp as app\n",
      "/usr/local/lib/python2.7/dist-packages/ipykernel/__main__.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  app.launch_new_instance()\n"
     ]
    }
   ],
   "source": [
    "temp['fragment'] = temp['best'].map(lambda x : x.split(\":\")[0])\n",
    "temp['score'] = temp['best'].map(lambda x : x.split(\":\")[1])\n",
    "temp['score'] = pd.to_numeric(temp['score'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0                                 TTTGAATTCAGCTCGCCGAGAG\n",
       "1                                AAACAACACAATCATTTAAAAAT\n",
       "2                                 TGAACGCTGGAAATGTGTCTGT\n",
       "3                                AAAAAGTATAGTAATAGTTAAGT\n",
       "4                                 AGAATGCAGAAAAATGCAATGT\n",
       "5                                 GCCAAGACGAAAAAATACAGTA\n",
       "6                               AGTGTGTGGTTACAATGCAGTTAA\n",
       "7      CATCCGTATATTTTTAGCAATTGAGAGCTCAAATTATTCAGCCTTT...\n",
       "8                                                       \n",
       "9                                CAGGAGAAAATTCCGATTTTCTA\n",
       "10                                 GTATGGTTGTTTAGCTACAGT\n",
       "11                              TTCAGATCATCGAGAGCTTTCATT\n",
       "12                              TTGTTGATGAAGTTTTTGTGAAGT\n",
       "13                                CGTATGTGGTATTTTGTAATAG\n",
       "14                           TTTGACGACGTTCAGAGCTCGGAAAAT\n",
       "15                                    GTACTTACCTTTCAAGTG\n",
       "16                                        ATTCATGTGAATTT\n",
       "17                                CGAATGGATTGAAAGCAATTGT\n",
       "18                              GACAAGATATTTAACAAAATTTTC\n",
       "19                                ATGTGGTATTTTGTAATAGTCA\n",
       "20                                  GAAATGTGTTGAAAGATGCA\n",
       "21                                  GGTTGGATCCAACAGTTCTT\n",
       "22                               TAAAATCGTATGAAATCCCGTTT\n",
       "23                              AATTAGAAAAAAAAAACAATTTTT\n",
       "24                               AATTGGATAGTGTGGATATTGTC\n",
       "25                               CAAAATCAATTTCTAACCTATTT\n",
       "26                                ATTTTAGATTTTAATTTAGAAA\n",
       "27                                TTTTTGTTTCTTGATTCTTGCA\n",
       "28                              ATGCCCACTGTTCTTCATGCCTAC\n",
       "29                                TAAAAATTTGAAAAAATTTCGA\n",
       "                             ...                        \n",
       "970                                CCTTTCAAGTGAAAATTGAGG\n",
       "971                               CATTTTTTTTGCGATTTTCAAG\n",
       "972                              AAATTGAAGTTTCTTGTGAAAAC\n",
       "973                               AGAAGCTGCAAAAAATTTGTTT\n",
       "974                                AATTCGGATAAAACCGATAGA\n",
       "975                                AAAATGCATTAGAAAAACCGA\n",
       "976    TTTCCGTCACTTCTTCTACCACCGTTTCCACAGTAATCTCATCGAA...\n",
       "977                                        GTTTTATTATTAA\n",
       "978                                        GTTTTATTATTAA\n",
       "979                                        GTTTTATTATTAA\n",
       "980                                        GTTTTATTATTAA\n",
       "981                                        GTTTTATTATTAA\n",
       "982                                        GTTTTATTATTAA\n",
       "983                                        GTTTTATTATTAA\n",
       "984                                        GTTTTATTATTAA\n",
       "985                                        GTTTTATTATTAA\n",
       "986                                          CTATCGTTTCT\n",
       "987                                          CTATCGTTTCT\n",
       "988                               AAAGCGAAAATTGGATAGTGTG\n",
       "989                            TTTTTGAGTTTTCTTTTTTCTATAA\n",
       "990                                AATCAGATTATTTCGTTGAAT\n",
       "991                                 GCGATGCTCGCAAAATGCAT\n",
       "992                              AAAAAGAAAACTCAAAAATTTAA\n",
       "993                               ACATGGCAAGCATTATTTCGTT\n",
       "994                            GCTACGTTGTCTTTTTAGTAAGTTT\n",
       "995                              ACATGGCAAGCATTATTTCGTTC\n",
       "996                               GAGGAGATAAAGATGGCTCGGC\n",
       "997                                         TTATCATAACAT\n",
       "998                                AAAATGTGATAGAGTGAATCA\n",
       "999                                TTTGTTTCGTATGTGGTATTT\n",
       "Name: fragment, dtype: object"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "temp['fragment']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "scoresTemp = list(temp[temp['score']>0]['score'].sort_values(ascending = False))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7fc519f4b710>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABJAAAAE4CAYAAAAEpwkNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+QXed5H/bvQxFoNpZBm/G1NSHFXTtyBEYdRqJC1k3s\neCERFpXIotPaETGjqZQgsTksYzexU1pOMwKn/0TOpLFbBRO13WrUxFrAP9pI9fgHrYg7rqdRsREl\nUaYBkq5nYcKUhWtbYuoOJkSkt3/cu9BysXsI7D27e3fx+czs6J7znD3nBeboYPfL931OtdYCAAAA\nAJu5abcHAAAAAMB0EyABAAAA0EmABAAAAEAnARIAAAAAnQRIAAAAAHQSIAEAAADQqZcAqarur6pz\nVfVsVT26Qf21VfXJqnqyqj5bVW/v47oAAAAAbL9qrU12gqqbkjyb5K1JXkiynOTB1tq5Ncd8KMmT\nrbUPVdWdSX6ptfatE10YAAAAgB3Rxwyke5M811o731q7nORUkgfWHfPVJIfGn78hye/1cF0AAAAA\ndsDNPZzjtiTPr9m+kFGotNZjSR6vqh9O8ieT3NfDdQEAAADYAX3MQKoN9q1fF3csyYdba69N8leT\n/MsergsAAADADuhjBtKFJHes2b49o15Iax1P8rYkaa19qqr+RFV9U2vtD9YeVFWTNWQCAAAA4Cqt\ntY0mAF2zPgKk5SSvq6rZJF9I8mBGM47WOp/RsrWPjJto/0frw6NVkzb15sZx4sSJnDhxYreHwR7g\nXuF6uF+4Vu4Vrof7hWvlXuF6uF+4VlUTZUdJeljC1lr7SpJHkjye5Okkp1prZ6vqsap6x/iwH0vy\nt6vqs0l+Jsl7Jr0uAAAAADujjxlIaa39SpLXr9v3/jWfzyb5zj6uBQAAAMDO6qOJNuyK+fn53R4C\ne4R7hevhfuFauVe4Hu4XrpV7hevhfmEn1TT1HKqqNk3jAQAAANjrqmriJtpmIAEAAADQSYAEAAAA\nQCcBEgAAAACdBEgAAAAAdBIgAQAAANBJgAQAAABAJwESe9JwOMzy8nKGw+F11QAAAIDrJ0Biz1lc\nPJ3Z2cM5evShzM4ezuLi6WuqAQAAAFtTrbXdHsMVVdWmaTxMn+FwmNnZw7l06YkkdyV5KjMzR3L+\n/Lkk2bQ2GAx2cdQAAACwe6oqrbWa5BxmILGnrKys5ODBuYwCoiS5KwcOzGZlZaWzBgAAAGzdzbs9\nALgec3NzeemllSRPZXWW0eXL5zM3N5cknTUAAABga8xAYk8ZDAZZWDiZmZkjOXTo7szMHMnCwskM\nBoPOGgAAALB1eiCxJw2Hw6ysrGRubu6qgKirBgAAADeaPnogCZAAAAAA9jFNtAEAAADYdgIkAAAA\nADoJkAAAAADoJEACAAAAoJMACQAAAIBOAiQAAAAAOvUSIFXV/VV1rqqerapHN6j/d1X1map6sqqe\nqao/6uO6AAAAAGy/aq1NdoKqm5I8m+StSV5IspzkwdbauU2OfyTJG1trf2uDWpt0PAAAAAB8TVWl\ntVaTnKOPGUj3JnmutXa+tXY5yakkD3QcfyzJYg/XBQAAAGAH9BEg3Zbk+TXbF8b7rlJVdySZS/LJ\nHq4LAAAAwA7oI0DaaArUZuvQHkzy89apAQAAAOwdN/dwjgtJ7lizfXtGvZA28mCSh7tOduLEiSuf\n5+fnMz8/P9noAAAAAG4gS0tLWVpa6vWcfTTRflWSZzJqov2FJGeSHGutnV133OuT/HJr7ds6zmVy\nEgAAAECPpqKJdmvtK0keSfJ4kqeTnGqtna2qx6rqHWsOfTCjBtsAAAAA7CETz0DqkxlIAAAAAP2a\nihlIAAAAAOxvAiQAAAAAOgmQAAAAAOgkQAIAAACgkwAJAAAAgE4CJAAAAAA6CZAAAAAA6CRAAgAA\nAKCTAAkAAACATgIkAAAAADoJkAAAAADoJEACAAAAoJMACQAAAIBOAiQAAAAAOgmQAAAAAOgkQAIA\nAACgkwAJAAAAgE4CJAAAAAA6CZAAAAAA6CRAAgAAAKCTAAkAAACATgIkAAAAADr1EiBV1f1Vda6q\nnq2qRzc55q9X1dNV9fmq+pd9XBcAAACA7VettclOUHVTkmeTvDXJC0mWkzzYWju35pjXJTmd5Ehr\n7d9V1Te11v5gg3O1SccDAAAAwNdUVVprNck5+piBdG+S51pr51trl5OcSvLAumP+dpJ/1lr7d0my\nUXgEAAAAwHTqI0C6Lcnza7YvjPet9WeTvL6qfqOq/q+qelsP1wUAAABgB9zcwzk2mgK1fh3azUle\nl+QvJ7kjyf9ZVW9YnZEEAAAAwPTqI0C6kFEotOr2jHohrT/m37TWvppkpaqeSfLtST69/mQnTpy4\n8nl+fj7z8/M9DBEAAADgxrC0tJSlpaVez9lHE+1XJXkmoybaX0hyJsmx1trZNce8bbzvvVX1TRkF\nR29srX1p3bk00QYAAADo0VQ00W6tfSXJI0keT/J0klOttbNV9VhVvWN8zK8m+cOqejrJv07yY+vD\nIwAAAACm08QzkPpkBhIAAABAv6ZiBhIAAAAA+5sACQAAAIBOAiQAAAAAOgmQAAAAAOgkQAIAAACg\nkwAJAAAAgE4CJAAAAAA6CZAAAAAA6CRAAgAAAKCTAAkAAACATgIkAAAAADoJkAAAAADoJEACAAAA\noJMACQAAAIBOAiQAAAAAOgmQAAAAAOgkQAIAAACgkwAJAAAAgE4CJAAAAAA6CZAAAAAA6CRAAgAA\nAKCTAAkAAACATr0ESFV1f1Wdq6pnq+rRDervqaqLVfXk+Otv9nFdAAAAALbfzZOeoKpuSvLBJG9N\n8kKS5ar6WGvt3LpDT7XWfnjS6wEAAACws/qYgXRvkudaa+dba5eTnErywAbHVQ/XAgAAAGCH9REg\n3Zbk+TXbF8b71vvPquqzVfWzVXV7D9cFAAAAYAf0ESBtNLOordv+eJK51tobk/zrJB/p4boAAAAA\n7ICJeyBlNOPojjXbt2fUC+mK1tqX1mz+T0k+sNnJTpw4ceXz/Px85ufnexgiAAAAwI1haWkpS0tL\nvZ6zWls/Weg6T1D1qiTPZNRE+wtJziQ51lo7u+aY17TWfn/8+a8l+futtb+4wbnapOMBAAAA4Guq\nKq21iXpTTzwDqbX2lap6JMnjGS2JW2itna2qx5Ist9Z+MckPV9U7k1xO8kdJ3jvpdQEAAADYGRPP\nQOqTGUgAAAAA/epjBlIfTbQBAAAA2McESAAAAAB0EiABAAAA0EmABAAAAEAnARIAAAAAnQRIAAAA\nAHQSIAEAAADQSYAEAAAAQCcBEgAAAACdBEgAAAAAdBIgAQAAANBJgAQAAABAJwESAAAAAJ0ESAAA\nAAB0EiABAAAA0EmABAAAAEAnARIAAAAAnQRIAAAAAHQSIAEAAADQSYAEAAAAQCcBEgAAAACdBEgA\nAAAAdOolQKqq+6vqXFU9W1WPdhz3/VX11aq6u4/rAgAAALD9Jg6QquqmJB9M8rYkb0hyrKoOb3Dc\nq5P8nSSfmvSaAAAAAOycPmYg3Zvkudba+dba5SSnkjywwXH/bZIPJPn3PVwTAAAAgB3SR4B0W5Ln\n12xfGO+7oqremOT21tov9XA9AAAAAHZQHwFSbbCvXSlWVZJ/muRHX+F7YNsNh8MsLy9nOBzuSA0A\nAAD2g5t7OMeFJHes2b49yQtrtr8+o95IS+Mw6TVJPlZV72ytPbn+ZCdOnLjyeX5+PvPz8z0MEZLF\nxdM5fvzhHDw4l5deWsnCwskcO/aubasBAADAblhaWsrS0lKv56zW2isf1XWCqlcleSbJW5N8IcmZ\nJMdaa2c3Of6JJH+vtfaZDWpt0vHARobDYWZnD+fSpSeS3JXkqczMHMn58+eSpPfaYDDY8T8jAAAA\nbKSq0lqbaDXYxDOQWmtfqapHkjye0ZK4hdba2ap6LMlya+0X139LLGFjh62srOTgwblcunTXeM9d\nOXBgNisrK0nSe02ABAAAwH7SxxK2tNZ+Jcnr1+17/ybHvqWPa8L1mJsbLTFLnsrqbKHLl89nbm4u\nSbalBgAAAPtFH020YeoNBoMsLJzMzMyRHDp0d2ZmjmRh4WQGg8G21AAAAGA/mbgHUp/0QGK7DYfD\nrKysZG5u7qqgZztqAAAAsNv66IEkQAIAAADYx/oIkCxhAwAAAKCTAAkAAACATgIkAAAAADoJkAAA\nAADoJEACAAAAoJMACQAAAIBOAiQAAAAAOgmQAAAAAOgkQAIAAACgkwAJAAAAgE4CJNhGw+Ewy8vL\nGQ6HO1IDAACA7SBAgm2yuHg6s7OHc/ToQ5mdPZzFxdPbWgMAAIDtUq213R7DFVXVpmk8sFXD4TCz\ns4dz6dITSe5K8lRmZo7k/PlzSdJ7bTAY7PifEQAAgL2hqtJaq0nOcXNfgwG+ZmVlJQcPzuXSpbvG\ne+7KgQOzWVlZSZLeawIkAAAAtpMACbbB3NxcXnppJclTWZ0tdPny+czNzSXJttQAAABgu+iBBNtg\nMBhkYeFkZmaO5NChuzMzcyQLCyczGAy2pQYAAADbSQ8k2EbD4TArKyuZm5u7KujZjhoAAACs10cP\nJAESAAAAwD7WR4BkCRsAAAAAnXoJkKrq/qo6V1XPVtWjG9R/qKqeqqrPVNWvV9XhPq4LAAAAwPab\neAlbVd2U5Nkkb03yQpLlJA+21s6tOebVrbU/Hn/+3iQPt9bevsG5LGEDAAAA6NG0LGG7N8lzrbXz\nrbXLSU4leWDtAavh0dirk3y1h+sC6wyHwywvL2c4HO72UAAAANhH+giQbkvy/JrtC+N9L1NVD1fV\nbyf5R0l+uIfrAmssLp7O7OzhHD36UGZnD2dx8fRuDwkAAIB9oo8AaaMpUFetQ2utnWytvS7Jo0n+\nYQ/XBcaGw2GOH384ly49kRdf/HQuXXoix48/bCYSAAAAvbi5h3NcSHLHmu3bM+qFtJnTSf75ZsUT\nJ05c+Tw/P5/5+fnJRgc3gJWVlRw8OJdLl+4a77krBw7MZmVlJYPBYFfHBgAAwM5aWlrK0tJSr+fs\no4n2q5I8k1ET7S8kOZPkWGvt7JpjXtda++3x5+9N8g9ba/ducC5NtGELhsNhZmcP59KlJ5LcleSp\nzMwcyfnz5wRIAAAAN7ipaKLdWvtKkkeSPJ7k6SSnWmtnq+qxqnrH+LBHquo3q+rJJP9VkvdMel3g\nawaDQRYWTmZm5kgOHbo7MzNHsrBwUngEAABALyaegdQnM5BgMsPhMCsrK5mbm7sqPNpqDQAAgL1t\nKmYgAdNjMBjknnvuuSoE6npDm7e3AQAA8ErMQIJ9rqs/UpJX7J20HbOazHgCAADYOWYgAa9o9Q1t\no4AoWfuGtq5asvWZS2Y8AQAA7C9mIME+t9UZSDtdMxMJAABge5iBBLyirje0ddW2OnNpkhlPAAAA\nTCczkOAGcb09iXZ65pIZSAAAANujjxlIN/c1GGC6rc44utba6uyk48eP5MCB2Vy+fP7K7KQk21ID\nAABgOpmBBHTyFjYAAIC9rY8ZSAIkYKpsJXgScgEAAGxOE21gX1lcPJ3Z2cM5evShzM4ezuLi6Ves\nbeV7JqkBAADciMxAAqbCVpp2f/rTv5E3v/k7NfoGAADooIk2sG+srKzk4MG5XLp013jPXTlwYDYr\nKytJsmHtzJkz1/09k9QESAAAwI3KEjZgKszNzeWll1aSPDXe81QuXz6fubm5TWv33nvvdX/PJLVV\nw+Ewy8vLGQ6HV/05pqkGAADQFwESMBUGg0EWFk5mZuZIDh26OzMzR7KwcDKDwWDT2p133nnd3zNJ\nLdn5nkt6NQEAANNADyRgqkzzW9i20qdpN2qW2gEAAGvpgQTsO6uzgK6ntpXv2UptK32adqMmQAIA\nAPomQAK4Ri/vjzSa9bO2P9I01QAAAPqkBxLANdqOvkrb1asJAACgT3ogAVynneq5NI01AABg7+mj\nB5IACWAfWlw8nePHH87Bg6NldwsLJ3Ps2LsmqgEAAHuTAAmAq2zH2+LMRAIAgL3LW9gAuMp2vC1O\ngAQAADe2XppoV9X9VXWuqp6tqkc3qP/dqnq6qj5bVb9WVa/t47oAXO3lb4tL1r6hbas1AADgxjZx\ngFRVNyX5YJK3JXlDkmNVdXjdYU8meXNr7Y1JfiHJP570ugBszJvdAACAvk3cA6mqviPJ+1trbx9v\n/3iS1lr7wCbHvzHJ/9Ba+64NanogAfRkmt7e5o1wAACwe/rogdTHErbbkjy/ZvvCeN9mjif55R6u\nC0CHwWCQe+65Z8PgZSu1xcXTmZ09nKNHH8rs7OEsLp7etRoAALCz+piB9P1Jvqe19oPj7Xcnuae1\n9iMbHPvuJA8n+e7W2uUN6mYgAUyh7Xiz21ZrZiIBAMD1mZa3sF1Icsea7duTvLD+oKq6L8n7kvzl\njcKjVSdOnLjyeX5+PvPz8z0MEYBJbMeb3bZaEyABAEC3paWlLC0t9XrOPmYgvSrJM0nemuQLSc4k\nOdZaO7vmmDcl+bkkb2ut/T8d5zIDCWAKmYEEAAB711T0QGqtfSXJI0keT/J0klOttbNV9VhVvWN8\n2E8m+bokP1dVn6mqfzXpdQHYOdvxZrdJ3wg3HA6zvLyc4XB41Xi3owYAADeyiWcg9ckMJIDpNi1v\nYVtcPJ3jxx/OwYNzeemllSwsnMyxY+/athoAAOxlfcxAEiABsKfs9HI6S+YAANjrpqWJNgDsmJ1u\n6C1AAgAAARIAe8zc3GiJWfJUVmcLXb58PnNzc0myLTUAALjRTdxEGwB20n5q2g0AAHuFHkgA7El7\nvWk3AADsFE20AWCbbUfTbn2VAADYSZpoA8A2246m3asB0lZnSgEAwE7TAwkAOry8aXeytsH2VmvJ\naHnb7OzhHD36UGZnD2dx8fSVa3bVAABgN1jCBgCvYLWX0YEDs7l8+fyGfY6up7bVZXFmIgEAsBV6\nIAHADumzMffy8nKOHn0oL7746SvHHTp0dz7xiQ8lyaa1e+65Z7v+eAAA7GMCJADYg8xAAgBgJ/UR\nIOmBBAA7bDAYZGHhZGZmjuTQobszM3MkCwsnMxgMOmurhsNhlpeXMxwOrzp3Vw0AALbKDCQA2CVb\nWRa32lfp4MFRk+6Nei5tVAMA4MZlCRsA3EAsfQMAYCssYQOAG8jKykoOHpzLKCBKkrty4MBsVlZW\nOmurtrr0bT/UAACYjAAJAPaIubnR0rTkqfGep3L58vnMzc111pLR8rbZ2cM5evShzM4ezuLi6Svn\n3e81AAAmZwkbAOwhq32ODhyYzeXL5zfsgbS+ttWlb/uhZvkeAEA/S9hu7mswAMD2O3bsXbnvvrds\n2GB7s9rq8rZLlzZe3rafawIkAIB+CJAAYI8ZDAabBiMb1V6+vG00Q2ft8rb9Xku633gHAMAr0wMJ\nAPa5wWCQhYWTmZk5kkOH7s7MzJEsLJy8Ejbt51qiPxIAQB/0QAKAG0TXLJz9Wuvq/7T2mI3OadYS\nALBf9NEDSYAEAOxby8vLOXr0obz44qev7Dt06O584hMfyj333HOl8fjBg6NlfquNxzfbDwCwF01N\ngFRV9yf5qYyWxC201j6wrv5d4/pdSd7VWvvfNjmPAAkA6M1W3kD36U//Rt785u/0VjcAYN/oI0Ca\nuAdSVd2U5INJ3pbkDUmOVdXhdYedT/KeJD8z6fUAAK5VV3+k1bfTjUKiZPXtbWfOnNlw/+ob35JR\nMLW8vJzhcLiTfxwAgF3TRxPte5M811o731q7nORUkgfWHtBa+93W2m8mMb0IANhRx469K+fPn8sn\nPvGhnD9/7spStJe/nS5ZfXvbvffeu+H+1be6vVJT7q5waadrAAB96SNAui3J82u2L4z3AQBMhcFg\nkHvuuedlS9A2m5105513bjpraTgc5vjxh3Pp0hN58cVP59KlJ3L8+MNXwpuucGmnawAAfZq4B1JV\nfX+S72mt/eB4+91J7mmt/cgGx344yf+hBxIAMC2u5y1sXU255+bmrrvf0nbV9GoCANbqowfSzT2M\n40KSO9Zs357kha2e7MSJE1c+z8/PZ35+fqunAgB4RYPBYMPAZaP9L1/2NgptVpe3rfZUunRp495J\nO1lbHfdm4Zja9NQAYDssLS1laWmp35O21ib6SvKqJL+dZDbJwSSfTXLnJsd+OMl/3nGuBgAwzT76\n0VNtZubWdujQm9rMzK3tox891Vpr7eLFi21m5taWfK4lrSWfazMzt7aLFy/ueG3tOG+55e6XjVNt\nemoAsFPGectk+c+kJxiNI/cneSbJc0l+fLzvsSTvGH/+Cxn1Sfp/kwyTfH6T82zbXxYAQF8uXrzY\nzpw5cyWsWbVZuLTTtWkKs9Q2D/m67qW9VANg+vURIPWxhC2ttV9J8vp1+96/5vO/TfLaPq4FALDb\nNlv2duzYu3LffW/ZcLnSTtamaTmd2ubLDBcXT+f48Ydz8OBoaeTCwskrbwncKzUAbhwTN9Hukyba\nAACTGw6HU9PQW23/1vTa0t8K2Dv6aKLdyxK2vr5iCRsAQC+mZTmd2sa1M2fOtFtuuXu8tG30dejQ\nm9qZM2f2TG3tn28aekpNUw1g2qSHJWxmIAEA7FPTNBtD7eU1s8T2b81MJGAa9TEDqZceSAAATJ/N\nejWp7X5tMBhkYeFkjh8/kgMHZnP58vksLJy8ctxeqC0vL09NT6lpqgmQgP3KDCQAANgl0zAbaqs1\ns6jMQAL2DjOQAABgD5uG2VBbre2HWVTbVQPYj8xAAgAAtmwaZkPttRrATutjBpIACQAAoEeLi6dz\n/PjDOXhwLi+9tJKFhZM5duxduz0s4AYmQAIAAJgiXb2hzEQCdksfAdJNfQ0GAADgRreyspKDB+cy\nCo+S9W9vGw6HWV5eznA4vOp793sN2NsESAAAAD2ZmxstW0ueGu95Kpcvn8/c3FwWF09ndvZwjh59\nKLOzh7O4ePrK9+33WjJdYdZ+r8G2aK1NzddoOAAAAHvXRz96qs3M3NoOHXpTm5m5tX30o6faxYsX\n28zMrS35XEtaSz7XZmZubRcvXtz3tbV/J7fccveVv5P1f19q/dRaa+3ixYvtzJkzV/7+d7PGdBjn\nLZNlNpOeoM8vARIAALAfrP+F+syZM+2WW+4ehyujr0OH3tTOnDmz72vTFGbt91preyvMYuf0ESDd\nvJuznwAAAPajwWDwsqbZL1/aNmquvbq0Lcm+rq32hbp0aeO+UGr91o4ffziXLj0xrj+V48eP5L77\n3rLjtcFg4I2E+4wACQAAYJsNBoMsLJzM8eNHcuDAbC5fPp+FhZNXQqb9XpuWMGu/16YtrOsKl9iD\nJp3C1OdXLGEDAAD2sWnqTbOTtY36Qqn1X5um5XRdyxrZeelhCVuNzjMdqqpN03gAAADox3A4zMrK\nSubm5q6agaLWX2112djamWCry8Z2sjYcDjM7eziXLj2R1ZlSMzNHcv78OTOQdkFVpbVWE51jmgIb\nARIAAABMZi+EWewsARIAAAAwtbqCJ3aOAAkAAACATn0ESDf1NRgAAAAA9icBEgAAAACdegmQqur+\nqjpXVc9W1aMb1A9W1amqeq6q/k1V3dHHdQEAAADYfhMHSFV1U5IPJnlbkjckOVZVh9cddjzJH7XW\nvj3JTyX5yUmvCwAAAMDO6GMG0r1JnmutnW+tXU5yKskD6455IMlHxp9/Pslbe7guAAAAADugjwDp\ntiTPr9m+MN634TGtta8k+XJV3drDtQEAAADYZn0ESBu9Bq69wjG1wTEAAAAATKGbezjHhSRrm2Lf\nnuSFdcc8n+S1SV6oqlclOdRa+9JGJztx4sSVz/Pz85mfn+9hiAAAAAA3hqWlpSwtLfV6zmptsolA\n40DomYz6Gn0hyZkkx1prZ9cc83CS/7i19nBVPZjk+1prD25wrjbpeAAAAAD4mqpKa22jFWTXbOIZ\nSK21r1TVI0kez2hJ3EJr7WxVPZZkubX2i0kWkvyLqnouyR8muSo8AgAAAGA6TTwDqU9mIAEAAAD0\nq48ZSH000QYAAABgHxMgAQAAANBJgAQAAABAJwESAAAAAJ0ESAAAAAB0EiABAAAA0EmABAAAAEAn\nARIAAAAAnQRIAAAAAHQSIAEAAADQSYAEAAAAQCcBEgAAAACdBEgAAAAAdBIgAQAAANBJgAQAAABA\nJwESAAAAAJ0ESAAAAAB0EiABAAAA0EmABAAAAEAnARIAAAAAnQRIAAAAAHQSIAEAAADQaaIAqaq+\nsaoer6pnqupXq+qWTY775ar6UlV9fJLrAQAAALDzJp2B9ONJPtFae32STyZ53ybH/WSSd094LXiZ\npaWl3R4Ce4R7hevhfuFauVe4Hu4XrpV7hevhfmEnTRogPZDkI+PPH0nyfRsd1Fp7IskfT3gteBkP\nS66Ve4Xr4X7hWrlXuB7uF66Ve4Xr4X5hJ00aIH1za+2LSdJa+/0kg8mHBAAAAMA0ufmVDqiqX0vy\nLWt3JWlJ/pvtGhQAAAAA06Naa1v/5qqzSeZba1+sqtckeaK1ducmx353kh9trb2z43xbHwwAAAAA\nG2qt1STf/4ozkF7Bx5O8N8kHkrwnycc6jq3x16Ym/cMAAAAA0L9JZyDdmuRnk7w2ye8m+YHW2per\n6s1Jfqi19oPj4349yeuTvDrJHyY53lr7tUkHDwAAAMD2myhAAgAAAGD/m/QtbL2oqp+sqrNV9dmq\n+oWqOrSm9r6qem5c/57dHCfToarur6pzVfVsVT262+NhulTV7VX1yar6rar6fFX98Hj/N1bV41X1\nTFX9alXdsttjZTpU1U1V9WRVfXy8PVdVnxrfK4tVNelyb/aJqrqlqn5u/DPJ01X1n3i2sJGq+rtV\n9ZtV9VRV/UxVHfRsYVVVLVTVF6vqqTX7Nn2WVNV/P/596LNV9cbdGTW7ZZP7xe/PXGWje2VN7ceq\n6qvjVWSr+6772TIVAVKSx5O8obX2xiTPJXlfklTVn0vy15PcmeTtSU5WlT5JN7CquinJB5O8Lckb\nkhyrqsO7OyqmzH9I8vdaa38uyX+a5L8c3yM/nuQTrbXXJ/lkxs8ZSPIjSX5rzfYHkvyT8b3y5STH\nd2VUTKMBuSZVAAAEy0lEQVSfTvJL4xeG/Pkk5+LZwjpV9aeT/J0kd7fW7sqo5+ixeLbwNR/O6GfZ\ntTZ8llTV25P8mdbatyf5oST/fCcHylTY6H7x+zMb2eheSVXdnuS+JOfX7NvSs2UqAqTW2idaa18d\nb34qye3jz+9Mcqq19h9aaysZ/Z/j3l0YItPj3iTPtdbOt9YuJzmV5IFdHhNTpLX2+621z44//3GS\nsxk9Ux5I8pHxYR9J8n27M0Kmyfgf1L+S5H9es/stSX5h/PkjSf7aTo+L6VNVX5/ku1prH06S8c8m\nL8azhY29KsnXjWcZzSR5IcmReLaQpLX2G0m+tG73+mfJA2v2/6/j7/u/k9xSVd+yE+NkOmx0v/j9\nmY1s8mxJkn+a5O+v27elZ8tUBEjr/M0kvzT+fFuS59fUfm+8jxvX+nviQtwTbKKq5pK8MaN/WL+l\ntfbFZBQyJRns3siYIqv/oLYkqao/leRLa34ou5DkT+/S2Jgu35bkD6rqw+Mlj/9jVf3JeLawTmvt\nhST/JKMXzPxekheTPJnky54tdPjmdc+Sbx7v9/sQr8Tvz2yqqr43yfOttc+vK23pXtmxAKmqfm28\nDnz16/Pj//3eNcf8gySXW2uLq7s2OJWu3zc29wTXpKpeneTnk/zIeCaS+4SXqaq/muSL4xlrq8+W\nytXPGfcOyWgZ0t1J/llr7e4k/19GS07cH7xMVX1DRv9ldzajkOjrMlpKsp57h2vhZ1825fdnulTV\nTJJ/kOT9G5U32PeK98qONe9rrR3tqlfVezJaRvCWNbsvJHntmu3bM5oCzI3rQpI71my7J7jKeMnA\nzyf5F621j413f7GqvqW19sWqek2Si7s3QqbEX0ryzqr6KxktMfn6JD+V0RTem8YzBTxjWHUho/+C\n92/H27+QUYDk2cJ69yX5ndbaHyVJVf3vSf5ikm/wbKHDZs8Svw+xIb8/cw3+TJK5JJ8b98K6PcmT\nVXVvtnivTMUStqq6P8l/neSdrbV/v6b08SQPjt9c8a1JXpfkzG6MkamxnOR1VTVbVQeTPJjRfQJr\n/S9Jfqu19tNr9n08yXvHn9+T5GPrv4kbS2vtJ1prd7TWvi2jZ8knW2vvTvJEkh8YH+ZeIUkyXlry\nfFX92fGutyZ5Op4tXO13k3xHVf2J8Q/sq/eKZwtrrZ/xuvZZ8t587f74eJL/Ikmq6jsyWgr5xZ0Z\nIlPkZfeL35/pcOVeaa39ZmvtNa21b2utfWtGodGbWmsXs8VnS7W2+zPaquq5JAeT/OF416daaw+P\na+/L6C0VlzNaivL47oySaTF+YP50RgHoQmvtH+3ykJgiVfWXkvx6ks9nNA2zJfmJjP7x/NmMkvbf\nTfIDrbUv79Y4mS5V9d1JfrS19s7xD1ynknxjks8kefe4aT83uKr68xk1XD+Q5HeS/I2MmiV7tvAy\nVfX+jILpyxk9R/5WRv9117OFVNVHk8wn+VNJvpjR8pJ/leTnssGzpKo+mOT+jJbO/o3W2pO7MGx2\nySb3y0/E78+ss9G9svryj3H9d5L8hTUzZK/72TIVARIAAAAA02sqlrABAAAAML0ESAAAAAB0EiAB\nAAAA0EmABAAAAEAnARIAAAAAnQRIAAAAAHQSIAEAAADQSYAEAAAAQKf/H4HonyGRWWIJAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fc51e8c75d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.rcParams['figure.figsize'] = 20, 5\n",
    "plt.scatter(range(len(scoresTemp)), scoresTemp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17537347"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(record_dict['X'].seq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "98291416\n"
     ]
    }
   ],
   "source": [
    "dna_concat = \"\"\n",
    "for sequence in dna_list:\n",
    "    dna_concat += sequence.seq\n",
    "print len(dna_concat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "26815\n"
     ]
    }
   ],
   "source": [
    "print record_dict['X'].seq[:100000].find('GCGCT')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Challenges\n",
    "\n",
    "### Challenges: Scale\n",
    "\n",
    "- Misaligned fragments file: ~1M records\n",
    "- Chromosome: ~98M bp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Domain\n",
    "\n",
    "A better understanding of current best practices in bioinformatics would have improved the process. A lot of time was spent hand-coding a solution.\n",
    "\n",
    "- Current Algorithms\n",
    "  - BLAST\n",
    "  - Bowtie\n",
    "  - Biopython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Challenges: Time\n",
    "\n",
    "For a problem like this, an iterative approach would have helped improve both the process and the solution. Much of day 1 was spent parsing the problem vs coding a solution. Although we didn't get to a final solution that could be implemented at scale, we still had fun and we learned a lot!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intended Implementation\n",
    "\n",
    "We originally wanted to write a score function to determine where missing DNA fragments originated from. It would take as input a list of unidentifiable fragments and the chromosomes they originated from. The function would then output any fragments with an alignment score greater than or equal to 70% as well as their locus on the chromosome. Unfortunately, we were unable to scale our solution to deal with larger file sizes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}