2_Complex_Queries_Basic_Visualization_py3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Complex Queries and Basic Visualization\n",
    "\n",
    "This notebook will cover how to make more complex queries, using both basic HTTP requests and using sharepa - the SHARE parsing and analysis library.\n",
    "\n",
    "We'll also go over aggregations, or queries that will return summary statistics about the whole dataset. We'll use those aggregations to make some simple data visualizations using pandas and matplotlib."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Here we'll define a helper function and specify the SHARE API url that we'll use for querying.\n",
    "\n",
    "We'll also define another helper function to nicely print out our results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import requests\n",
    "\n",
    "SHARE_API = 'https://staging-share.osf.io/api/search/abstractcreativework/_search'\n",
    "\n",
    "def query_share(url, query):\n",
    "    # A helper function that will use the requests library,\n",
    "    # pass along the correct headers,\n",
    "    # and make the query we want\n",
    "    headers = {'Content-Type': 'application/json'}\n",
    "    data = json.dumps(query)\n",
    "    return requests.post(url, headers=headers, data=data, verify=False).json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def print_numbered_results(results):\n",
    "    print(\n",
    "        'There are {} total results and {} results on this page'.format(\n",
    "            results['hits']['total'],\n",
    "            len(results['hits']['hits'])\n",
    "        )\n",
    "    )\n",
    "    print('---------------')    \n",
    "    for result in enumerate(results['hits']['hits']):\n",
    "        print('{}. {}'.format(result[0] + 1, result[1]['_source']['title']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex Queries\n",
    "\n",
    "### Pagination\n",
    "\n",
    "One request to the SHARE API will return just 10 results by default. To get more results, you can use the \"size\" parameter in your request, or paginate through the results you get back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 10 results on this page\n",
      "---------------\n",
      "1. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "2. New genus of diminutive microhylid frogs from Papua New Guinea\n",
      "3. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "4. Two new frogs of the genus Platymantis (Anura: Ceratobatrachidae) from the Isabel Island group, Solomon Islands\n",
      "5. A new species of Sycorax Curtis (Diptera, Psychodidae, Sycoracinae) collected on harlequin frogs (Anura: Bufonidae, Atelopus) in the Ecuadorian Andes\n",
      "6. Two new fringe-limbed frogs of the genus Ecnomiohyla (Anura: Hylidae) from Panama\n",
      "7. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "8. New species amongst Sri Lanka’s extinct shrub frogs (Amphibia: Rhacophoridae: Philautus)\n",
      "9. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "10. Dispersal success of European tree frogs thanks to habitat connectivity measures: A genetic evaluation\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    }
   ],
   "source": [
    "basic_query = {\n",
    "    \"query\": {\n",
    "        \"query_string\": {\n",
    "            \"query\": \"frogs\"\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "query_results = query_share(SHARE_API, basic_query)\n",
    "\n",
    "print_numbered_results(query_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get more results either by changing the number of results returned, or by paginating through the results.\n",
    "\n",
    "First, we'll return 20 results by specifying the size in our query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 20 results on this page\n",
      "---------------\n",
      "1. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "2. New genus of diminutive microhylid frogs from Papua New Guinea\n",
      "3. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "4. Two new frogs of the genus Platymantis (Anura: Ceratobatrachidae) from the Isabel Island group, Solomon Islands\n",
      "5. A new species of Sycorax Curtis (Diptera, Psychodidae, Sycoracinae) collected on harlequin frogs (Anura: Bufonidae, Atelopus) in the Ecuadorian Andes\n",
      "6. Two new fringe-limbed frogs of the genus Ecnomiohyla (Anura: Hylidae) from Panama\n",
      "7. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "8. New species amongst Sri Lanka’s extinct shrub frogs (Amphibia: Rhacophoridae: Philautus)\n",
      "9. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "10. Dispersal success of European tree frogs thanks to habitat connectivity measures: A genetic evaluation\n",
      "11. Genomic imprinting or mutation and interclonal selection in triploid hybrid frogs? A comment on Tunner.\n",
      "12. Dispersal rates and distances in adult water frogs, Rana lessonae, R. ridibunda  and their hybridogenetic associate R. esculenta\n",
      "13. Patterns of natural selection on size at metamorphosis in water frogs.\n",
      "14. The ecology of coexistence in two closely related species of frogs (rana)\n",
      "15. Species boundaries among barred river frogs, Mixophyes (Anura: Myobatrachidae) in north­eastern Australia, with descriptions of two new species\n",
      "16. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "17. Comparative larval morphology in Madagascan frogs of the genus Guibemantis (Amphibia: Mantellidae)\n",
      "18. Evolutionary history of Lost World frogs\n",
      "19. The evolution of colour in myobatrachid frogs\n",
      "20. A taxonomic revision of frogs of the genus Cryptobatrachus (Anura: Hemiphractidae)\n"
     ]
    }
   ],
   "source": [
    "basic_query = {\n",
    "    \"query\": {\n",
    "        \"query_string\": {\n",
    "            \"query\": \"frogs\"\n",
    "        }\n",
    "    },\n",
    "    \"size\": 20\n",
    "}\n",
    "\n",
    "query_results = query_share(SHARE_API, basic_query)\n",
    "\n",
    "print_numbered_results(query_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also paginate through results by specifying the place to start in all of the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 10 results on this page\n",
      "---------------\n",
      "1. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "2. New genus of diminutive microhylid frogs from Papua New Guinea\n",
      "3. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "4. Two new frogs of the genus Platymantis (Anura: Ceratobatrachidae) from the Isabel Island group, Solomon Islands\n",
      "5. A new species of Sycorax Curtis (Diptera, Psychodidae, Sycoracinae) collected on harlequin frogs (Anura: Bufonidae, Atelopus) in the Ecuadorian Andes\n",
      "6. Two new fringe-limbed frogs of the genus Ecnomiohyla (Anura: Hylidae) from Panama\n",
      "7. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "8. New species amongst Sri Lanka’s extinct shrub frogs (Amphibia: Rhacophoridae: Philautus)\n",
      "9. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "10. Dispersal success of European tree frogs thanks to habitat connectivity measures: A genetic evaluation\n",
      "------------------------------------------\n",
      "*** Making another query for the next page ***\n",
      "*** These next titles will be different! ***\n",
      "------------------------------------------\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 10 results on this page\n",
      "---------------\n",
      "1. Genomic imprinting or mutation and interclonal selection in triploid hybrid frogs? A comment on Tunner.\n",
      "2. Dispersal rates and distances in adult water frogs, Rana lessonae, R. ridibunda  and their hybridogenetic associate R. esculenta\n",
      "3. Patterns of natural selection on size at metamorphosis in water frogs.\n",
      "4. The ecology of coexistence in two closely related species of frogs (rana)\n",
      "5. Species boundaries among barred river frogs, Mixophyes (Anura: Myobatrachidae) in north­eastern Australia, with descriptions of two new species\n",
      "6. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "7. Comparative larval morphology in Madagascan frogs of the genus Guibemantis (Amphibia: Mantellidae)\n",
      "8. Evolutionary history of Lost World frogs\n",
      "9. The evolution of colour in myobatrachid frogs\n",
      "10. A taxonomic revision of frogs of the genus Cryptobatrachus (Anura: Hemiphractidae)\n"
     ]
    }
   ],
   "source": [
    "basic_query = {\n",
    "    \"query\": {\n",
    "        \"query_string\": {\n",
    "            \"query\": \"frogs\"\n",
    "        }\n",
    "    }\n",
    "}\n",
    "\n",
    "query_results = query_share(SHARE_API, basic_query)\n",
    "print_numbered_results(query_results)\n",
    "\n",
    "print('------------------------------------------')\n",
    "print('*** Making another query for the next page ***')\n",
    "print('*** These next titles will be different! ***')\n",
    "print('------------------------------------------')\n",
    "\n",
    "basic_query['from'] = 10  # Add the 'from' parameter to the query to pick up at the next page of results\n",
    "\n",
    "query_results = query_share(SHARE_API, basic_query)\n",
    "print_numbered_results(query_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pagination with sharepa\n",
    "\n",
    "You can also use sharepa to paginate through all of the results in your query, and to access slices of your query at any time. The ShareSearch object returns a generator that you can use to access all results, using slices.\n",
    "\n",
    "First, we'll redefine our helper function for nicer printing with data returned from sharepa."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def print_numbered_sharepa_results(search_obj):\n",
    "    results = search_obj.execute()\n",
    "    print(\n",
    "        'There are {} total results and {} results on this page'.format(\n",
    "            search_obj.count(),\n",
    "            len(results.hits)\n",
    "        )\n",
    "    )\n",
    "    print('---------------')    \n",
    "    for result in enumerate(results.hits):\n",
    "        print('{}. {}'.format(result[0] + 1, result[1]['title']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sharepa import ShareSearch\n",
    "from sharepa.helpers import pretty_print\n",
    "\n",
    "frogs_search = ShareSearch()\n",
    "\n",
    "frogs_search = frogs_search.query(\n",
    "    'query_string',\n",
    "    query='frogs'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 10 results on this page\n",
      "---------------\n",
      "1. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "2. New genus of diminutive microhylid frogs from Papua New Guinea\n",
      "3. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "4. Two new frogs of the genus Platymantis (Anura: Ceratobatrachidae) from the Isabel Island group, Solomon Islands\n",
      "5. A new species of Sycorax Curtis (Diptera, Psychodidae, Sycoracinae) collected on harlequin frogs (Anura: Bufonidae, Atelopus) in the Ecuadorian Andes\n",
      "6. Two new fringe-limbed frogs of the genus Ecnomiohyla (Anura: Hylidae) from Panama\n",
      "7. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "8. New species amongst Sri Lanka’s extinct shrub frogs (Amphibia: Rhacophoridae: Philautus)\n",
      "9. Genetic evidence for human-mediated introduction of Anatolian water frogs (<i>Pelophylax</i> cf. <i>bedriagae</i>) to Cyprus (Amphibia: Ranidae)\n",
      "10. Dispersal success of European tree frogs thanks to habitat connectivity measures: A genetic evaluation\n"
     ]
    }
   ],
   "source": [
    "print_numbered_sharepa_results(frogs_search)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 106 total results and 10 results on this page\n",
      "---------------\n",
      "1. Genomic imprinting or mutation and interclonal selection in triploid hybrid frogs? A comment on Tunner.\n",
      "2. Dispersal rates and distances in adult water frogs, Rana lessonae, R. ridibunda  and their hybridogenetic associate R. esculenta\n",
      "3. Patterns of natural selection on size at metamorphosis in water frogs.\n",
      "4. The ecology of coexistence in two closely related species of frogs (rana)\n",
      "5. Species boundaries among barred river frogs, Mixophyes (Anura: Myobatrachidae) in north­eastern Australia, with descriptions of two new species\n",
      "6. Reproductive features of three maternal-caring species of <i>Leptodactylus</i> (Anura: Leptodactylidae) with a report on alloparental care in frogs\n",
      "7. Comparative larval morphology in Madagascan frogs of the genus Guibemantis (Amphibia: Mantellidae)\n",
      "8. Evolutionary history of Lost World frogs\n",
      "9. The evolution of colour in myobatrachid frogs\n",
      "10. A taxonomic revision of frogs of the genus Cryptobatrachus (Anura: Hemiphractidae)\n"
     ]
    }
   ],
   "source": [
    "print_numbered_sharepa_results(frogs_search[10:20])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregations\n",
    "\n",
    "While searching for individual results is useful, sharepa also lets you make aggregation queries that give you results across the entirety of the SHARE dataset at once. This is useful if you're curious about the completeness of data sets.\n",
    "\n",
    "For example, we can find the number of documents per source that are missing tags."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "missing_tags_aggregation = {\n",
    "    \"query\": {\n",
    "        \"bool\": {\n",
    "            \"must_not\": {\n",
    "                \"exists\": {\n",
    "                    \"field\": \"tags\"\n",
    "                  }\n",
    "            }\n",
    "        }\n",
    "    },\n",
    "    \"aggregations\": {\n",
    "        \"sources\": {\n",
    "            \"terms\": {\n",
    "                \"field\": \"sources\", # A field where the SHARE source is stored                \n",
    "                \"min_doc_count\": 0, \n",
    "                \"size\": 0  # Will return all sources, regardless if there are results\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "providers.org.crossref has 10948 documents without tags\n",
      "providers.gov.scitech has 648 documents without tags\n",
      "providers.com.figshare has 1 documents without tags\n",
      "providers.au.uow has 0 documents without tags\n",
      "providers.be.ghent has 0 documents without tags\n",
      "providers.ca.umontreal has 0 documents without tags\n",
      "providers.com.nature has 0 documents without tags\n",
      "providers.edu.asu has 0 documents without tags\n",
      "providers.edu.boise_state has 0 documents without tags\n",
      "providers.edu.calhoun has 0 documents without tags\n",
      "providers.gov.nist has 0 documents without tags\n",
      "providers.org.arxiv.oai has 0 documents without tags\n",
      "providers.org.datacite has 0 documents without tags\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    }
   ],
   "source": [
    "results_without_tags = query_share(SHARE_API, missing_tags_aggregation)\n",
    "\n",
    "missing_tags_counts = results_without_tags['aggregations']['sources']['buckets']\n",
    "\n",
    "for source in missing_tags_counts:\n",
    "    print('{} has {} documents without tags'.format(source['key'], source['doc_count'], ))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This information isn't terribly useful if we don't also know how many documents are in each source.\n",
    "\n",
    "Let's get that information as well, along stats for what percentage of documents from each source are missing titles. \n",
    "\n",
    "We'll do this with an elasticsearch \"sigificant terms\" aggregation. We're only interested in results that have 1 document or more, meaning all documents from the other sources have titles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "no_tags_query = {\n",
    "    \"query\": {\n",
    "        \"bool\": {\n",
    "            \"must_not\": {\n",
    "                \"exists\": {\n",
    "                    \"field\": \"tags\"\n",
    "                  }\n",
    "            }\n",
    "        }\n",
    "    },\n",
    "    \"aggs\": {\n",
    "        \"sources\":{\n",
    "            \"significant_terms\":{\n",
    "                \"field\": \"sources\", # A field where the SHARE source is stored                \n",
    "                \"min_doc_count\": 0, \n",
    "                \"size\": 0,  # Will return all sources, regardless if there are results\n",
    "                \"percentage\": {} # This will make the \"score\" parameter a percentage\n",
    "            }\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py:821: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html\n",
      "  InsecureRequestWarning)\n"
     ]
    }
   ],
   "source": [
    "docs_with_no_tags_results = query_share(SHARE_API, no_tags_query)\n",
    "docs_with_no_tags = docs_with_no_tags_results['aggregations']['sources']['buckets']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100.00% (or 1/1) of documents from providers.com.figshare have no tags\n",
      "99.87% (or 10948/10962) of documents from providers.org.crossref have no tags\n",
      "44.78% (or 648/1447) of documents from providers.gov.scitech have no tags\n"
     ]
    }
   ],
   "source": [
    "for source in docs_with_no_tags:\n",
    "    print(\n",
    "        '{}% (or {}/{}) of documents from {} have no tags'.format(\n",
    "            format(source['score']*100, '.2f'),\n",
    "            source['doc_count'],\n",
    "            source['bg_count'],\n",
    "            source['key']\n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Aggregations with sharepa\n",
    "\n",
    "You can also use sharepa to do aggregations.\n",
    "\n",
    "Let's make a sharepa search object that will give us the number of documents per sourcethat don't have language specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SignificantTerms(field='sources', min_doc_count=1, percentage={}, size=0)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "no_language_search = ShareSearch()\n",
    "\n",
    "no_language_search = no_language_search.query(\n",
    "    'bool',\n",
    "    must_not={\"exists\": {\"field\": \"language\"}}\n",
    ")\n",
    "\n",
    "no_language_search.aggs.bucket(\n",
    "    'sources',  # Every aggregation needs a name\n",
    "    'significant_terms',  # There are many kinds of aggregations\n",
    "    field='sources',  # We store the source of a document in its type, so this will aggregate by source\n",
    "    min_doc_count=1,\n",
    "    percentage={},\n",
    "    size=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see which query is actually going to be sent to elasticsearch by printing out the query. This is very similar to the queries we were defining by hand up above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"query\": {\n",
      "        \"bool\": {\n",
      "            \"must_not\": [\n",
      "                {\n",
      "                    \"exists\": {\n",
      "                        \"field\": \"language\"\n",
      "                    }\n",
      "                }\n",
      "            ]\n",
      "        }\n",
      "    },\n",
      "    \"aggs\": {\n",
      "        \"sources\": {\n",
      "            \"significant_terms\": {\n",
      "                \"percentage\": {},\n",
      "                \"field\": \"sources\",\n",
      "                \"size\": 0,\n",
      "                \"min_doc_count\": 1\n",
      "            }\n",
      "        }\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "pretty_print(no_language_search.to_dict())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100.00% of documents from providers.au.uow do not have language\n",
      "100.00% of documents from providers.com.figshare do not have language\n",
      "100.00% of documents from providers.gov.nist do not have language\n",
      "100.00% of documents from providers.edu.boise_state do not have language\n",
      "99.87% of documents from providers.org.crossref do not have language\n",
      "99.64% of documents from providers.org.arxiv.oai do not have language\n",
      "94.03% of documents from providers.edu.asu do not have language\n",
      "76.32% of documents from providers.edu.calhoun do not have language\n",
      "56.66% of documents from providers.org.datacite do not have language\n",
      "41.10% of documents from providers.be.ghent do not have language\n",
      "20.00% of documents from providers.ca.umontreal do not have language\n",
      "0.43% of documents from providers.gov.scitech do not have language\n"
     ]
    }
   ],
   "source": [
    "aggregated_results = no_language_search.execute()\n",
    "\n",
    "for source in aggregated_results.aggregations['sources']['buckets']:\n",
    "    print(\n",
    "        '{}% of documents from {} do not have language'.format(\n",
    "            format(source['score']*100, '.2f'),\n",
    "            source['key'] \n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Top tags \n",
    "\n",
    "Let's do an elasticsearch query to find out what the most used tags are used in the dataset across all sources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "    {\n",
      "        \"key\": \"dataset\",\n",
      "        \"doc_count\": 403057\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"text\",\n",
      "        \"doc_count\": 358318\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"ethz\",\n",
      "        \"doc_count\": 310978\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"cdl\",\n",
      "        \"doc_count\": 285391\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"tib\",\n",
      "        \"doc_count\": 257715\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"image\",\n",
      "        \"doc_count\": 252736\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"application\",\n",
      "        \"doc_count\": 212269\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"cdl.digsci\",\n",
      "        \"doc_count\": 164037\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"pdf\",\n",
      "        \"doc_count\": 152197\n",
      "    },\n",
      "    {\n",
      "        \"key\": \"bild\",\n",
      "        \"doc_count\": 130861\n",
      "    }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "top_tag_search = ShareSearch()\n",
    "\n",
    "top_tag_search.aggs.bucket(\n",
    "    'tagsTermFilter',  # Every aggregation needs a name\n",
    "    'terms',  # There are many kinds of aggregations\n",
    "    field='tags',  # We store the source of a document in its type, so this will aggregate by source\n",
    "    min_doc_count=1,\n",
    "    exclude= \"of|and|or\",\n",
    "    size=10\n",
    ")\n",
    "\n",
    "# pretty_print(top_tag_search.to_dict())\n",
    "\n",
    "top_tag_results_executed = top_tag_search.execute()\n",
    "top_tag_results = top_tag_results_executed.aggregations.tagsTermFilter.to_dict()['buckets']\n",
    "\n",
    "pretty_print(top_tag_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Plotting\n",
    "\n",
    "Sharepa has some basic functions to get you started making plots using matplotlib and pandas.\n",
    "\n",
    "### Creating a dataframe from sharepa data\n",
    "\n",
    "Raw sharepa data is in the same format as elasticsearch results, represented as a nested structure. To convert the data into a format that pandas can recognize, we have to convert it into a dataframe.\n",
    "\n",
    "Let's take our top tags aggregation, make it into a pandas data frame, and plot a bar graph. Then, we'll plot the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>doc_count</th>\n",
       "      <th>key</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>403057</td>\n",
       "      <td>dataset</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>358318</td>\n",
       "      <td>text</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>310978</td>\n",
       "      <td>ethz</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>285391</td>\n",
       "      <td>cdl</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>257715</td>\n",
       "      <td>tib</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>252736</td>\n",
       "      <td>image</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>212269</td>\n",
       "      <td>application</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>164037</td>\n",
       "      <td>cdl.digsci</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>152197</td>\n",
       "      <td>pdf</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>130861</td>\n",
       "      <td>bild</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   doc_count          key\n",
       "0     403057      dataset\n",
       "1     358318         text\n",
       "2     310978         ethz\n",
       "3     285391          cdl\n",
       "4     257715          tib\n",
       "5     252736        image\n",
       "6     212269  application\n",
       "7     164037   cdl.digsci\n",
       "8     152197          pdf\n",
       "9     130861         bild"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "top_tags_dataframe = pd.DataFrame(top_tag_results)\n",
    "top_tags_dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAE8CAYAAAArE33IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X+8VXWd7/HXG4hMAwIbgUB+mGDamIqBOXVvJ72ImqnV\nJU85QkUzc9PSah5zR7k3AU1L77Wxug+dblGC2SBRjVgOkCOnH/5CFMMRQqwLCAI2QKQ5/gA/94/1\nPbA46/z07L32lv1+Ph77wdrfvb5rffYBzmet7/e7vl9FBGZmZnl9ah2AmZnVHycHMzMrcHIwM7MC\nJwczMytwcjAzswInBzMzK+h2cpDUR9IqSYvT+1mSNkt6JL3OzO17haT1ktZKOiNXPkHSaklPSLox\nV95f0oJU535Jo3KfTU/7r5M0rfdf2czMutKTO4fLgMfblH01Iiak1xIASccCHwGOBc4CbpKktP/N\nwIyIGA+MlzQllc8AdkbEOOBG4Pp0rMHAlcBE4BRglqRBPf2SZmbWM91KDpJGAmcD3277UTu7nwcs\niIg9EbEBWA9MkjQMGBARD6X95gPn5+rMS9uLgNPS9hRgWUTsjog/AMuAfXcoZmZWHd29c/gH4O+A\nto9Tf0bSo5K+nbuiHwE8ldtnSyobAWzOlW9OZQfUiYi9wG5JQzo5lpmZVVG/rnaQ9H5ge0Q8Kqkp\n99FNwFUREZK+BNwAfKpCcbV3R9JZjJ4DxMzsVYiIdn/fdufO4d3AuZJ+B/wTcJqk+RHx+9g/MdO3\ngElpewtwZK7+yFTWUfkBdST1BQZGxM5UPqqDOm2/YK9es2bN6vUxKvGqhzjqIYZ6iaMeYqiXOOoh\nhnqJox5iqEQcnekyOUTEzIgYFRFHAc3APRExLfUhtPoQ8G9pezHQnEYgjQWOBlZExDay5qJJqYN6\nGnBHrs70tD0VuCdtLwUmSxqUOqcnpzIzM6uiLpuVOnG9pBOBV4ANwN8ARMQaSQuBNcDLwMWxP0Vd\nAtwCHALcFWmEEzAXuFXSemAHWRIiInZJuhpYSdbfMSeyjmkzM6uiHiWHiPg58PO03eEzBxHxZeDL\n7ZQ/DBzfTvmLZMNf2zvWLWQJpaqampqqfYpuqYc46iEGqI846iEGqI846iEGqI846iEGqG4c6qrd\n6bVAUhwM38PMrEySiA46pHvTrGRmBsCYMWPYuHFjrcOwDowePZoNGzb0qI7vHMys19IVaK3DsA50\n9PfT2Z2DJ94zM7MCJwczMytwcjAzswInBzMzK3ByMDOzAicHM6uKYcPGIKlqr2HDxrzq2D7xiU9w\n5ZVXVu7LHoScHMysKrZv30g26011Xtnxbc6cOUybVvlFMp0czMyswMnBzA56q1at4uSTT2bQoEE0\nNzfzwgsv7PvsW9/6FuPGjePNb34z559/Plu3bt332eOPP84ZZ5zB4YcfzvDhw/nKV77S6XleeeUV\nrr32Wo4++mgGDRrExIkT2bIlW2XgvvvuY9KkSQwePJhTTjmF+++/f1+9sWPHcs899+x7P2fOHC66\n6CIANm7cSJ8+fZg/fz6jR4/miCOO4NprrwVg6dKlXHvttdx+++0MGDCAk046qfc/rMTJwcwOai+/\n/DIf/OAHmT59Ojt37mTq1Kn88Ic/BGD58uXMnDmTRYsWsXXrVkaNGkVzczMAzz33HJMnT+bss89m\n69atPPnkk5x++umdnuuGG27g9ttvZ8mSJezevZvvfOc7HHrooezatYtzzjmHz33uc+zYsYPPf/7z\nvP/972fXrl0dHitb2WC/e++9l/Xr13P33Xdz1VVXsW7dOqZMmcLMmTO54IILePbZZ1m1alUvf1r7\nOTmY2UHtgQceYM+ePVx66aX07duXD3/4w0ycOBGA2267jRkzZnDCCSfwute9ji9/+cs88MADbNq0\niZ/85CcMHz6cz33uc/Tv35/DDjtsX72OzJ07l2uuuYajjz4agOOPP57Bgwfz05/+lPHjx/Oxj32M\nPn360NzczNve9jbuvPPObn0HScyePZv+/fvzjne8gxNOOIFf//rXvfvBdMHJwcwOak8//TQjRhy4\n9Pzo0aOJCJ5++mlGjx69r/ywww5jyJAhbNmyhaeeeoq3vvWtPTrXU089xVFHHdVuDPnztMbQ2uTU\nHUOHDt23feihh/Lcc8/1KLaecnIws4Pa8OHDC7+EN23ahCRGjBhxwGylf/rTn9ixYwcjRozgyCOP\n5Le//W2PzjVq1Kh267zlLW8pzIq6adOmfUnrsMMO4/nnn9/32bZt27p9zrbNT5XS7eQgqY+kRyQt\nTu8HS1omaZ2kpZIG5fa9QtJ6SWslnZErnyBptaQnJN2YK+8vaUGqc7+kUbnPpqf910mq/HgtM6uK\noUNHA6raKzt+10499VT69evHN77xDfbs2cOPfvQjVqxYAUBzczO33HILq1ev5sUXX2TmzJm8613v\nYtSoUZxzzjls27aNr3/967z00ks899xz++p1ZMaMGXzxi1/kySefBOCxxx5j165dnH322axfv54F\nCxawd+9ebr/9dtauXcs555wDwIknnsiCBQvYs2cPK1euZNGiRQcct7MZb4cOHcqGDRsqPytuDxai\n/jzwPWBxen8d8N/T9t8DX0nbxwGryNaKGAM8yf6pwR8EJqbtu4ApafvTwE1p+wJgQdoeDPwWGAS8\nqXW7ndjCzGqn3v8PPvzww3HSSSfFwIEDo7m5OZqbm+OLX/xiRER885vfjLe+9a1x+OGHxwc+8IHY\nsmXLvnqPP/54nH766TF48OAYPnx4XHfddZ2eZ+/evXHNNdfE2LFjY+DAgTFp0qR9x7v33nvj5JNP\njje96U3xzne+M+6777599X73u9/FKaecEgMGDIhzzjknLrvssrjooosiImLDhg3Rp0+f2Lt37779\n3/e+98XcuXMjImLHjh3xnve8JwYPHhwnn3xyu3F19PeTytv9nd+t9RwkjQS+C1wDfCEizpX0G+C9\nEbFd0jCgJSLeJunydMLrUt1/AWYDG4F7IuK4VN6c6n9a0hJgVkQ8KKkvsDUijsjvk+rcnM5ze5v4\nojvfw8yqw+s51LdqrufwD8DfkT2a2GpoRGwHiIhtwBGpfATwVG6/LalsBLA5V745lR1QJyL2Arsl\nDenkWD1WiUf5e/O4vpnZa0mXy4RKej+wPSIeldTUya6VvGzocQ/L7Nmz9203NTUVFt7e/yj/q7d9\ne3U6fszstePss8/ml7/85b6O4IhAEjNnzuTyyy+vcXSda2lpoaWlpVv7dtmsJOla4C+BPcAbgAHA\nj4F3Ak25ZqXlEXFsO81KS4BZZM1KyyPi2FTe3Walpoj4b6nOP6Zj9LhZKfuL7G3+8q2zWXvcrFTf\nqtKsFBEzI2JURBwFNJP1G1wE3Al8PO02HbgjbS8GmtMIpLHA0cCK1PS0W9IkZb+pp7WpMz1tTwVa\nnyNfCkyWNEjSYGByKjMzsyrqslmpE18BFkr6JNldwUcAImKNpIXAGuBl4OLcZf0lwC3AIcBdEbEk\nlc8FbpW0HthBloSIiF2SrgZWkl32z4mIP/QiZjMz64ZujVaqd25WMqutMWPGsHGjp9CuV6NHjy48\nhAedNys5OfTsTE4OZnbQqMRQVjMzayBODmZmVuDkYGZmBU4OZmZW4ORgZmYFTg5mZlbg5GBmZgVO\nDmZmVuDkYGZmBU4OZmZW4ORgZmYFTg5mZlbg5GBmZgVODmZmVuDkYGZmBV0mB0mvl/SgpFWSHpM0\nK5XPkrRZ0iPpdWauzhWS1ktaK+mMXPkESaslPSHpxlx5f0kLUp37JY3KfTY97b9O0rTKffXyDRs2\nBkm9eg0bNqbWX8PMGkC3FvuRdGhEPC+pL3AvcClwFvBsRHy1zb7HAt8HJgIjgbuBcRERkh4EPhMR\nD0m6C/haRCyV9Gng+Ii4WNIFwAcjojmtG70SmAAIeBiYEBG725zzNbHYTz3EYGbWqteL/UTE82nz\n9WTrTrf+dmrvoOcBCyJiT0RsANYDkyQNAwZExENpv/nA+bk689L2IuC0tD0FWBYRu9Pa0cuAfXco\nZmZWHd1KDpL6SFoFbAN+lvsF/xlJj0r6tqRBqWwE8FSu+pZUNgLYnCvfnMoOqBMRe4HdkoZ0ciwz\nM6uift3ZKSJeAU6SNBD4saTjgJuAq1Jz0ZeAG4BPVSiudm9zOjN79ux9201NTTQ1NVUoFDOzg0NL\nSwstLS3d2rdbyaFVRPxRUgtwZpu+hm8Bd6btLcCRuc9GprKOyvN1nk79GgMjYqekLUBTmzrL24st\nnxzMzKyo7YXznDlzOty3O6OV3tzaZCTpDcBk4DepD6HVh4B/S9uLgeY0AmkscDSwIiK2kTUXTVLW\nMzsNuCNXZ3rangrck7aXApMlDUqd05NTmZmZVVF37hyGA/Mk9SFLJrdHxF2S5ks6EXgF2AD8DUBE\nrJG0EFgDvAxcnBtKdAlwC3AIcFdELEnlc4FbJa0HdgDN6Vi7JF1NNmIpgDmpY9rMzKqoW0NZ652H\nspqZ9Vyvh7KamVljcXIwM7MCJwczMytwcjAzswInBzMzK3ByMDOzAicHMzMrcHIwM7MCJwczMytw\ncjAzswInBzMzK3ByMDOzAicHMzMrcHIwM7MCJ4cGNGzYGCT16jVs2Jhafw0zqyKv59CzMx0U6znU\nSxxmVlu9Ws9B0uslPShplaTHJM1K5YMlLZO0TtLS1qVE02dXSFovaa2kM3LlEyStlvSEpBtz5f0l\nLUh17pc0KvfZ9LT/OknTXu0PwczMuq/L5BARLwLvi4iTgBOBsyRNAi4H7o6IY8jWfL4CQNJxwEeA\nY4GzgJvSmtEANwMzImI8MF7SlFQ+A9gZEeOAG4Hr07EGA1cCE4FTgFn5JGRmZtXRrT6HiHg+bb6e\nbN3pAM4D5qXyecD5aftcYEFE7ImIDcB6YJKkYcCAiHgo7Tc/Vyd/rEXAaWl7CrAsInantaOXAWf2\n6BuamVmPdSs5SOojaRWwDfhZ+gU/NCK2A0TENuCItPsI4Klc9S2pbASwOVe+OZUdUCci9gK7JQ3p\n5FhmZlZF/bqzU0S8ApwkaSDwY0lvp9ijWcneyXY7SDoze/bsfdtNTU00NTVVMBwzs9e+lpYWWlpa\nurVvt5JDq4j4o6QWsqad7ZKGRsT21GT0TNptC3BkrtrIVNZReb7O05L6AgMjYqekLUBTmzrL24st\nnxzMzKyo7YXznDlzOty3O6OV3tzaCSzpDcBkYC2wGPh42m06cEfaXgw0pxFIY4GjgRWp6Wm3pEmp\ng3pamzrT0/ZUsg5ugKXAZEmDUuf05FRmZmZV1J07h+HAPEl9yJLJ7RFxl6QHgIWSPglsJBuhRESs\nkbQQWAO8DFycewjhEuAW4BDgrohYksrnArdKWg/sAJrTsXZJuhpYSdZsNSd1TJuZWRX5IbienckP\nwVUwDjOrrV49BGdmZo3HycHMzAqcHKwmPPmfWX1zn0PPzuQ+hwrFUQ8xmDU69zmYmVmPODmYmVmB\nk4OZmRU4OZiZWYGTg5mZFTg5mJlZgZODmZkVODmYmVmBk4OZmRU4OZiZWYGTg5mZFXRnJbiRku6R\n9LikxyR9NpXPkrRZ0iPpdWauzhWS1ktaK+mMXPkESaslPSHpxlx5f0kLUp37JY3KfTY97b9O0rTK\nfXUzM+tId+4c9gBfiIi3A6cCn5H0tvTZVyNiQnotAZB0LNmqcMcCZwE3pWVBAW4GZkTEeGC8pCmp\nfAawMyLGATcC16djDQauBCYCpwCzWpcsNauE3s4O65lh7WDVZXKIiG0R8Wjafo5s/egR6eP2ZvM7\nD1gQEXsiYgOwHpgkaRgwICIeSvvNB87P1ZmXthcBp6XtKcCyiNidlgddBuy7QzHrre3bN5LNDvvq\nXll9s4NPj/ocJI0BTgQeTEWfkfSopG/nruhHAE/lqm1JZSOAzbnyzexPMvvqRMReYLekIZ0cy+yg\n4bUtrB51OzlIeiPZVf1l6Q7iJuCoiDgR2AbcUMG42p1f3Oxg1Nu7F9/BWDX0685OkvqRJYZbI+IO\ngIj4fW6XbwF3pu0twJG5z0amso7K83WeltQXGBgROyVtAZra1FneXoyzZ8/et93U1ERTU1N7u5mZ\nNayWlhZaWlq6tW+3VoKTNB/494j4Qq5sWERsS9ufByZGxMckHQfcRtaBPAL4GTAuIkLSA8ClwEPA\nT4GvR8QSSRcDfx4RF0tqBs6PiObUIb0SmEB2l7MSODn1P+Tj80pwr7E46iGGysRRDzFUJg5rPJ2t\nBNflnYOkdwMXAo9JWkX2r3gm8DFJJwKvABuAvwGIiDWSFgJrgJeBi3O/uS8BbgEOAe5qHeEEzAVu\nlbQe2AE0p2PtknQ1WVIIYE7bxGBmZpXnNaR7diZfLVcojnqIoTJx1EMMlYnDGo/XkDYzsx5xcjAz\nswInBzMzK3ByMDOzAicHMzMrcHIwM7MCJwczMytwcjAzswInBzMzK3ByMDOzAicHMzMrcHIwM7MC\nJwczMytwcjAzswInBzMDer+WtdexPrh4PYeenclrGFQojnqIoTJx1EMM9RKH15R4renVeg6SRkq6\nR9Ljkh6TdGkqHyxpmaR1kpZKGpSrc4Wk9ZLWSjojVz5B0mpJT0i6MVfeX9KCVOd+SaNyn01P+6+T\nNO3V/hDMzKz7utOstAf4QkS8HTgVuETS24DLgbsj4hjgHuAKgLSG9EeAY4GzgJuUXZIA3AzMiIjx\nwHhJU1L5DGBnRIwDbgSuT8caDFwJTCRbk3pWPgmZmVl1dJkcImJbRDyatp8D1gIjgfOAeWm3ecD5\naftcYEFE7ImIDcB6YJKkYcCAiHgo7Tc/Vyd/rEXAaWl7CrAsInantaOXAWe+mi9qZmbd16MOaUlj\ngBOBB4ChEbEdsgQCHJF2GwE8lau2JZWNADbnyjensgPqRMReYLekIZ0cy8zMqqhfd3eU9Eayq/rL\nIuI5SW17nirZE9VuB0lnZs+evW+7qamJpqamCoZjZvba19LSQktLS7f27VZykNSPLDHcGhF3pOLt\nkoZGxPbUZPRMKt8CHJmrPjKVdVSer/O0pL7AwIjYKWkL0NSmzvL2YswnBzMzK2p74TxnzpwO9+1u\ns9J3gDUR8bVc2WLg42l7OnBHrrw5jUAaCxwNrEhNT7slTUod1NPa1JmetqeSdXADLAUmSxqUOqcn\npzIzM6uiLu8cJL0buBB4TNIqsuajmcB1wEJJnwQ2ko1QIiLWSFoIrAFeBi7OPYRwCXALcAhwV0Qs\nSeVzgVslrQd2AM3pWLskXQ2sTOedkzqmzcysivwQXM/O5Ae/KhRHPcRQmTjqIYZ6icMPwb3W9Ooh\nODMzazxODmZmVuDkYGZmBU4OZmZW4ORgZmYFTg5mZlbg5GBmZgVODmZmVuDkYGZmBU4OZmZW4ORg\nZmYFTg5mZlbg5GBmZgVODmZWN4YNG4OkXr2GDRtT669xUPCU3T07k6eprlAc9RBDZeKohxjqJY56\niKEycTQKT9ltZmY90mVykDRX0nZJq3NlsyRtlvRIep2Z++wKSeslrZV0Rq58gqTVkp6QdGOuvL+k\nBanO/ZJG5T6bnvZfJ2laZb6ymZl1pTt3Dt8FprRT/tWImJBeSwAkHUu2XOixwFnATWm9aICbgRkR\nMR4YL6n1mDOAnRExDrgRuD4dazBwJTAROAWYJWnQq/mSZmbWM10mh4j4FbCrnY/aa6c6D1gQEXsi\nYgOwHpgkaRgwICIeSvvNB87P1ZmXthcBp6XtKcCyiNid1o1eBuy7QzEzs+rpTZ/DZyQ9KunbuSv6\nEcBTuX22pLIRwOZc+eZUdkCdiNgL7JY0pJNjmZlVVW9HTR0MI6b6vcp6NwFXRURI+hJwA/CpCsXU\nbs95V2bPnr1vu6mpiaampgqFY2aNZvv2jfRm1NT27a/q11jVtbS00NLS0q19X1VyiIjf595+C7gz\nbW8Bjsx9NjKVdVSer/O0pL7AwIjYKWkL0NSmzvKOYsonBzMzK2p74TxnzpwO9+1us5LIXdGnPoRW\nHwL+LW0vBprTCKSxwNHAiojYRtZcNCl1UE8D7sjVmZ62pwL3pO2lwGRJg1Ln9ORUZmZmVdblnYOk\n75NdwR8uaRMwC3ifpBOBV4ANwN8ARMQaSQuBNcDLwMW5p9MuAW4BDgHuah3hBMwFbpW0HtgBNKdj\n7ZJ0NbCS7P5uTuqYNjOzKvMT0j07k58KrlAc9RBDZeKohxjqJY56iKFe4nhtPKXtJ6TNzKxHnBzM\nzKzAycHMzAqcHMzMrMDJwczMCpwczMzqUK0XPvJQ1p6dycM3KxRHPcRQmTjqIYZ6iaMeYqiXOOoh\nhq7j8FBWMzPrEScHMzMrcHIwM7MCJwczMytwcjAzswInBzMzK3ByMDOzAicHMzMrcHIwM7OCLpOD\npLmStktanSsbLGmZpHWSlkoalPvsCknrJa2VdEaufIKk1ZKekHRjrry/pAWpzv2SRuU+m572Xydp\nWmW+spmZdaU7dw7fBaa0KbscuDsijiFb8/kKAEnHAR8BjgXOAm5Ka0YD3AzMiIjxwHhJrcecAeyM\niHHAjcD16ViDgSuBicApwKx8EjIzs+rpMjlExK+AXW2KzwPmpe15wPlp+1xgQUTsiYgNwHpgkqRh\nwICIeCjtNz9XJ3+sRcBpaXsKsCwidqe1o5cBZ/bgu5mZ2av0avscjoiI7QARsQ04IpWPAJ7K7bcl\nlY0ANufKN6eyA+pExF5gt6QhnRzLzMyqrF+FjlPJqV3bnSGwK7Nnz9633dTURFNTU4XCMTM7OLS0\ntNDS0tKtfV9tctguaWhEbE9NRs+k8i3Akbn9RqayjsrzdZ6W1BcYGBE7JW0BmtrUWd5RQPnkYGZm\nRW0vnOfMmdPhvt1tVhIHXtEvBj6etqcDd+TKm9MIpLHA0cCK1PS0W9Kk1EE9rU2d6Wl7KlkHN8BS\nYLKkQalzenIqMzOzKuvyzkHS98mu4A+XtAmYBXwF+IGkTwIbyUYoERFrJC0E1gAvAxfnVuG5BLgF\nOAS4KyKWpPK5wK2S1gM7gOZ0rF2SrgZWkjVbzUkd02ZmVmVeCa5nZ/LqZxWKox5iqEwc9RBDvcRR\nDzHUSxz1EEPXcXglODMz6xEnBzMzK3ByMDOzAicHMzMrcHIwM7MCJwczMytwcjAzswInBzMzK3By\nMDOzAicHMzMrcHIwM7MCJwczMytwcjAzswInBzMzK3ByMDOzAicHMzMr6FVykLRB0q8lrZK0IpUN\nlrRM0jpJSyUNyu1/haT1ktZKOiNXPkHSaklPSLoxV95f0oJU535Jo3oTr5mZdU9v7xxeAZoi4qSI\nmJTKLgfujohjyNaDvgJA0nFky4keC5wF3JTWkwa4GZgREeOB8ZKmpPIZwM6IGAfcCFzfy3jNzKwb\nepsc1M4xzgPmpe15wPlp+1xgQUTsiYgNwHpgkqRhwICIeCjtNz9XJ3+sRcDpvYzXzMy6obfJIYCf\nSXpI0qdS2dCI2A4QEduAI1L5COCpXN0tqWwEsDlXvjmVHVAnIvYCf5A0pJcxm5lZF/r1sv67I2Kr\npD8DlklaR3FF7N6ukJ3X7kLYALNnz9633dTURFNTUwVPa2b22tfS0kJLS0u39lVEZX53S5oFPAd8\niqwfYntqMloeEcdKuhyIiLgu7b8EmAVsbN0nlTcD742IT7fuExEPSuoLbI2II9o5d3T1PbLujd5+\nV9Gbn1c9xFAvcdRDDJWJox5iqJc46iGGeomjHmLoOg5JRES7F92vullJ0qGS3pi2DwPOAB4DFgMf\nT7tNB+5I24uB5jQCaSxwNLAiNT3tljQpdVBPa1NnetqeStbBbWZmVdabZqWhwI8lRTrObRGxTNJK\nYKGkT5LdFXwEICLWSFoIrAFeBi7OXe5fAtwCHALcFRFLUvlc4FZJ64EdQHMv4jUzs26qWLNSLblZ\n6bUXRz3EUJk46iGGeomjHmKolzjqIYau46hKs5KZmR28nBzMzKzAycHMzAqcHMzMrMDJwczMCpwc\nzMyswMnBzMwKnBzMzKzAycHMzAqcHMzMrMDJwczMCpwczMyswMnBzMwKnBzMzKzAycHMzApeE8lB\n0pmSfiPpCUl/X+t4zMwOdnWfHCT1Af4PMAV4O/BRSW+r/JlaKn/IV6Wl1gFQHzFAfcTRUusAkpZa\nB0B9xAD1EUdLrQNIWqp25LpPDsAkYH1EbIyIl4EFwHmVP01L5Q/5qrTUOgDqIwaojzhaah1A0lLr\nAKiPGKA+4mipdQBJS9WO/FpIDiOAp3LvN6cyMzOrktdCcjAzs5Kpt4tgV5ukdwGzI+LM9P5yICLi\nutw+9f0lzMzqVESovfLXQnLoC6wDTge2AiuAj0bE2poGZmZ2EOtX6wC6EhF7JX0GWEbWDDbXicHM\nrLrq/s7BzMzK5w5pMzMrcHIwM7OChk4Oki7rTtnBHkM655HtlA0rO456IGmCpEslfVbShFrHY7Ul\naWr6c2ytYylTQ/c5SHokIia0KVsVESc1UgzpnHuAHwAzIuL5jmKr0rkfA9r7hyiyYcvvqHYMuViu\nBKYCP0pF5wM/iIgvlRVDLhYBFwJHRcRVkkYBwyJiRclxjABGkxvAEhG/KOG8p0XEPZI+1N7nEfGj\n9sqrEMcjETGhrP8PHcTwhc4+j4ivVvqcdT9aqRokfRT4GDBW0uLcRwOAnY0SQxuPAb8EfiVpakT8\nluyXcxnOKek83XEhcEJEvAAg6SvAo0DpyQG4CXgFOA24CngW+CEwsawAJF0HXACsAfam4gCqnhyA\n9wL3AB9o57NgfwKvth2SllH8v5oFEnFuCTEMSH8eQ/b33xrHB8iG91dcQyYH4D6yZybeDNyQK38W\nWN1AMeRFRNwk6dfAnWn221JuKyNiYxnn6aangUOAF9L71wNbahTLKemKdRVAROyS1L/kGM4HjomI\nF0s+LxExK/35ibLP3cb7gQnArRz4f7U0ETEHQNIvgAkR8Wx6Pxv4aTXO2ZDJIf0y2gicKmk0MC4i\n7pb0BuANZL+gS4lB0oyIWJP/TFIT5c/spRTXvZJOBxYCVZj9tp0TS8/SSSKKiIElxPCNFMNu4HFJ\nP0vvJ1PwutJAAAANZUlEQVSlK7NueDk9BBopxj8ju5Mo0++A1wGlJ4dWkq4Fro+IP6T3g4G/jYj/\nWcb5I+Il4AFJfxERvy/jnJ0YCryUe/9SKqu4hkwOrST9FfDXwBDgrcBI4B/JnsYuy0JJ84H/RXbF\nej3wTuDUEmMAOLt1IyK2Snof8BdlnDgiBgBIuprsbupWsmR1ITC8jBiAlenPh4Ef58pbSjp/e75O\nFssRkq4B/itQyi/EnOeBRyX9K7kEERGXlhjDWRExM3fuXZLOpqSfhaQ72Z+gC5+X1KzUaj6wQlLr\nv9HzgVuqcaJG75B+lGxK8AdbO4AlPRYRx5cYw2HAdcDJZO2KtwHXRUSpV4iSXg98GBjDgR2PV5UY\nw68j4oSuyhpJWrvkdLJk+a9lzw4gaXp75RExr8QYVgMTW5u20h3+yoh4e0nnf2/a/BAwDPheev9R\nYHtEfL6MOHLxTAD+U3r7i4hYVY3zNPSdA/BiRLzUejUgqR8ltbPnvAz8B1lz1iHA/ys7MSR3kDWp\nPEztmhD+JOlCsjU7guw/35/KOLGkhRHxkY5GTpU5YioX0xDgGeCfcmWvS+ualCIi5qV+jvGpaF2Z\n509uA/5V0nfT+08ApSWniPg5gKQbIuKduY/ulLSyg2oVlf4ttNqQXvs+i4iKD2Jp9OTwc0kzgTdI\nmgxcDNxZcgwPkf1inkjWOf2Pkj4cEVNLjmNk68y3NfQx4GvpFcC9ZAmiDK3PlqwF/i5XLrKmvlp4\nBDgS2JXieBOwTdJ24K8i4uFqB5D6v+aR/TIScKSk6WUMZW0VEdelgRL/JRVdHRFLyzp/zmGSjoqI\n38G+5x4OK+ncD5P9n2ht12q9gFHaPqriZ4yIhn2RPQT4V2Tj+xelbZUcwzvbKbuoBj+L/wscX+O/\nj3nA4Nz7IcB3So7hkXbKVtfo5/EtYEru/RnAN4F3kTWFlhHDw2SjlVrfjwceLvnncBjQJ20fA5wL\nvK4Gfx9nApvI+qFayBLmlLLjKOvV0H0Oeem2bWRElD6MVNJ7yEZMfVfSm4EBEfH/Sjp3azNKP2Ac\n2eiUF6nNA2iFh//KeiBQ0qfJ7hyPAn6b+2gAcG9E/GW1Y2gnpkL/l6TVEfEOSY9GxIklxLC67b+B\n9sqqHMPDZG3sg4FfkQ0eeCkiLiwrhhTHIcDfkg0YGQT8DPiHSM/EVPncb4uI33T0xH5EPFLpczZ0\ns5KkFrKrkH5kV0jPSLovSuxgkjSL7B/bMcB3gf5kHV7vLimEenoArY+kwRGxC/Yl7LL+jX4f+Bfg\ny8DlufJnowrtud20NT1vsiC9vwDYnoa3ltUvtVLSt9nfCXsh+0d2lUUR8bykGcDNEXF9GkxStvnA\nH8lGkUHWDHor2RP11fYFspGVN3Bgn1hrs9JplT5hQycHYFBE/FHSp4D5ETErjYwo0weBk8jal4mI\npyUN6LxK5UR6AE3SrRFxUf4zSbcCF7VbsTpuAO6X9IP0fipwTRknjojdZB3yZfVxdMfHgFnAP6f3\n96ayvsBHSorh08AlQOvQ1V+SPbldJkk6lSwxzUhlfUuOAeDPI+K43PvlktZ0uHcFRcRfp82zye5w\n30OWFH4J3FyNczZ6cugnaTjZf7T/UaMYXoqIUFrqNA1trYUDhgWmkVsnlxlARMxPoz9ar4I+FG0e\nEGwkEfHvwGc7+PjJkmJ4EfhqetXK54ArgB9HxOOSjgKW1yCORyS9KyIeAJB0CuXfRc2jePcynypc\nLDR6crgKWAr8KiIeSv/o1pccw0JJ3wTelB7K+yRZR2QpJF0BtI7Y+mNrMdmTl/+3rDhapWTQsAkh\nLz0R/d/JEvchreURUfEmhHbOXTdDeyMbSvrz3Pvfsf9OpkwnA/dJ2pTejwLWtf6MSvqZlHb30tDJ\nISJ+QDZSqfX978geBCvTn5GNlPojWb/Dlewfsld1EfFl4MuSriebfO+oiJijNANoWXFYu24Dbifr\nF/pvwHSgrOkbWof21rxPKv+Ecs5usqv2b5bRIZzUeqg3lHj30tCjldLogxkUr8w+WWIM7U3ZXepo\nkHTOfySbdfO0iDg2zV+zLCJKmwHUDiTp4Yg4Of/vQdJDZf6dSLouIv6+q7Iqx/A1souo1ocBLyC7\nmApgYNu+soNR7g7udWQXkZvS+9HAb9rcTVREQ985kI00+A0whayJ6UKyh6CqLj90sk0n+ACyjsey\nTYrazwBqB2p9EnmrpPeTzRg7pJP9q2Ey0DYRnNVOWTX9RZuEeGdrkpT0eIlx1FLpd3CNnhyOjoip\nks6LbJqA75P1/peh3oZO1sMMoHagL0kaRDa2/hvAQKCUYdZ1dvHyRkmjImJTim0U8Mb02UsdVzt4\nRA2mtW/05NB6ZfYHSX8ObAOOKOPEdTh0sh5mALWciPhJ2twNvK/k09fTxcvfki1C1boA1Vjg4jSy\nr7Q5lhpNo/c5fIpsZa3jyaa9fSPwxYj4Zi3jqpVazwBqB0pz93yW4ky5ZU4R3RrLERzYL7epk92r\ncf7Xs399kXUldkI3rEZPDmPbTlPRXplZLaTJ5uaSjSLb18SXhnaWFcMHyJ5xeAvZDLGjgbVRwnTZ\nqpM1pBtVozcr/ZBs+b+8RZT88JdZB16IiK93vVtVfYlsor+7I+IkZYtAlTXPVL2sId2QGjI5pOaT\ntwOD2lyVDCR362xWY19Lc28t48BV2Co+yVonXo6IHZL6SOoTEcsl3VjGiaN+1pBuSA2ZHMjGCZ9D\nNj9+/qrkWbJpu83qwfFkc1udxv5mpapMstaJP0h6I/AL4DZJz1DeAkxf6OzziKjllB4HvUbvczg1\nIu6vdRxm7ZH0JHBcZAvc1yqGw8hWKuxD9hzQIOC2iNhRwrlnpc1jyBbDWpzefwBYUYtp1BtJoyeH\nmj8hbdYRSf8M/HVEPFPDGMYCW1tHBylbv3loRGwoMYZfAO+PiGfT+wHATyPiP5cVQyPqU+sAauxW\nsvmDppBN7DWSrGnJrB68CfiNpKWSFre+So7hBxz4MORecvORlWQoBz7s9lIqsypq1D6HVrV8Qtqs\nK7O63qXq+uWbtSLipRpMqzIfWCHpx+n9+WTPJVkVNXpyqNkT0mZdKfN5hk78XtK5EbEYQNJ5wL+X\nGUBEXCPpX8iWCgX4RESsKjOGRtTofQ5+QtrqjqRfRcR7JD1LO0tCRsTAEmN5K9nU4W9J538KmBYR\npSw2ZLXTkMmhgyFySn+Gh8iZHSgNZyUinqt1LACSfhIRNV9r4mDWqM1KrWs0tztEriYRmdURSX8Z\nEd9reyElZddQdXAB5eeRqqwhk0NEzIF9Q+Qm5IbIzQZ+WsPQzOpF61rmAzrdq0YiYmutYzjYNWSz\nUitJ64B3pEXUW2d+XB0Rx9Q2MjPraP1q9ve9lLpaYqNpyDuHHA+RM2uHpE4n/IuIS0sIw30KNdTQ\ndw4Akiawf4jcLzxEzgwkTe/s84jwIjsHuYZPDmbWNUkDyZpySptBoJ2hvAcoc0hvI2r0ZiUz64Sk\ndwLfJeuYlqQ/AJ+MiIerfe6IGJBiuBrYSjbdjcgmABxe7fM3Ot85mFmHJK0GLomIX6b37wFuKrMz\nWNKvI+KErsqsshp94j0z69ze1sQAEBG/AvaUHMOfJF0oqW9adOhCSlpTopH5zsHMOpRWfXsD8E9k\n7f8XAC8A34NyVqWTNAb4GvDuFMO9wGURsbHa525kTg5m1iFJyzv5OCKi6qvSSZoHfC4idqX3Q4D/\n7XVXqssd0mbWoYh4X61jIHtQdVfrm4jYKemkWgbUCNznYGYdknS4pK9LekTSw5K+JunwksPoI2lw\nLqYh+MK26vwDNrPOLAB+AXw4vb8QuB34LyXGcANwv6TWFeimAteUeP6G5D4HM+uQpH+LiD9vU/ZY\nRBxfchzHAa39G/dExJoyz9+IfOdgZp1ZJqkZWJje/1dgadlBpGTghFAi3zmYWYfSFBaHAXtTUV/2\nP2NQ6qp0Vi4nBzPrVOoAHgcc0lpWJ+tbWxW5WcnMOpTWWb8MGAk8CrwLuA84vZZxWfV5KKuZdeYy\nsqV0N6ZnHk4Cdtc2JCuDk4OZdeaFiHgBspUSI+I3ZGuv20HOzUpm1pnNkt4E/DPwM0m7AM9p1ADc\nIW1m3SLpvcAgYElEvFTreKy6nBzMzKzAfQ5mZlbg5GBmZgVODmZmVuDkYNZLkkZLeqzWcZhVkpOD\nWWV4ZIcdVJwczCpI0lFpYZyJkq6X9KCkRyX9Vfp8nqRzc/t/T9IHahexWfucHMwqRNJ4YBEwHTgR\n+ENEnAJMAv5a0mhgLvCJtP9A4FTgp7WJ2KxjTg5mlXEE2VPEH42Ix4AzgGmSVgEPAkOAcRHxC+Do\ntNTmR4EfRsQrtQrarCOePsOsMnYDm4D/DKwDBHw2In7Wzr7zgYuAZuDjZQVo1hNODmaV8SLwQbKV\n054jWy3tYknLI2KPpHHA5oj4D2AesALYmiayM6s7Tg5mFRIR/yHpHGAZcDXwOPCIJAHPAOen/Z6R\ntBb4cc2CNeuC51YyK5mkQ4FfAxMi4tlax2PWHndIm5VI0unAGuDrTgxWz3znYGZmBb5zMDOzAicH\nMzMrcHIwM7MCJwczMytwcjAzs4L/Dzw7WUZ+c2X3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10dba2e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot\n",
    "%matplotlib inline\n",
    "\n",
    "top_tags_dataframe.plot(kind='bar', x='key', y='doc_count')\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Complex Queries and Dataframes\n",
    "\n",
    "Let's plot the number of document that each source has. We'll limit it to the top 30 sources to make sure that the graph is readable. Here we'll use the sharepa helper function bucket_to_dataframe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/erin/miniconda3/envs/share_tutorials/lib/python3.5/site-packages/ipykernel/__main__.py:22: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x10de66cc0>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEDCAYAAADJHVh5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmUVeWd7vHvUxQOKEMhYS4LRCBgciPQQmzUFBpFTS9F\nWl04G4l/BKNeM7QS7lJYepvA0mj0tiTaiOAQRLETW7lYTmU6uQ4gIioECZFZypZiWCCJVPG7f+xN\neSiqqF1VB5mez1q12Ofd77D3qWI/Z+93n3MUEZiZmWVRsL83wMzMDh4ODTMzy8yhYWZmmTk0zMws\nM4eGmZll5tAwM7PMGgwNSVMlVUhalFM2WdISSQslzZbUJmfdWEnL0vXn5JQPlLRI0keS7sspP0LS\nzLTNG5KOz1l3TVp/qaSrc8p7SHozXfdbSYXNfSLMzKxhWc40pgHDa5WVASdFxMnAMmAsgKT+wKVA\nP+A84EFJSttMAUZHRB+gj6RdfY4GKiOiN3AfMDntqwi4HTgFGALcIalt2mYScE/a16a0DzMz28ca\nDI2I+COwsVbZyxGxM334JtA9Xb4AmBkRVRGxgiRQBkvqDLSOiHlpvRnAiHT5QmB6uvwMcGa6PBwo\ni4jNEbGJJKjOTdedCcxOl6cDF2XYVzMza6Z8zGlcB8xJl7sBq3PWrU3LugFrcsrXpGW7tYmIamCz\npPb19SXpOGBjTmitAbrmYT/MzKwBzQoNSeOAHRHx2zxtD4AarpKpjpmZ5VmTJ5AlXQucz5eXkyA5\nGyjOedw9LauvPLfNOkktgDYRUSlpLVBaq81rEbFBUltJBenZRm5fdW2nP1zLzKwJImKPF+hZzzRE\nzqt7SecCPwMuiIi/59R7DhiV3hHVEzgReDsi1pNcdhqcToxfDfw+p8016fIlwKvp8ovA2WlAFAFn\np2UAr6V1Sdvu6qtOEdGonzvuuKPRbQ7EMTzOgTuGxzlwx/A4yU99GjzTkPQkySv+4yStAu4Afg4c\nAbyU3hz1ZkSMiYjFkmYBi4EdwJj4cvQbgEeBo4A5ETE3LZ8KPCZpGbABGJUe6DdKuhOYDwQwIZIJ\ncYDbgJnp+nfTPszMbB9rMDQi4vI6iqftpf5EYGId5e8A36yj/O8kt+nW1dejJEFTu/xjkttwm6xz\n5x5UVKysd/2ECRP2KOvUqYT161c0Z1gzs4PaYfuO8CQwop6f1+os31vINFZpaWne+vI4B98YHufA\nHcPj7J32du3qUCAp6trH5LJaY/dde73WZ2Z2qJBE1DER7o/fMKtDjx49WLkyf2eWZgeqkpISVqxY\nkbm+zzQa15vPNA4T6aus/b0ZZvtcfX/r9Z1pHLZzGmZm1ngODTMzy8yhYWZmmTk0zA5Tw4YN45FH\nHtnfm7Hfvf766xQXFzdc0QCHhllmnTv3QNI+++ncuUejt2nIkCH85S9/4eOPP2bQoEH53+mDRHMP\n/F9+7Y81xKFhltHe3xDa/J/Gvnm0qqqKVatWceKJJ/LOO+8c1qEREYfNgb+6unq/ju/QMDtIvf/+\n+/Tv3x+A+fPnM2DAgL3Wf+mll+jXrx9FRUXceOONu91mGRHcdddd9OjRg86dO3PttdeyZcuWmvV/\n/OMfGTp0KEVFRZSUlDBjxgxgz0tc06dP5/TTT695XFBQwJQpU+jTpw9t27bl9ttv569//StDhw6l\nXbt2jBo1iqqqqpr6zz//PAMGDKCoqIjTTjuN999/v2Zdz549ueeee/jWt75FUVERo0aN4osvvuDz\nzz/n/PPPZ926dbRu3Zo2bdqwfv165s2bxymnnELbtm3p0qULP/3pT+t9biKCiRMn8rWvfY0TTjiB\nJ598suZ57dy5827P1bPPPsvJJ59cZz9z5szhpJNOok2bNhQXF/PLX/6yZt3DDz9M79696dChAyNG\njOCTTz4BYOXKlRQUFLBz586aurnP6/Tp0znttNP48Y9/TIcOHWo+4ujhhx+mf//+tGnThm984xss\nXLgQgE8++YSLL76Yjh070qtXLx544IF697tJvopPWNyfP8ku7gkIiEb+1N2XHXrq+l037W8m/39f\n06ZNi3bt2kWrVq3i6KOPjqKioigsLIw2bdpEUVFRrFixYo82n332WbRu3TqeffbZqKqqinvvvTcK\nCwtj6tSpERExderU6N27d6xYsSK2bdsWI0eOjKuuuioiIlasWBGtW7eOp556KqqqqqKysjLee++9\niIgoLS2t6SMi4tFHH43TTz+95rGkGDFiRGzdujUWL14cRx55ZHz3u9+NFStWxJYtW6J///4xY8aM\niIhYsGBBdOzYMebNmxc7d+6MGTNmRI8ePeKLL76IiIgePXrEkCFDYv369bFx48bo169f/OY3v4mI\niPLy8iguLt5tn0899dR4/PHHIyJi27Zt8dZbb9X5fJaXl0dhYWH89Kc/jS+++CJef/31OOaYY+Kj\njz6KiIiTTjop5s6dW1P/oosuinvvvbfOvrp06RJ/+tOfIiJi06ZN8e6770ZExCuvvBIdOnSIhQsX\nxhdffBE33nhjnHHGGTXPb0FBQVRXV9f0k/u8Pvroo1FYWBj/9m//FtXV1fG3v/0tZs2aFd27d493\n3nknIiKWL18eq1atip07d8agQYPirrvuiqqqqvj444+jV69eUVZWVuf2RtT/d5eW73lMravwUPpx\naFhTHMihscvpp58e7733XqxatSoGDBiw17ozZsyIU089dbey7t271xyYzjrrrJgyZUrNuqVLl8YR\nRxwR1dXVMXHixBg5cmSd/WYJjTfeeKPm8aBBg2Ly5Mk1j3/yk5/ELbfcEhERP/zhD+P222/frf++\nffvGH/7wh4hIQuPJJ5+sWfcv//Iv8cMf/jAi6g6N73znOzF+/Pj47LPP6ntaatq2bNkytm/fXlN2\n6aWXxl133RUREZMmTYorrrgiIiI2bNgQrVq1ivXr19fZV0lJSTz00EOxZcuW3cpHjx4dt956a83j\nrVu3RsuWLWPlypWZQqOkpGS3/oYPHx7333//HuO/9dZbe9SdOHFiXHfddfXuf2NDw5enzA4iGzdu\npKioiHbt2vHGG29QWlpK3759Wbp0Ke3bt+f++++vs926dev2mCjOfbxu3TpKSkpqHpeUlFBVVUVF\nRQWrV6+mV69eTd7mjh071iwfffTRdOrUabfHW7duBZLLNPfccw/t27enffv2FBUVsWbNGtatW1dT\nP7dtq1atatrWZerUqSxdupSvf/3rDBkyhBdeeKHeukVFRRx11FE1j0tKSmrGvfLKK3n++efZvn07\ns2bN4owzzthtO3LNnj2bF154gZKSEoYNG8Zbb70F7Pn8HnPMMRx33HGsXVvv98ftpvbvrr7fycqV\nK1m7du1uz+HEiRP59NNPM42ThT97yuwgUlRUxMaNG3nqqacoLy9nypQpjBw5khtvvJFhw4bV265L\nly6sWrVqt7LVq1fXLHft2nW3z9pauXIlhYWFdOrUieLiYt5+++06+z3mmGP4/PPPax6vX7++qbtG\ncXEx48aNY+zYsY1uW9ckeK9evWrmJmbPns3FF19MZWUlRx999B51N27cyPbt22vWrVq1im9+M/km\nh65du3Lqqacye/ZsHn/8ccaMGVPvdgwaNIjf/e53VFdX88ADD3DJJZewatWqPZ7fbdu2sWHDBrp3\n714z5ueff86xxx4L7Pk81t6/4uJili9fvsf4xcXFnHDCCSxdurTebWwun2mYHYTeeecdBg4cCMC7\n775bs1yf733veyxevLjmgParX/1qtwPTZZddxr333suKFSvYunUr48aNY9SoURQUFHDFFVfwyiuv\n8Mwzz1BdXU1lZSXvvfceACeffDLPPvss27dv5y9/+QtTpzb9+9Cuv/56fv3rX9cE1LZt25gzZw7b\ntm1rsG2nTp3YsGHDbpP3TzzxBJ999hkAbdu2RRIFBXUf8iKSb7fbsWMH//Vf/8ULL7zAJZdcUrP+\nqquuYvLkyXzwwQeMHDmyzj527NjBk08+yZYtW2jRogWtW7emRYsWQPL8Tps2jUWLFvH3v/+dn//8\n53z729+muLiYDh060K1bNx5//HF27tzJI488Umcg5PrBD37A3XffzYIFCwBYvnw5q1evZvDgwbRu\n3ZrJkyfzt7/9jerqaj788EPmz5/f4HOYlUPD7CC0YMECBg4cSGVlJYWFhbRt23av9Y877jiefvpp\nbr31Vjp06MDy5cs57bTTatZfd911XHXVVZxxxhn06tWLVq1a1VzqKi4uZs6cOdx99920b9+eAQMG\nsGjRIgBuueUWWrZsSefOnfn+97/PlVdeudu4tV8h7+222EGDBvHwww/zox/9iPbt29OnTx+mT5+e\nqW3fvn257LLLOOGEE2jfvj3r169n7ty5NXcy3XLLLTz11FMceeSRdbbv0qULRUVFdO3alauuuorf\n/OY39OnTp2b9RRddxMqVKxk5cuRul7Fqe+yxx+jZsyft2rXjoYceqjnTOeuss7jzzjsZOXIk3bp1\n4+OPP2bmzJk17R5++GEmT55Mhw4dWLJkCUOHDq13DICLL76YcePGcfnll9OmTRsuuugiKisrKSgo\n4Pnnn2fhwoX07NmTjh07cv311+8Wps3lT7ltXG8c6s+XJer65M+Gvu2xufzNkAe2E088kYceeogz\nzzxzf29KXjX2U249p2GWkQ/oh6/Zs2dTUFBwyAVGUzg0zMz2YtiwYSxZsoTHH398f2/KAcGXpxrX\nmy9PHSb8JUx2uPCXMJmZ2T7j0DAzs8wcGmZmlpknws3qUFJScth81LYd3nI/3iQLT4Q3rjdPjprZ\nYcET4WZm1mwODTMzy6zB0JA0VVKFpEU5ZUWSyiQtlfSipLY568ZKWiZpiaRzcsoHSlok6SNJ9+WU\nHyFpZtrmDUnH56y7Jq2/VNLVOeU9JL2ZrvutJM/NmJl9BbKcaUwDhtcquw14OSL6Aq8CYwEk9Qcu\nBfoB5wEP6svZxCnA6IjoA/SRtKvP0UBlRPQG7gMmp30VAbcDpwBDgDtywmkScE/a16a0DzMz28ca\nDI2I+COwsVbxhcCuj5+cDoxIly8AZkZEVUSsAJYBgyV1BlpHxLy03oycNrl9PQPs+nCX4UBZRGyO\niE1AGXBuuu5MYHbO+Bc1tB9mZtZ8TZ3T6BgRFQARsR7Y9dVc3YDVOfXWpmXdgDU55WvSst3aREQ1\nsFlS+/r6knQcsDEidub01bWJ+2FmZo2Qr4nwfN6HmuXmeN9Ab2a2HzR1ArlCUqeIqEgvPe36Atq1\nQO6X2XZPy+orz22zTlILoE1EVEpaC5TWavNaRGyQ1FZSQXq2kdtXncaPH1+zXFpaSmlpab11zcwO\nR+Xl5ZSXlzdYL9Ob+yT1AP4zIr6ZPp5EMnk9SdKtQFFE3JZOhD9BMnHdDXgJ6B0RIelN4CZgHvAC\ncH9EzJU0BvhGRIyRNAoYERGj0onw+cBAkjOi+cCgiNgk6Sng2Yh4StIU4L2I+HU92+4395mZNVJ9\nb+5rMDQkPUnyiv84oAK4A/gd8DTJGcJK4NJ0shpJY0nuZtoB3BwRZWn5IOBR4ChgTkTcnJYfCTwG\nDAA2AKPSSXQkXQuMIzm63xURM9LynsBMoAh4F7gyInbUs/0ODTOzRmpyaBzsHBpmZo3njxExM7Nm\nc2iYmVlmDg0zM8vMoWFmZpk5NMzMLDOHhpmZZebQMDOzzBwaZmaWmUPDzMwyc2iYmVlmDg0zM8vM\noWFmZpk5NMzMLDOHhpmZZebQMDOzzBwaZmaWmUPDzMwyc2iYmVlmDg0zM8vMoWFmZpk5NMzMLDOH\nhpmZZebQMDOzzBwaZmaWmUPDzMwyc2iYmVlmDg0zM8vMoWFmZpk5NMzMLLNmhYakWyR9IGmRpCck\nHSGpSFKZpKWSXpTUNqf+WEnLJC2RdE5O+cC0j48k3ZdTfoSkmWmbNyQdn7PumrT+UklXN2c/zMws\nmyaHhqSuwI3AwIj4H0AhcBlwG/ByRPQFXgXGpvX7A5cC/YDzgAclKe1uCjA6IvoAfSQNT8tHA5UR\n0Ru4D5ic9lUE3A6cAgwB7sgNJzMz2zeae3mqBXCMpELgaGAtcCEwPV0/HRiRLl8AzIyIqohYASwD\nBkvqDLSOiHlpvRk5bXL7egY4M10eDpRFxOaI2ASUAec2c1/MzKwBTQ6NiFgH3AOsIgmLzRHxMtAp\nIirSOuuBjmmTbsDqnC7WpmXdgDU55WvSst3aREQ1sFlS+730ZWZm+1BhUxtKakdyJlACbAaelnQF\nELWq1n7cHGq4yp7Gjx9fs1xaWkppaWmeNsfM7NBQXl5OeXl5g/WaHBrAd4G/RkQlgKT/AP4RqJDU\nKSIq0ktPn6b11wLFOe27p2X1lee2WSepBdAmIiolrQVKa7V5rb4NzQ0NMzPbU+0X1BMmTKizXnPm\nNFYB35Z0VDqhfRawGHgOuDatcw3w+3T5OWBUekdUT+BE4O30EtZmSYPTfq6u1eaadPkSkol1gBeB\nsyW1TSfFz07LzMxsH2rymUZEvC3pGeBdYEf670NAa2CWpOuAlSR3TBERiyXNIgmWHcCYiNh16eoG\n4FHgKGBORMxNy6cCj0laBmwARqV9bZR0JzCf5PLXhHRC3MzM9iF9edw+NEmKuvYxOalp7L6LQ/35\nMjOD5BgZEXvMI/sd4WZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWXm0DAzs8wc\nGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNo\nmJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLrFmh\nIamtpKclLZH0oaQhkooklUlaKulFSW1z6o+VtCytf05O+UBJiyR9JOm+nPIjJM1M27wh6ficddek\n9ZdKuro5+2FmZtk090zjV8CciOgHfAv4M3Ab8HJE9AVeBcYCSOoPXAr0A84DHpSktJ8pwOiI6AP0\nkTQ8LR8NVEZEb+A+YHLaVxFwO3AKMAS4IzeczMxs32hyaEhqA5weEdMAIqIqIjYDFwLT02rTgRHp\n8gXAzLTeCmAZMFhSZ6B1RMxL683IaZPb1zPAmenycKAsIjZHxCagDDi3qftiZmbZNOdMoyfwmaRp\nkhZIekhSK6BTRFQARMR6oGNavxuwOqf92rSsG7Amp3xNWrZbm4ioBjZLar+XvszMbB8qbGbbgcAN\nETFf0r0kl6aiVr3aj5tDDVfZ0/jx42uWS0tLKS0tzdPmmJkdGsrLyykvL2+wXnNCYw2wOiLmp49n\nk4RGhaROEVGRXnr6NF2/FijOad89LauvPLfNOkktgDYRUSlpLVBaq81r9W1obmiYmdmear+gnjBh\nQp31mnx5Kr0EtVpSn7ToLOBD4Dng2rTsGuD36fJzwKj0jqiewInA2+klrM2SBqcT41fXanNNunwJ\nycQ6wIvA2endW0XA2WmZmZntQ8050wC4CXhCUkvgr8D3gRbALEnXAStJ7pgiIhZLmgUsBnYAYyJi\n16WrG4BHgaNI7saam5ZPBR6TtAzYAIxK+9oo6U5gPsnlrwnphLiZme1D+vK4fWiSFHXtY3JS09h9\nF4f682VmBskxMiL2mEf2O8LNzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFh\nZmaZOTTMzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZ\nmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWXm0DAzs8wcGmZm\nllmzQ0NSgaQFkp5LHxdJKpO0VNKLktrm1B0raZmkJZLOySkfKGmRpI8k3ZdTfoSkmWmbNyQdn7Pu\nmrT+UklXN3c/zMysYfk407gZWJzz+Dbg5YjoC7wKjAWQ1B+4FOgHnAc8KElpmynA6IjoA/SRNDwt\nHw1URkRv4D5gctpXEXA7cAowBLgjN5zMzGzfaFZoSOoOnA/8e07xhcD0dHk6MCJdvgCYGRFVEbEC\nWAYMltQZaB0R89J6M3La5Pb1DHBmujwcKIuIzRGxCSgDzm3OvpiZWcOae6ZxL/AzIHLKOkVEBUBE\nrAc6puXdgNU59damZd2ANTnla9Ky3dpERDWwWVL7vfRlZmb7UJNDQ9L3gIqIWAhoL1VjL+saPWwe\n+zIzs0YqbEbbocAFks4HjgZaS3oMWC+pU0RUpJeePk3rrwWKc9p3T8vqK89ts05SC6BNRFRKWguU\n1mrzWn0bOn78+Jrl0tJSSktL66tqZnZYKi8vp7y8vMF6imj+iYCk7wA/iYgLJE0GNkTEJEm3AkUR\ncVs6Ef4EycR1N+AloHdEhKQ3gZuAecALwP0RMVfSGOAbETFG0ihgRESMSifC5wMDSc6W5gOD0vmN\n2tsWde1jMgff2H0X+Xi+zMwOdJKIiD2u7jTnTKM+vwBmSboOWElyxxQRsVjSLJI7rXYAY3KO5jcA\njwJHAXMiYm5aPhV4TNIyYAMwKu1ro6Q7ScIigAl1BYaZmeVXXs40DmQ+0zAza7z6zjT8jnAzM8vM\noWFmZpk5NMzMLDOHhpmZZebQMDOzzBwaZmaWmUPDzMwyc2iYmVlmDg0zM8vMoWFmZpk5NMzMLDOH\nhpmZZebQMDOzzBwaZmaWmUPDzMwyc2iYmVlmDg0zM8vMoWFmZpk5NMzMLDOHhpmZZebQMDOzzBwa\nZmaWmUPDzMwyc2iYmVlmDg0zM8vMoWFmZpk5NMzMLDOHhpmZZdbk0JDUXdKrkj6U9L6km9LyIkll\nkpZKelFS25w2YyUtk7RE0jk55QMlLZL0kaT7csqPkDQzbfOGpONz1l2T1l8q6eqm7oeZmWXXnDON\nKuDHEXEScCpwg6SvA7cBL0dEX+BVYCyApP7ApUA/4DzgQUlK+5oCjI6IPkAfScPT8tFAZUT0Bu4D\nJqd9FQG3A6cAQ4A7csPJzMz2jSaHRkSsj4iF6fJWYAnQHbgQmJ5Wmw6MSJcvAGZGRFVErACWAYMl\ndQZaR8S8tN6MnDa5fT0DnJkuDwfKImJzRGwCyoBzm7ovZmaWTV7mNCT1AE4G3gQ6RUQFJMECdEyr\ndQNW5zRbm5Z1A9bklK9Jy3ZrExHVwGZJ7ffSl5mZ7UPNDg1Jx5KcBdycnnFErSq1HzdruDz2ZWZm\njVTYnMaSCkkC47GI+H1aXCGpU0RUpJeePk3L1wLFOc27p2X1lee2WSepBdAmIiolrQVKa7V5rb7t\nHD9+fM1yaWkppaWl9VU1MzsslZeXU15e3mA9RTT9REDSDOCziPhxTtkkksnrSZJuBYoi4rZ0IvwJ\nkonrbsBLQO+ICElvAjcB84AXgPsjYq6kMcA3ImKMpFHAiIgYlU6EzwcGkpwtzQcGpfMbtbcx6trH\nZA6+sfsumvN8mZkdLCQREXtc3WlyaEgaCvwBeJ/k6BvAz4G3gVkkZwgrgUt3HcwljSW5I2oHyeWs\nsrR8EPAocBQwJyJuTsuPBB4DBgAbgFHpJDqSrgXGpePeFREz6tlOh4aZWSPlPTQOFg4NM7PGqy80\n/I5wMzPLzKFhZmaZOTTMzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZ\nOTTMzCwzh4aZmWXm0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWXm\n0DAzs8wcGmZmlplDw8zMMnNomJlZZg4NMzPLzKFhZmaZOTTMzCwzh4aZmWV2UIeGpHMl/VnSR5Ju\n3d/bY2Z2qDtoQ0NSAfB/gOHAScBlkr6en97L89PN3kYo3/djeJwDdwyPc+CO4XH27qANDWAwsCwi\nVkbEDmAmcGF+ui7PTzdA5849kLTHz7Bhw+osl0Tnzj3yNv7B+Ee5v8c5lPblUBvnUNqXg3Wcgzk0\nugGrcx6vScsOKBUVK4Go4+eOesojbdM49YXThAkT8hpOjR0nnwFoZvvfwRwaluOrCqfGjtOUMczs\nwKWI2N/b0CSSvg2Mj4hz08e3ARERk2rVOzh30MxsP4sI1S47mEOjBbAUOAv4BHgbuCwiluzXDTMz\nO4QV7u8NaKqIqJb0I6CM5DLbVAeGmdm+ddCeaZiZ2VfPE+FmZpaZQ8PMzDI7aOc08iV9F/mFfPke\nj7XAcwfr/Ei6P92AtyJia075uRExN4/jDCa5W22epP7AucCfI2JOvsaoY8wZEXH1vuo/HeM0kjeO\nfhARZXnsdwiwJCK2SDoauA0YCCwG/jUiNudpnJuA/4iI1Q1WbvoYRwCjgHUR8bKky4F/BJYAD6Vv\nts3XWCcAI4FioBr4CHgyIrbkawxrnMN6TiP9vKrLSN5NviYt7k7yH2JmRPziK9qO70fEtDz0cxNw\nA8l/3pOBmyPi9+m6BRExsLljpH3dAZxH8qLjJWAI8BpwNvBiRPzvPIzxXO0iYBjwKkBEXNDcMdJx\n3o6Iweny9STP338A5wD/ma+/AUkfAt+KiCpJDwGfA8+Q3P33rYgYmadxNgPbgOXAb4GnI+K/89F3\nzhhPkPzuWwGbgGOBZ0n2RRFxTZ7GuQn4J+APwPnAu+l4FwFjIqI8H+NYI0XEYftD8qqlZR3lR5B8\nRMlXtR2r8tTP+8Cx6XIPYD5JcAC8m8ftfR9oQXLQ2AK0ScuPBhblaYwFwONAKfCd9N9P0uXv5HFf\n3s1Zngd8LV0+Bng/j+Msyd23WusW5nN/SC47nwNMBf4bmAtcA7TO0xiL0n8LgQqgRfpY+fr95/6d\npcutgPJ0+fg8/z23BX4B/BmoBDaQvPD6BdAuX+M0sA3/N499tQEmAo8Bl9da92Bz+z/cL0/tBLoC\ntd+23CVdlzeSFtW3CuiUp2EKIr0kFRErJJUCz0gqScfJl6qIqAY+l7Q80ksFEbFdUr6et38AbgbG\nAT+LiIWStkfE63nqf5cCSUUkB9oWkb4qj4htkqryOM4HOWeU70n6h4iYL6kPkLfLOSSXDHeS3Ipe\nJqklyVmP2fyXAAAClElEQVThZcDdwNfyMEZBeonqGJKDeVuSg+2RQMs89J+rkOSy1JEkZzRExKp0\nv/JlFskZbGlErAeQ1JkkaGeRBHCzSarvTF8kVwbyZRqwDJgNXCfpn0nC4+/At5vb+eEeGv8TeEXS\nMr78HKvjgROBH+V5rE4kn8i7sVa5gP+XpzEqJJ0cEQsBImKrpH8CHgG+macxAL6Q1CoiPgcG7SqU\n1JY8hW164LtX0tPpvxXsm7/XtsA7JL+HkNQlIj6RdCz5DdofAL+S9L+Az4A3JK0m+bv7QR7H2W2b\nI5lfeA54TlKrPI0xleRVeQuSUH9a0l9JDkgz8zQGwL8D8yS9BZwOTAKQ9DWSkMqXHlHrkyTS8Jgk\n6bo8jjMPeJ26/67a5XGcXhHxz+ny7ySNA16VlJ9Luukpy2Er/Yj1wew+ET4vfSWdz3GmAtMi4o91\nrHsyIi7PwxjdSc4C1texbmhE/Km5Y6R9HZm+aqld3gHoEhHv52OcWn1/DxgaET/Pd9/1jNcK6BQR\nH+e53zZAT5IAXBMRFXnuv09EfJTPPusZpytARKyT1A74Lsll1rfzPM5JQD+SGxP+nM++c8YoA14G\npu/6fUjqBFwLnB0R383TOB8AF0XEsjrWrY6I4jyNswQ4KX3htavsWuBnJJevS5rV/+EeGmZ2eEsv\nT95Gchdlx7S4guQM7RcRUfvqQFPHuZhknmxpHetGRMTv8jTOZKAsIl6uVX4u8EBE9G5W/w4NM7O6\n5evOxkNpHIeGmVk9JK2KiOM9zpcO94lwMzvMfUV3Nh4y4zg0zOxw91Xc2XjIjOPQMLPD3fMkdxUt\nrL1CUrnHqdWH5zTMzCwrf8qtmZll5tAwM7PMHBpmZpaZQ8PMzDJzaJiZWWb/H2Xg5U9kJVVdAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10de56780>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sharepa import bucket_to_dataframe\n",
    "\n",
    "all_results = ShareSearch()\n",
    "\n",
    "all_results = all_results.query(\n",
    "    'query_string', # Type of query, will accept a lucene query string\n",
    "    query='*', # This lucene query string will find all documents that don't have tags\n",
    "    analyze_wildcard=True  # This will make elasticsearch pay attention to the asterisk (which matches anything)\n",
    ")\n",
    "\n",
    "all_results.aggs.bucket(\n",
    "    'sources',  # Every aggregation needs a name\n",
    "    'terms',  # There are many kinds of aggregations, terms is a pretty useful one though\n",
    "    field='sources',  # We store the source of a document in its type, so this will aggregate by source\n",
    "    size=0,  # These are just to make sure we get numbers for all the sources, to make it easier to combine graphs\n",
    "    min_doc_count=0\n",
    ")\n",
    "\n",
    "all_results = all_results.execute()\n",
    "\n",
    "all_results_frame = bucket_to_dataframe('# documents by source', all_results.aggregations.sources.buckets)\n",
    "all_results_frame_sorted = all_results_frame.sort(ascending=False,  columns='# documents by source')\n",
    "all_results_frame_sorted[:30].plot(kind='bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can choose different types of plots to generate. Here, we'll make a pie chart of the data sources with the top 10 most results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x10de66c18>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWwAAADtCAYAAACbBE9wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8W9X5x/HPo2Fbku3sRSDsskLYM4xQWqBQChTKLAVa\nOhilLbSlvwJlFroplELLKl0pGwq0zLJHgDADCRAIBAIBMm3LljWf3x9HJk7IkB1JR9d63q/XfTmy\nHflLQr66Ovfcc0RVMcYYU/tCvgMYY4wpjRW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2\nMcYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYE\nhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2McYEhBW2\nMcYEhBW2McYERMR3AGP6Q0QEGAyMBkYCcaCx19G09GNpgoYERGIQiYNEoZCGfApyKch0gWaANNAF\nJItHZ6+PC4APVTVd3f9aYxwrbFNTRCQGrAWMwpXxKAiPgeZ1ILIW6CjIDIfwIGjIw5A0jCxAAtfR\nTQKxUK8jDPEwNIWW7vIwkMX1c6Z4dCukcpDMQ1sOOgrQrq6vOwQWh6EtJhJLQdMCCH8IhXchOQuy\n7wMfAHN7DlVNVfdPzwx0oqq+M5g6JCIjgI3dEZsA8a0hsyGkhsKwFIzMw1iBNRtgzUYYI0s6vOek\nOuYheQGYz5Je7uno2d3wbgbmKHwUgUVNEO2G+GzIvwRtzwLTi8cctX94ph+ssE3FFIct1gY2AzaG\nQVtDeAJ0reuGJNZPwYQIbJH4pLtZj4Hxxk+BecBruI5+OQ3Pd8PrUegKQ+Id0Jdg8VTg1eI3vauq\nBX+ZTa2zwjZlIyJDgO0hshO07gldW7phiU2zsGUMxje4Ut4Ed4YsfgN7sxCYgevoaRl4PgUzotAR\ngZbXoPNeSD8CPKmqi/xmNbXECtv0S/HseV1gV2jdC5gE6REwvgsmxWHnKOwAjPWaM1jagGeAxwpw\nfxJeiEHTXCg8CB0PAI/jzsLtH22dssI2JRORdYB9Yci+kJ4IDY2wax72aoZdgPEMjOGMWpEFXgKe\nAO7vgMcjkOuGhidh0T24Ap+mqnmvMU3VWGGbFRKRMLADNB4ITYdCYRTsW4B94rArbry5Xoc1fFBg\nFq6nH0rB/3JupmHDvdB2E3Cfqi72GtFUlBW2WYqItAJ7waBDIbOPm5HxlRgcGIHtcdPhTO2YDfwH\nuKkdnmqC+CvQdj0UblPVN32nM+VlhW0QkXVB9ochR0LnVrBdNxzeCvsB6/iOZ0rWBTwI3NoNtyrw\nIXT9A7I3Aa/Y2HfwWWHXKREZBqGjoOVk0DVhf+DgGHwOaPEdz6y2PDAFuDED12ch1QG5yZC6WlVn\n+E5n+scKu46ISAQ33HEypD8L++XgOwnYAxvqGMgUeBGYnIWrciBvwuLfAzeqatJzONMHVth1QEQ2\ngti3QL4O64bh5GY4XNxSHKa+ZIG7gT8k4bEwRG+B5GXAMzZkUvussAeo4sXDQ2HwKaAbwNcjcHwU\nNvUdzdSMD4Dr8vDHbkjOh85LIf83VZ3vO5lZPivsAUZENoaWMyFzMHw2Byc0wz5A1Hc0U7MUeBT4\nYxfcGYbG/0Hb74EH7Ky7tlhhDxAisj0MOg8Ku8OpETgpAiN8xzKBsxiYrPCbTlgwF9p/Ctxqa5zU\nBivsACveHv55GHQBNGwGZzbBN0JuqVFjVkcBuAs4IwmzF0PHmcBkVc16DlbXrLADqHgH4iHQej4M\nGwPnNsPh2LCHKT/Fze0+KwkvpyB1HhSusbW+/bDCDhARaQI5BhLnwIbNcF4z7Ivt9GaqYwrwsyQ8\nkYfsLyB7uaq2+05VT6ywA8DNnw59A5ougp2icE5xsSVjfHgZOLcL/qvApdD9S1Vt852qHlhh17Di\nGPV+0PxH2GwYXJaAbX3HMqboTeCcFNyWgfSPIH+trRxYWVbYNUpEtoHWK2DIpvDHhBv6sJXxTC16\nDvhWEma+Dx3Hq+rjvhMNVFbYNUZERkHL7yB0EPyyCb4htsa0qX0KXA98twuy90H7Kar6nu9UA41d\nraoRIhIVaTgVYm/BcQfD7Bh828raBIQARwCz43DyfhB7XSR2roj42Cl5wLIz7BogIpOg+TrYYjhc\nlXB7HhoTZO/gzrYf6oTOE4Fb7K7J1WeF7ZE7+0j8DhqOgatjcBA2Tm0GloeA4zth/jRoP0JV3/Gd\nKMissD0RkW2h+RbYcwRcE4NhviMZUyE54Dc5OD8N6e9D/ho72+4fK+wqE5EoxM6GyKnw5yY4wk6p\nTZ14BfhKJ3zwDLQfpapzfScKGivsKiqupHcLbLUOTI7DWN+RjKmyDHBeFi7uhq6vq+rNvhMFiRV2\nFYhICBpOgciF8OtGOCFkY9Wmvj0DHNQF7TdD8gRV7fKdKAissCtMRNaE1ptg3c3hpgRs6DuSMTWi\nHfhmCv77MSQPUNWXfCeqdTYPu4JEZEeIvwynbgtTrayNWUorcEMMLh8HiadEGk7wnajWlXSGXZz8\nPk5VX698pIFBJHIUNF0F/4q5HcmNMSs2E/hcFyy4DjpPsTVJlm+VZ9gisj9uy+V7io+3FJE7Kh0s\nqEQkJJL4FQy/Ep6ysjamJBsCL8Zhi2Oh5V4RafadqBaVMiRyDrA9bu8gVPVFYN0KZgosEUlAy39g\n4xPh5Ths7juSMQEyBHg4DgdNhObn3PUf01sphZ1dzlq3dqVyGSKyFjQ/D/vvDk8mYKTvSMYEUBS4\nrgnOWA/iL4rI1r4T1ZJSCvtVETkSCIvIhiLyB+DJCucKFBHZAeIvwZnrwT9i0Og7kjEBJsBPIvC3\noZB4VEQO8J2oVpRS2N8FNgPSwGSgDfh+JUMFiUj4QEg8CNcPgdMjNr/amHI5WOChBAz5l0jTj4ob\netQ1m4e9GkTkS9ByPTwcA3vnZkxlvAvs2Qlzr4bOH9TzOiSlzBK5X0QG93o8RETurWys2ici+0Hz\n9fCglbUxFTUOeDoBaxwPsQt8p/GplCGR4aq6uOeBqi6izq+oicje0HwjPBCzPRaNqYahwOMJGPl9\nkdgZvtP4UkphF0RkXM8DEVmbOp4lIiJ7QuJWuDcOO/iOY0wdGQk8EYehPxVp/J7vND6Usv/UGcDj\nIvII7orarsC3KpqqRrmdYRJ3wN1x2Nl3HGPq0FhcaW9/oUi0UzV7te9E1bTSi47Fq7JrAilgx+Kn\np6jq/CpkqykisivE74G74rCH7zjG1LmZwA4paP+mau6fvtNUyypniYjINFWt61v2RGQ7iD8E/07A\n53zHMcYA8CowMQVtX1XVW32nqYZSxrCfd4VVn0RkrDuznmxlbUxN2Qx4KAYtfxeR3X2nqYZSzrBf\nAzYAZgOduHFsVdUJlY/nl1ulsOVZOH0jOKOU8X5jTNXdBxy0CLo2G+jbjpVS2Gsv7/OqOrsiiWqE\nG79vuQk+vy/cHLM7GI2pZWdn4eJp0LGjqmZ9p6mUUoZEdAXHANf4Q1hrH/i7lbUxNe/sKGyzMSR+\n4ztJJZV00RFX0AI04ZZWfV1VN6t8PD9EZCK03A/TYrDcNxjGmJqzANi0Cz7+qqre5jtNJfR5LZHi\ncocnqurxlYnkl4iMgPgMuGEYfNF3HGNMnzwLTEpC19aqOtN3mnLr856Oqvo8A/QWPxEJQ8utcEKL\nlbUxQbQd8Os4NN8tInHfacqtlCGRU3s9DOFWOhqmqntXMpgPIg0/gM3PdwvN2KQQY4JJgcNScM+d\nqu2H+U5TTqUU9tm9HuaAd4BbVLW7grmqzs23jr0Bz8dhY99xjDGrpRPYpAveO0pVb/edplxKHsPu\n2RRTVZMVTeSJyKC74KS94MKo7yzGmHJ4BNhvAXSuq6odvtOUQynrYY8XkRdw94G+KiLPicj4yker\nHhHZBxJ7wFlW1sYMGLsDB8Yh8SvfScqllCGRJ4EzVPWh4uNJwIWqOiCWq3N3MyZmwc2jYR/fcYwx\nZbUAWC8F7bup6lTfaVZXKbNEEj1lDaCqDwOJiiWquvjZsGerlbUxA9Ew4PdN0HrtQNgTspTCniUi\nZ4nIOsXjTGBWpYNVg4hsDHIKXD7gpv8YY3ocIzB2PeAI30lWVymF/XVgBHBr8Rhe/FyguVfb1r/C\nBY1uUXRjzMAUAq5MQOLSoM/N7tOdju7GEhKq2l65SNXh9mVc92Z4o9nmXBtTDw7sgnt/q5r6me8k\n/VXKLJHJItIqIglgGjBdRH5U+WiVNuhcOMfK2pi68bs4yKk9U5SDqJQhkU2LZ9QHAnfjFn86uqKp\nKkxEtoLw5nC47yjGmKpZD7e9X+hYz0H6rZTCjopIFFfYdxTXmg348qqtZ8HpTdDgO4gxpqp+koDE\n/4lIn9dRqgWlhP4z7nb0BPBocUODwI5hi8g4yH0Bvh3IvzBjzOrYBRjbCnzBd5L+6M/yqgKEVTVX\nmUiVJdJ8GXzzm3CxnV4bU5f+AXz3adVFO/pO0ld9LuwgE5HB0PQBvBGDtXzHMcZ4kQFGpWDxDqo6\nzXeavqizYYHoCfAltbI2pp41AD9ogJbTfSfpq7o5w3ZDOc0fwsMjYRvfcYwxXs0DxnVD99qq+rHv\nNKUqZR72cyJykogMqUagCtoM4gm3/4Ixpr6NAL5UAAK1wUEpQyKHAWsAz4rI9SKydzAXUYkcCAdH\nbAd0Y4xzSByGHuo7RV/0ZQODEG6jwyuAPPAX4BJVXVi5eOUjMnQ63LgJfM53FGNMTVgMjEpDZoiq\npnynKUVJFx1FZALwW+DXwC3AV3BzsR+sXLTyEZHR0L0e7OY7ijGmZgwGxqeBSZ6DlGyVC2mIyHO4\nl6JrgJ+oarr4padFZGIlw5XRfrBXFhoafQcxxtSSQ5rh9YNwy27UvFJ2nFlPVQO9/rXIsAfh0j3g\nKN9RjDE15WVgl4+gY4wGYMpcKUMibSJyqYg8X5wxcomIDKt4sjJx698mdw7onajGmIraHIi2ABv6\nTlKKUgr7etykxYOBQ4q/vqGSocBtjCsir4nIGyKyOhPcJ8HmaRharmjGmAFDgC8KyL6+k5SilMIe\no6rnq+rbxeMCYFQlQxVnpFwG7A1sBhzhtvPqj/A2sEegd5kwxlTS/jEY8mXfKUpRSmHfJyKHi0io\neBwK3FvhXNsDM1V1dnE51+uBA/r3VIMmwla2S4ExZgXGA7nP+E5RihUWmYh04Na9FuD7wN+LXwoD\nSeCHFcw1Fniv1+M5uBLvh9wE2KIMkYwxA9O6QOdwEYkWTxBr1goLW1VbqhmkEtwFx+gICMSLpzHG\ni0ZgaDfMWweY6TnMStXqan3vA+N6PV6z+Lm+Wh/GdEG0PKmMMQPUBjlgA98pVqVWC/tZYAMRWVtE\nGnCbL97Rj+dZH9av+bmVxhjfNmskAFP7avJinKrmReRk4D7ci8o1qjqjH0+1HmwSK286Y8zAs2kT\nNG/mO8WqlHJr+vrAHFVNi8gkYALwN1VdXMlgqnoPsNHqPUvLprCRbQVmjFmFDYHGzX2nWJVShkRu\nAfIisgFwJW67lskVTVU20TFu3VtjjFmZDYDMer5TrEophV0obrh7EPAHVf0RMKayscpFonbB0Riz\naiOBbL9nxonIZ0QkLyLa67i4jAGB0go7KyJHAMcAdxU/F5AWlGiNDtMbY2pKA5BfnbJYA9enzwF3\nAgXcbr9lVUphHwfsBPxcVd8WkXVZchNNrYtYYRtjVq0ByIdX4wl2KX78LW4cNo1bcLusSmmzz6vq\nKT0PiqXdXe4gFWKFbZZjLdzNs7XG1/Z1Qd82r1z5C2ERkX4us9qzeFTP9T0Fyn7XZCln2Mcs53PH\nljlHhagVtlmOWixrcP/GfRyFgB/5Mh2AW3qjT0RkP+DD4h/mHJYs6fGVvj7XqqxsLZEjgCOBdUWk\n900rLUAg9nHEzrDNcvWcQOVw92g9BjwN8mqB8Byl0Bmi4E7bosVDWFIN2V7PAEA47I5QyB2996hW\nhUIB8nl3qEIkAtEoNDQoDQ3Q1KQ0NSmxGMRiSiIhxOOQSISIx4WmJj45YjH3sbFx6V83NrrnFHE/\nr+fo+Zn5/NKfX97Xe39c0dd7/97lfX5539P7MSz9e3p/zOdX/vUVfew5Svk6rPjrU6dS/Gvuq4nA\n7sVfJ4rPIUBrP55rpVbWZk8Cc4HhuHGZHh24bRoCQCP9eME0dSOCuzyzk3uohMj1fC0HvECWx8jy\nIsj0Ag2zCtAeJpN3jdyA0ghE80IoDyGIFCg0pilE0ghZJF9AMiA5oNkdGs9mNZbNkujqoglCUZAI\n7u1uz1texQ2CdohoRzhcSIbDpEIhTYdCZN3zSV4VVZVCoSCfvCCIuOLueUFoalIaG5e8GMTjEI9L\n8UUhRDzOUi8Iyx49Lwo9LwwS9OGTFchk4AtfyGs+3+fCVtWfisjdwD24wXBwf5WvlTMirHzxp9nA\nbD75vzmo7M500x8RYLvigSvzdE+fFoDnIf2EkH7elXn07QLSFsplc6FchBCDKDCcAqMIMwxohnaB\n9iRCG0I77tQnCU1J8tFuNJRBNE8oq0ga9y+/VZWhuZyMy+V0NDAWQmMgNAJ3JtX7YwvQrsq8dJqF\n6TQLQBaCLAIWAW3Q+8fSCbSHQoWOSES7QiHtDoUkI0IWyLsXBNFCQbT3O4Te7w7cmf3S7w7cOwP3\nYpBIyCpfBHofUY+Tz1IpCIfTq/7GFZqD+yvr6dQ8cOJq51pGKXc6fhn4JW6iohQPVdWyn+5XwBx4\nf7zvEGagCQHbFg9cmWd6lXnmZZj3WIh5z4d47VUlOiuPtAnZXIgI0EqB4SijCbEx0j2UcPcwoPci\nCgXoboPuhcjHi5DXetq2EyJJCo2dFCKZJWfxaZAC7ix+MOhwKIwCXQNkrCt42ZilS344EC0UQmRK\nm32WARbkcszP5ViQSskCYFHxBWFxMV7PC0Jn8WNHJJLvLL476BYhK+LeHQCFQmHpdwfQ82LQ84Lg\nXhTcC0LPuwM3VOSGiyh+/dMvArHY0l9rbHTDVSuSTEI02lnSH8RyqOrb9JruLCJn4U52n+rvcy5P\nKQO8vwL27+daHp4lX4VZexP8y+AmMELAlsUDUIRMz7hcATKvwPzHQ8yfCq9PV6KzcsiiENlciDAU\nz8yVkYQZjtvZblOg155JOQjlljdhIAVtC6FtITJ7MWHXmCBJaOwkF01BKIsU8oRyinTjFhYdBDoM\ndAToGNCxEB4NsuwZ/HBgNH28ay6XC5PLrfLbFHfWPy+TYUEmw/xe7w7aWPoFode7A02Gw4WucFhT\noRAZEcm6Px8pFIeKlvvuoPdwkTvcGLb77X0mImsC/8L9EeWA64DPA7/oz/Ot9GeVsGv6E6o6sdw/\nuBpE5DtwzO/gOlsAytS4AjAddwH0eeBVpeHNPLI4RC4bIkTPmXnPMIswFFfocfp3SpLHNeHC4sde\nZ/HRDgqNXRTCGYQckiuOxRdwQy9DoDActOcsfg0IjVxOyQ+jNi7754AFuA1pFxZ/3XuoaCrwJExZ\nrNrnIWARGY2bh/1T3H/uZ4ArVPUH5Unf62eVUNiX4F5Yb8ddBwFAVW8td5hyE5G9Yccb4KlBvrMY\n038F4HVcmU8FpisNM/PIIlfmbj7Cp8t8GP0v8xXpwjVeT8kXT3kliTYlyUfTIFlChTzSMxYfw53F\nDwUdCYXRIGtCaBRI73Lv+XWizJFLcQlwJlzdofrN1X0uEbkdt4zH/1Y/2dJKefFrxf017dXrcwrU\nfGED78A7Nk3EBFwI2KR4fAtAyPT82y0AM2HhYyEWPhfijVeUhjdzyIKly3xYrzIfhjsz708zxovH\nmkt/WkFSEEkt+/056FoMXQuQuYuRV9sJ9ZzFN3SQb0ih4TRCHskVCPWcEfY+ix+9ZCw+vLyLrUNZ\n/blgr0MmCas97Csi6+DGw55e3eda7vP376aeYBCRJogkoTts0/tMfXoLeAR3Zv6qEp2ZJ7QgRD4T\nQllS5qOXGWZpxt+VnyTuDL731cwkhJJoYyeFaDcqWUKFApJRJIN7DSmOxRdG9LrYuuxZfM/H+DI/\ncldofxy+qqp39je2iDQDDwPnq+q/+/s8K/0ZJQyJfAa4AhilquNFZALwJVW9oBKByk2keRHMGOxu\nRzbGLPE28CiuzKcp0TeWLvOWTy6Ahhjeq8xbqK3L+Dk+PUxTnLv4yVm8mzIp+QKhbiheEoDBxYJ/\nASQNG6rqrP5EEJEIbnG8u1X1krL8dy3v55RQ2I8APwL+rKpbFT/3iqoGYrqcyLBpcNt42M13FGMC\nZDZLzsxfKZb5/BD59JIyH4oyCmE4oU+GWZqp3Y0HexRw00wW4Ar+Y2AKWQo0qWp/7nRERP4GzFfV\nU8sX9NNKGcOOq+ozsvQdTquep1MzMlNgymawWy2dExhT49YGvlY8ELK9u+JdWPxYiMXPwqxXlOjr\nOULzXJm7aSRaHDN3Zd77zLwWyjyEy9Kz+vVbwAvM0K6+l7WIXIPbK2AI8JKIvIC7xvfT4q5ZZVVK\nYc8vbhOmxYCH4G5ZD4jknXD7ofDjINzoY0wAjAOOKh7LlvkH0PaI0PZs2JX5a3lC84R8tyvzZpRh\n5BlFaKkyb8Vfmb9HgSwP9vN3/wX4A27bxC3LmGq5ShkSWQ+3NdjOuMsAb+MG59+pdLhyEJFWaJgH\nixuWvpXMGFNdH+LGzJ8GXoXIa3nCH0EhHSavrsyHkmckIUb0KvNBVLbMr6adORytqnes+ps/TUTW\nBu5U1QllTvbpn1XqLBERSQAhVe2obKTyExn6Elw/YemZicaY2vExrsyfAV6ByIximXe7Mk+gDO01\nzNIzZr66ZZ4DLiJDnlH93Vi8moVdyloig3EDWesAkZ6x7N6bGtS+5K1w9yawV0C2NjOm3owEDike\nQI7wkitl86HjMaFjSpjZPWX+4dJlPqRY5suema9qNu8cIMo7mutfWVdbKWPY/wWmANPo31qxNSB7\nD/z7NLjYCtuYwBmOu653kHu4VJkvLJb502HenQaR6TnCHwqFlCvzePHMfCQwgvAnZT4YV+avkSXL\nTVX/T+qnUsawn1fVrauUpyJEJAyN7fB2PDAbvhtjVtNi3O38U3DDLNPzhD+AQipEToU4Sholx/aq\n+lx/f0rx7sY7VXXz8uReyc8qobB/gLv36C6WXkskKLvOACAy9B64ZG842ncUY4x37cA1wBkLITW8\nn/s4IiKTgUm4lVs+As5W1b+ULeYyShkSyQC/Bs5gyW4ACqxXqVCVsehG+OdEOLrZdxJjjG+twPsZ\n0Gv6W9YAqnpkGUOtUiln2LOA7VV1fnUiVYaItEDTRzArZsMixtQ7BUZ2wfwdVXWa7zSlKmVCzJu4\n1foCzU1HjN4M1+RX/d3GmIHtKSCzAHjFd5K+KOUM+zZgM+Ahlh7DDtC0PkdEtoWRD8PcRG3cI2uM\n8eNLnXD3OarZ3/hO0helFPYxy/u8qv61IokqSEQEBr0Gkz8D+/qOY4zx4g1gyySkxqhq0neavhjQ\n62Evj4gcDTtdDk/axUdj6tLXUnDj71S7z/SdpK9KOcN+myWzQz6hqgGbJeKISAPEP4QnhnyyUaox\npk7MATZMQfdaqrrAd5q+KmVa37a9ft0EfAV3r1AgqWpGpOE3cNEZcMOyG08YYwa0X2cgfG0Qyxr6\nOSQiIs+p6jYVyFMVIjIEmubAy3HY0HccY0xVLADGdUPXhqo6x3ea/ihl8afet6WHcGfctbBzfb+p\n6iKRxp/DST+F+xK+8xhjquHSHIRvCWpZQ2lj2A/1epjDrYf9W1V9vZLBKk1EGiHxNtwyBvb2HccY\nU1EdwNgUdGwV5O6qu1kivYnIl2DtyTAzAbaQnzED1ylp+Ou/VdsO851kdazy7hERubC4JnbP4yEi\nEogd00twJyx6ES4L6LKxxphVmwpck4b2k3wnWV2lDIm80LNbeq/PBX7J1R4isgk0P+fWGBnhO44x\npqyywPhOmPkd1cI/fKdZXaXcnx12472OiMSAxpV8f6Co6gzgOji923cWY0y5/S4PH74E+k/fScqh\nlDPs04H9cbsDAxwH3KGqv6pwtqpxQz7x2fB4K2y16t9gjAmAWcDmKegar6qzfKcph5IuOorIPsDn\nig/vV9V7K5rKA5HI8bDh7+GFhLs/yBgTXApM6oQpF6imf+E7TbmUWtijgO1xfwrPqOrHlQ5WbW5h\nqJa74PA94coBM+RjTH36u8JJb0LHpqqaW/X3B0Mps0QOxe09fwhwKPC0iBxS6WDV5nad6DgSJi+C\nG+p3rqMxgTcXOLkbOo4aSGUNpY1hvwR8vuesWkRGAA+o6hZVyFd1IrINJB6DF2Owge84xpg+SQM7\ndsLrv1XtOtt3mnIrZZZIaJkhkAUl/r5AcrsnZ06HL3aCTRwxJjgU+FY3vPUopM71naYSSinee0Tk\nXhE5VkSOBf4D/LeysXzLXgYfPATftcY2JjD+WIBb50LHoao6IG+GK/Wi48HAxOLDx1T1toqmqgEi\nMggSM+Dq0XC4+M5jjFmZR4B9O6BrK1V9y3eaSqnrtURWxa1UmHgcno3BJr7jGGOWazawRQraDlDV\n+32nqaQVDomISIeItK/oqGZIX1T1eeg+AXbrcv9TGGNqSxewTyd0nzPQyxpWsq61qrYAiMj5uHky\nfwcEOAoYU5V0NUA191eRpmEw8XyYGofRviMZYwB3kfHoFMy5G9K/9p2mGkqa1rfsFL7lfW6gE4mf\nD2NPhWfiMMR3HGPqnALfS8N1M6Fje1VN+U5UDaXMEukUkaNEJCwiIRE5CuisdLDak/oZfHAt7NEJ\nSd9hjKljCvwkA3+ZDR271UtZQ2mFfSTuDsePisdXip+rK+5OyK5T4K3bYZ8um6NtjC/nZuHy9yE5\nUVUX+U5TTTZLpI9EJAzNt8Oun4U74gHf3tKYgLkwBxfNheR2qvqR7zTVNmDvWKwUVc1D8mB4fCp8\nNQV535GMqQMKnJWFC+dCcqd6LGuwM+x+E5E4tDwAu20BN8dtSVZjKkWBH2bgyveKwyB1WdZgZ9j9\npqpd0LGWeeLFAAAILUlEQVQHPHo/7NYJi31HMmYAKgAnpeGqtyC5Qz2XNZS2vOqZvX5t60T3oqpp\n6PgyTP8rbNsJc3xHMmYASQIHpOAfM6BjJ1Vd4DuRbyu70/F0EdkJtw52j6cqHylY3CIznSfD++fD\nVl3wou9IxgwA7wBbd8LDt0PHjqra5jtRLVjZGfZruCl864nIYyJyFTBMRDaqTrTgUFVVTf0SFhwL\nu3TBHb4jGRNgjwBbpuDdn0HyKPdO1sBKLjqKyO7A08CTwHa41Y/+AzwIbKSqO1crZJCIyHYQvwd+\n1go/jri7+Y0xpbm8AD/qhK4vq+oDvtPUmpUV9oXADsC2wHXAy8Bpqrpp1dIFlIisBc0PwP7j4Kom\nSPiOZEyNywAndsMNH0Ly86r6pu9EtajULcK+AWwN/Bx4HVikqvtXPl5wiUgztFwDg78It8VhG9+R\njKlR83A7PM2YAh1fVtW6WA20P0qZ1nevqk5V1SuBOaq6C3BchXMFnqomVdsPgznfgN2S8Mu8m6Jk\njFliCrB5F7xyOXTsZWW9cn26cUZEtlDVlyqYZ0ASkXWg5TaYsCHcmIA1fEcyxrM0cEYGruiGruNU\n9VbfiYLA7nSsEhGJQNPZ0HAa/C0GB/iOZIwnzwGHdsK8x6Dj2Hq/GaYvrLCrTEQmQuJWOLwVLm2C\nuO9IxlRJBrfS3sVp6P426L/UCqhPrLA9cBv8tvwFhu3lhki28x3JmAp7CXdWPfcZ6Piqqn7gO1EQ\n2VoiHqhqm2r7l2H2t2H3djiuG+b7jmVMBeSA83KwUye8dQp07Gll3X92hu2ZiAyG5l+AfA0uaoTv\nhCDsO5YxZfAscFwnvPcCtB+pqu/5ThR0Vtg1QkQmQOu1MHpjuDYBE31HMqafZgOndcHdGUidBvoX\nG6suDyvsGiIiAhwO8ctg/xhcHKujDepN4LUBP8/AZXnQi6H7IlW1DVDLyAq7Brm7JOPngXwHzm6A\n74ch6juWMSuQA64swP+lQe+AjtNU9X3fqQYiK+wa5lZGbL0aBm0Fv0jAYdj4tqkdilsP7qROWPwK\ntH9HVW194Qqywq5xxWGSvWDQL6F5A7ggAUdhZ9zGrxeAk5IwbSEkTwT+a+PUlWeFHRDF4p7kirtp\nUzg3DscK2CZAppqeAs7uhMdzkP4/KFylqjnfqeqFFXYAubslB/0ColvDz5rg+BDEfMcyA5YC9wFn\nJWF6F6TOg8K1qprynazeWGEHmIhsD4MuAtkJzizO4ba1t025ZIGbgPOT8P4C6DgTuEFVs56D1S0r\n7AFARLaEQReCToITInBiFMb5jmUCayHwpzz8Ng356dB2PnCX27/U+GSFPYCIyMaQ+B7kvwa7FOC0\nZtgLW4HAlOZV4JI0/EMheie0X6SqL/hOZZawwh6ARCQBHA6DfgyxsfDdJjgubDfhmE+bB0xW+FMS\n3stB4UpIXWrrfdQmK+wBrDizZDtoORmyh8DEPJzUDPsBDb7jGW/SuPnTf0rCoxFouhvargAeVNW8\n53BmJayw60TxrPtgGPI9yG8Cx4ThyAbYHhsyqQeKW4zp2jT8UyE6HRb9EbjZtuUKDivsOiQiG0Dj\ncdB4NISGw8EChzbBJOzMe6B5B/hXAa7ogkVJyFwJmetU9W3fyUzfWWHXORH5DIQPgtavQXp92DcH\nhyVgH6DZdzzTZ1ngCeCOLNzS7caoI7dDx5+BJ+1uxGCzwjafEJE1gANgyNHQtQ3skoajWmB/YLjv\neGaFPgTuAW5Jwv+i0Dgbum6CzB3AVJuON3BYYZvlchsrsB8M+Sp0TYKN0rB3AvaIwM7AIM8J61kB\nmArcmYebO+HtKMQfgUU3APeo6oeeA5oKscI2qyQiMWBniE6Cli9Ax+awTjfs1QSfbYBdgRGeUw5k\nnbidxp9SeCgJT0ZB5kH2VkjdDjxhdx/WByts02ci0ghsA6HdYfB+0Lk1jMzB56KwZxPsBqzlO2ZA\nKTATmAI81g2PZOCdGDS/BZmHoPNR4ClVne03p/HBCtusNhEJAxOA3WDIvtC9EzSGYZMsbBeDLRpg\nPLApdiGzN8WNP78MTCnAg0l4rhGkA6JPw6IHcM39oqp2e41qaoIVtim74g07awLjQcbD4B2ALaFj\nHAxNw/gCbJeACWFX5BsxsJeJzeH2OXwDmA682AUvZeHNJiALsZnQ+SCkHweeVtW5PtOa2mWFbaqm\neCa+PjAewpvDoB0hPx46R8OoFIwrwPpR2CAGa4tbwGotYA1qdxVCBdqBj3sds4EZ3fBqGmaFYF4c\nmhZB49uQeg66XgRmADNUdZ6/7CZorLCNd8Ux8Q1w7TwOIutAy8YQWhcyYyE1BMIKQ7thdAHGhmBc\nAwxvdEUex31c9tfLPo4Bedxc5cxKPvb+dRduLvPHwNwsfJCGD/LwEbAgCu2NEM5B02KILAD5GNJv\nQHI68GbxeEdV0xX+YzR1wArb1LziEEsrMLp4jHGHtELjIGgYBOFWCLWANIMmoJAAjUGuCfJNkGuE\nbBSkAOE8hPKuaCUPoVzxyALZJR+l2NiFuZB6D7o/wDV3T4N/DMyz8WVTLVbYpm6IiNidfibIrLCN\nMSYgbJk2Y4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJ\nCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCts\nY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJCCtsY4wJiP8H4ii/\nJTZHh4wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d806b00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "all_results_frame_sorted[:10].plot(kind='pie', y=\"# documents by source\", legend=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}