DSC-540_Final_Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DSC540-T301 Data Preparation (2203-1)\n",
    "## Final Project\n",
    "### Ambrose M Malagon Cordero\n",
    "________________________"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Instructions: During the course, you will be working on a term project to either pull data from an API or scrape a webpage. You will need to select either an API (different than Twitter) or a Webpage and create a process in Python that will extract data into a formatted dataset.* "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this term project, I've chosen to use the Marvel Comics API (https://developer.marvel.com) which allows access to information about Marvel's library of comics. In order to access the API, you will need to register in order to obtain both a public and private key. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# RUN THIS CELL - Otherwise the program won't work.\n",
    "public_key = '385cebbcb01e23edbff1c01d0be2dca3'\n",
    "private_key = '4c672a897525001a358fe052fc0ff08e036950bd'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How the API works\n",
    "Per the documentation provided by Marvel (https://developer.marvel.com/documentation/authorization), under **Authentication for Server-Side Applications**:\n",
    "\n",
    "*Server-side applications must pass two parameters in addition to the apikey parameter:\n",
    "ts - a timestamp (or other long string which can change on a request-by-request basis)\n",
    "hash - a md5 digest of the ts parameter, your private key and your public key (e.g. md5(ts+privateKey+publicKey)\n",
    "For example, a user with a public key of \"1234\" and a private key of \"abcd\" could construct a valid call as follows: http://gateway.marvel.com/v1/public/comics?ts=1&apikey=1234&hash=ffd275c5130566a2916217b101f26150 (the hash value is the md5 digest of 1abcd1234)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_________________\n",
    "#### Part I -A formatted dataset with at least 15-20 variables.\n",
    "First we start by importing all of the necessary modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hashlib  # Required in order to create the hash value.\n",
    "import time     # To generate the the time stamp. \n",
    "import requests # To make the call to the API.\n",
    "import pandas as pd # To store the retrieved data into a data frame.\n",
    "import re # Use regex to extract certain elements from the data. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "________________\n",
    "Starting to work with the API, I noticed that some queries would not work as intended. For example: When attempting to retrieve data using the query to get characters by name as established in the documentation, with the return response being that the hash/public ID, or another sequence of values provided was incorrect. This challenge does not show up when using character/event/comic id's. To test out the API, I am going to get specific character information by using the query as explained in the documentation provided by the Marvel Developer website. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "char_id = ['1009220', '1009664', '1009368', '1009189', '1009351', '1009338', '1009610', '1009562', '1009697', '1010338', '1009477', '1017111', '1009718']\n",
    "response_ = []\n",
    "for req_id in char_id:\n",
    "    # Create the hash.\n",
    "    time_stamp = str(time.time()) # Creating the timestamp and converting it to a string.\n",
    "    time_stamp_byte = bytes(time_stamp, 'utf-8') # Now converting the time stamp into a string. \n",
    "    hash_ = hashlib.md5(time_stamp_byte) # Hashing the timestamp using the md5 format per Marvel's instructions.\n",
    "    hash_.update(b\"4c672a897525001a358fe052fc0ff08e036950bd\") # Next, adding the Private Key as a bytes.\n",
    "    hash_.update(b\"385cebbcb01e23edbff1c01d0be2dca3\") # Adding the public key to the hash as bytes.\n",
    "    hash_t = hash_.hexdigest() # Finally, we translate the hash into a hexadecimal string and save it as the hash_t variable.\n",
    "    hash_t\n",
    "    # Next we declare the various pieces to construct the API url:\n",
    "    base_url = \"https://gateway.marvel.com\"\n",
    "    query = \"/v1/public/characters/\" + req_id + \"?\"\n",
    "    query_url = base_url + query +\"ts=\" + time_stamp + \"&apikey=\" + public_key + \"&hash=\" + hash_t\n",
    "    character_data = requests.get(query_url)\n",
    "    response_.append(character_data.json()) #Append the resulting JSON data into a list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'code': 200,\n",
       " 'status': 'Ok',\n",
       " 'copyright': '© 2020 MARVEL',\n",
       " 'attributionText': 'Data provided by Marvel. © 2020 MARVEL',\n",
       " 'attributionHTML': '<a href=\"http://marvel.com\">Data provided by Marvel. © 2020 MARVEL</a>',\n",
       " 'etag': '011e23e9a61d757b757d24075050b75cc547c8ac',\n",
       " 'data': {'offset': 0,\n",
       "  'limit': 20,\n",
       "  'total': 1,\n",
       "  'count': 1,\n",
       "  'results': [{'id': 1009220,\n",
       "    'name': 'Captain America',\n",
       "    'description': \"Vowing to serve his country any way he could, young Steve Rogers took the super soldier serum to become America's one-man army. Fighting for the red, white and blue for over 60 years, Captain America is the living, breathing symbol of freedom and liberty.\",\n",
       "    'modified': '2016-09-06T11:37:19-0400',\n",
       "    'thumbnail': {'path': 'http://i.annihil.us/u/prod/marvel/i/mg/3/50/537ba56d31087',\n",
       "     'extension': 'jpg'},\n",
       "    'resourceURI': 'http://gateway.marvel.com/v1/public/characters/1009220',\n",
       "    'comics': {'available': 2350,\n",
       "     'collectionURI': 'http://gateway.marvel.com/v1/public/characters/1009220/comics',\n",
       "     'items': [{'resourceURI': 'http://gateway.marvel.com/v1/public/comics/43488',\n",
       "       'name': 'A+X (2012) #1'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/43501',\n",
       "       'name': 'A+X (2012) #4'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/43508',\n",
       "       'name': 'A+X (2012) #9'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17743',\n",
       "       'name': 'A-Next (1998) #2'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17744',\n",
       "       'name': 'A-Next (1998) #3'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17745',\n",
       "       'name': 'A-Next (1998) #4'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17746',\n",
       "       'name': 'A-Next (1998) #5'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17747',\n",
       "       'name': 'A-Next (1998) #6'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17748',\n",
       "       'name': 'A-Next (1998) #7'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17749',\n",
       "       'name': 'A-Next (1998) #8'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17750',\n",
       "       'name': 'A-Next (1998) #9'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17740',\n",
       "       'name': 'A-Next (1998) #10'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17741',\n",
       "       'name': 'A-Next (1998) #11'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/17742',\n",
       "       'name': 'A-Next (1998) #12'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/66978',\n",
       "       'name': 'Adventures of Captain America (1991) #1'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/66979',\n",
       "       'name': 'Adventures of Captain America (1991) #2'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/66980',\n",
       "       'name': 'Adventures of Captain America (1991) #3'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/66981',\n",
       "       'name': 'Adventures of Captain America (1991) #4'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/42539',\n",
       "       'name': 'Age of Apocalypse (2011) #2 (Avengers Art Appreciation Variant)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/comics/30090',\n",
       "       'name': 'Age of Heroes (2010) #1'}],\n",
       "     'returned': 20},\n",
       "    'series': {'available': 643,\n",
       "     'collectionURI': 'http://gateway.marvel.com/v1/public/characters/1009220/series',\n",
       "     'items': [{'resourceURI': 'http://gateway.marvel.com/v1/public/series/16450',\n",
       "       'name': 'A+X (2012 - 2014)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/3620',\n",
       "       'name': 'A-Next (1998 - 1999)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/24227',\n",
       "       'name': 'Adventures of Captain America (1991 - 1992)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/15331',\n",
       "       'name': 'Age of Apocalypse (2011 - 2013)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/9790',\n",
       "       'name': 'Age of Heroes (2010)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/10235',\n",
       "       'name': 'AGE OF HEROES TPB (2011)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/13896',\n",
       "       'name': 'Age of X: Universe (2011)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/7534',\n",
       "       'name': 'All Winners Comics 70th Anniversary Special (2009)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/20682',\n",
       "       'name': 'All-New Wolverine (2015 - 2018)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/2114',\n",
       "       'name': 'All-Winners Comics (1941 - 1947)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/9865',\n",
       "       'name': 'All-Winners Squad: Band of Heroes (2011)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/2116',\n",
       "       'name': 'Alpha Flight (1983 - 1994)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/454',\n",
       "       'name': 'Amazing Spider-Man (1999 - 2013)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/15540',\n",
       "       'name': 'Amazing Spider-Man Annual (2012)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/2984',\n",
       "       'name': 'Amazing Spider-Man Annual (1964 - 2018)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/9802',\n",
       "       'name': 'Amazing Spider-Man Annual (2010)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/1489',\n",
       "       'name': 'AMAZING SPIDER-MAN VOL. 10: NEW AVENGERS TPB (2005)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/14818',\n",
       "       'name': 'Annihilators: Earthfall (2011)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/24323',\n",
       "       'name': 'Ant-Man and the Wasp Adventures (2018)'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/series/14696',\n",
       "       'name': 'Art of Marvel Movies: The Art of Captain America - The First Avenger (2011 - Present)'}],\n",
       "     'returned': 20},\n",
       "    'stories': {'available': 3491,\n",
       "     'collectionURI': 'http://gateway.marvel.com/v1/public/characters/1009220/stories',\n",
       "     'items': [{'resourceURI': 'http://gateway.marvel.com/v1/public/stories/670',\n",
       "       'name': 'X-MEN (2004) #186',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/892',\n",
       "       'name': 'THOR (1998) #81',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/960',\n",
       "       'name': '3 of ?',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1026',\n",
       "       'name': 'Avengers (1998) #81',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1041',\n",
       "       'name': 'Avengers (1998) #502',\n",
       "       'type': 'interiorStory'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1042',\n",
       "       'name': 'Avengers (1998) #503',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1054',\n",
       "       'name': 'Avengers (1998) #72',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1163',\n",
       "       'name': 'Amazing Spider-Man (1999) #519',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1165',\n",
       "       'name': 'Amazing Spider-Man (1999) #520',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1167',\n",
       "       'name': 'Amazing Spider-Man (1999) #521',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1175',\n",
       "       'name': 'Amazing Spider-Man (1999) #523',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1193',\n",
       "       'name': 'Amazing Spider-Man (1999) #534',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1199',\n",
       "       'name': 'Amazing Spider-Man (1999) #537',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1203',\n",
       "       'name': 'Amazing Spider-Man (1999) #538',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1414',\n",
       "       'name': 'Cover #1414',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1500',\n",
       "       'name': 'Interior #1500',\n",
       "       'type': 'interiorStory'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1501',\n",
       "       'name': 'CAPTAIN AMERICA (2002) #21',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1503',\n",
       "       'name': 'CAPTAIN AMERICA (2002) #22',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1505',\n",
       "       'name': 'CAPTAIN AMERICA (2002) #23',\n",
       "       'type': 'cover'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/stories/1606',\n",
       "       'name': 'WEAPON X (2002) #14',\n",
       "       'type': 'cover'}],\n",
       "     'returned': 20},\n",
       "    'events': {'available': 30,\n",
       "     'collectionURI': 'http://gateway.marvel.com/v1/public/characters/1009220/events',\n",
       "     'items': [{'resourceURI': 'http://gateway.marvel.com/v1/public/events/116',\n",
       "       'name': 'Acts of Vengeance!'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/314',\n",
       "       'name': 'Age of Ultron'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/303',\n",
       "       'name': 'Age of X'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/231',\n",
       "       'name': 'Armor Wars'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/234',\n",
       "       'name': 'Avengers Disassembled'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/310',\n",
       "       'name': 'Avengers VS X-Men'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/320',\n",
       "       'name': 'Axis'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/296',\n",
       "       'name': 'Chaos War'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/238',\n",
       "       'name': 'Civil War'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/248',\n",
       "       'name': 'Fall of the Mutants'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/302',\n",
       "       'name': 'Fear Itself'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/251',\n",
       "       'name': 'House of M'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/252',\n",
       "       'name': 'Inferno'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/315',\n",
       "       'name': 'Infinity'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/29',\n",
       "       'name': 'Infinity War'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/317',\n",
       "       'name': 'Inhumanity'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/151',\n",
       "       'name': 'Maximum Carnage'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/37',\n",
       "       'name': 'Maximum Security'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/333',\n",
       "       'name': 'Monsters Unleashed'},\n",
       "      {'resourceURI': 'http://gateway.marvel.com/v1/public/events/337',\n",
       "       'name': 'Monsters Unleashed'}],\n",
       "     'returned': 20},\n",
       "    'urls': [{'type': 'detail',\n",
       "      'url': 'http://marvel.com/comics/characters/1009220/captain_america?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3'},\n",
       "     {'type': 'wiki',\n",
       "      'url': 'http://marvel.com/universe/Captain_America_(Steve_Rogers)?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3'},\n",
       "     {'type': 'comiclink',\n",
       "      'url': 'http://marvel.com/comics/characters/1009220/captain_america?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3'}]}]}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's inspect one of the resulting JSON files by looking at the first element of the list.  \n",
    "response_[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "________________\n",
    "Looking at the response, we can see that the data we want is in the following pathway: \n",
    ">response -> data -> results\n",
    "\n",
    "From it, we can extract the following data:\n",
    "* Character ID.\n",
    "* Character Name.\n",
    "* Description.\n",
    "* Last Modify Date for the Record.\n",
    "* Comics Data, which includes:\n",
    "    * Total Number of Comics the Character has appeared in.\n",
    "    * Comic ID.\n",
    "    * Comic Name.\n",
    "* Series Data, which includes:\n",
    "    * Total Number of Series the Character has appeared in.\n",
    "    * Series ID.\n",
    "    * Series Name.\n",
    "* Stories Data, which includes:\n",
    "    * Total Number of Stories the Character has appeared in.\n",
    "    * Stories ID.\n",
    "    * Story Name.\n",
    "    * Story Type.\n",
    "* Event Data, which includes:\n",
    "    * Total Number of Events the Character has appeared in.\n",
    "    * Event ID.\n",
    "    * Event Name.\n",
    "__________________"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Next Step: Creating a Dataframe.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the results data and save it into a list.\n",
    "char_data = [] \n",
    "for i in response_:\n",
    "    char_data.append(i['data']['results'])\n",
    "\n",
    "    marvel_ = [] \n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        details = []\n",
    "        details.append(data['id'])\n",
    "        details.append(data['name'])\n",
    "        details.append(data['description'])\n",
    "        details.append(data['modified'])\n",
    "        details.append(data['comics']['available'])\n",
    "        details.append(data['series']['available'])\n",
    "        details.append(data['stories']['available'])\n",
    "        details.append(data['events']['available'])\n",
    "        for i in range(len(data['urls'])):\n",
    "            type_ = data['urls'][i]['type']\n",
    "            url_ = data['urls'][i]['url']\n",
    "            details.append(f'{type_}: {url_}')\n",
    "        marvel_.append(details)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "10\n",
      "11\n"
     ]
    }
   ],
   "source": [
    "# Before we convert this list into a dataframe, we need to check the length of each individual element. \n",
    "for i in marvel_:\n",
    "    print(len(i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1017111,\n",
       " 'She-Hulk (HAS)',\n",
       " \"She-Hulk is the Hulk's action-loving cousin. She's unbelievably strong, pilots the group's heavily armed Jump Jet, and uses a pair of Gamma Gauntlets that give her fists an added energy wallop. As the most competitive member of the team, She-Hulk never backs down from a fight, arm-wrestling match, or burping contest!  \",\n",
       " '2013-09-18T15:53:40-0400',\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 'detail: http://marvel.com/characters/51/she-hulk?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3',\n",
       " 'comiclink: http://marvel.com/comics/characters/1017111/she-hulk_has?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Inspecting the 12th list against the first list.\n",
    "marvel_[11]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1009220,\n",
       " 'Captain America',\n",
       " \"Vowing to serve his country any way he could, young Steve Rogers took the super soldier serum to become America's one-man army. Fighting for the red, white and blue for over 60 years, Captain America is the living, breathing symbol of freedom and liberty.\",\n",
       " '2016-09-06T11:37:19-0400',\n",
       " 2350,\n",
       " 643,\n",
       " 3491,\n",
       " 30,\n",
       " 'detail: http://marvel.com/comics/characters/1009220/captain_america?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3',\n",
       " 'wiki: http://marvel.com/universe/Captain_America_(Steve_Rogers)?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3',\n",
       " 'comiclink: http://marvel.com/comics/characters/1009220/captain_america?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3']"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "marvel_[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "She_Hulk does not have wiki url. Let's take a look at the total count of urls available per character before we continue any further."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Captain America ['detail', 'wiki', 'comiclink']\n",
      "Thor ['detail', 'wiki', 'comiclink']\n",
      "Iron Man ['detail', 'wiki', 'comiclink']\n",
      "Black Widow ['detail', 'wiki', 'comiclink']\n",
      "Hulk ['detail', 'wiki', 'comiclink']\n",
      "Hawkeye ['detail', 'wiki', 'comiclink']\n",
      "Spider-Man ['detail', 'wiki', 'comiclink']\n",
      "Scarlet Witch ['detail', 'wiki', 'comiclink']\n",
      "Vision ['detail', 'wiki', 'comiclink']\n",
      "Captain Marvel (Carol Danvers) ['detail', 'wiki', 'comiclink']\n",
      "Nova ['detail', 'wiki', 'comiclink']\n",
      "She-Hulk (HAS) ['detail', 'comiclink']\n",
      "Wolverine ['detail', 'wiki', 'comiclink']\n"
     ]
    }
   ],
   "source": [
    "# Iterate over the urls to get a list of what types of url's each character has.\n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        url_t = []\n",
    "        for i in range(len(data['urls'])):\n",
    "            url_t.append(data['urls'][i]['type'])\n",
    "        print(data['name'],url_t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "____________\n",
    "With the pre-selected data, it appears that at most, each character can have up to 3 urls. In this case, with She-Hulk missing the wiki url, we can make the following adjustment to our code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "marvel_ = [] \n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        details = []\n",
    "        details.append(data['id'])\n",
    "        details.append(data['name'])\n",
    "        details.append(data['description'])\n",
    "        details.append(data['modified'])\n",
    "        details.append(data['comics']['available'])\n",
    "        details.append(data['series']['available'])\n",
    "        details.append(data['stories']['available'])\n",
    "        details.append(data['events']['available'])\n",
    "        for i in range(len(data['urls'])):\n",
    "            type_ = data['urls'][i]['type']\n",
    "            url_ = data['urls'][i]['url']\n",
    "            details.append(f'{type_}: {url_}')\n",
    "        if len(data['urls']) <= 2: # Added conditional statement, if the url dict length is less then two, then add a blank url.\n",
    "            details.append(f'wiki: NULL')  \n",
    "        marvel_.append(details)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n",
      "11\n"
     ]
    }
   ],
   "source": [
    "# Taking a look at length of the resulting lists once more.\n",
    "for i in marvel_:\n",
    "    print(len(i))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1017111,\n",
       " 'She-Hulk (HAS)',\n",
       " \"She-Hulk is the Hulk's action-loving cousin. She's unbelievably strong, pilots the group's heavily armed Jump Jet, and uses a pair of Gamma Gauntlets that give her fists an added energy wallop. As the most competitive member of the team, She-Hulk never backs down from a fight, arm-wrestling match, or burping contest!  \",\n",
       " '2013-09-18T15:53:40-0400',\n",
       " 0,\n",
       " 1,\n",
       " 1,\n",
       " 0,\n",
       " 'detail: http://marvel.com/characters/51/she-hulk?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3',\n",
       " 'comiclink: http://marvel.com/comics/characters/1017111/she-hulk_has?utm_campaign=apiRef&utm_source=385cebbcb01e23edbff1c01d0be2dca3',\n",
       " 'wiki: NULL']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "marvel_[11]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "______________\n",
    "Now that the lists are of equal length, let's work on the next item - The record modify date. Let's split the date from the time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in marvel_:\n",
    "    date, time = i[3].split(\"T\")\n",
    "    i.append(date)\n",
    "    i.append(time)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n",
      "13\n"
     ]
    }
   ],
   "source": [
    "# Checking the length of the lists to ensure they still match\n",
    "for i in marvel_:\n",
    "    print(len(i))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---------------\n",
    "Next, we drill into the nested data that lies within the comics and series data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 200,
   "metadata": {},
   "outputs": [],
   "source": [
    "comics_= []\n",
    "series_= []\n",
    "stories_= []\n",
    "event_= []\n",
    "\n",
    "# Iterate over the results, to then iterate over the data to drill into the respective data. \n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        for i in range(len(data['comics']['items'])):\n",
    "            details = [] # Temp list to hold the individual records\n",
    "            details.append(data['id']) # Append the character ID - We will use this later to join the data.\n",
    "            details.append(data['comics']['items'][i]['resourceURI'].rpartition('/')[2]) # Extract the comic ID from the URL.\n",
    "            details.append((data['comics']['items'][i]['name']))\n",
    "            comics_.append(details)\n",
    "\n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        for i in range(len(data['series']['items'])):\n",
    "            details = []\n",
    "            details.append(data['id'])\n",
    "            details.append(data['series']['items'][i]['resourceURI'].rpartition('/')[2])\n",
    "            details.append((data['series']['items'][i]['name']))\n",
    "            series_.append(details)\n",
    "\n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        for i in range(len(data['stories']['items'])):\n",
    "            details = []\n",
    "            details.append(data['id'])\n",
    "            details.append(data['stories']['items'][i]['resourceURI'].rpartition('/')[2])\n",
    "            details.append((data['stories']['items'][i]['name']))\n",
    "            details.append((data['stories']['items'][i]['type']))\n",
    "            stories_.append(details)\n",
    "\n",
    "for results in char_data:\n",
    "    for data in results:\n",
    "        for i in range(len(data['events']['items'])):\n",
    "            details = []\n",
    "            details.append(data['id'])\n",
    "            details.append(data['events']['items'][i]['resourceURI'].rpartition('/')[2])\n",
    "            details.append((data['events']['items'][i]['name']))\n",
    "            event_.append(details)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "___________________\n",
    "Let's take a look at the lists for each of these data to see what we can get. First, we want to extract the name, year and issue data from the comics. We will accomplish this with the use of regex. Inspecting the data, we notice that the year is encapsulated on the first set of parentheses. Though this isn't always the case as some data does not contain the year as seen below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1009220, '43488', 'A+X (2012) #1'],\n",
       " [1009220, '43501', 'A+X (2012) #4'],\n",
       " [1009220, '43508', 'A+X (2012) #9'],\n",
       " [1009220, '17743', 'A-Next (1998) #2'],\n",
       " [1009220, '17744', 'A-Next (1998) #3'],\n",
       " [1009220, '17745', 'A-Next (1998) #4'],\n",
       " [1009220, '17746', 'A-Next (1998) #5'],\n",
       " [1009220, '17747', 'A-Next (1998) #6'],\n",
       " [1009220, '17748', 'A-Next (1998) #7'],\n",
       " [1009220, '17749', 'A-Next (1998) #8'],\n",
       " [1009220, '17750', 'A-Next (1998) #9'],\n",
       " [1009220, '17740', 'A-Next (1998) #10'],\n",
       " [1009220, '17741', 'A-Next (1998) #11'],\n",
       " [1009220, '17742', 'A-Next (1998) #12'],\n",
       " [1009220, '66978', 'Adventures of Captain America (1991) #1'],\n",
       " [1009220, '66979', 'Adventures of Captain America (1991) #2'],\n",
       " [1009220, '66980', 'Adventures of Captain America (1991) #3'],\n",
       " [1009220, '66981', 'Adventures of Captain America (1991) #4'],\n",
       " [1009220,\n",
       "  '42539',\n",
       "  'Age of Apocalypse (2011) #2 (Avengers Art Appreciation Variant)'],\n",
       " [1009220, '30090', 'Age of Heroes (2010) #1'],\n",
       " [1009664, '43506', 'A+X (2012) #7'],\n",
       " [1009664, '30090', 'Age of Heroes (2010) #1'],\n",
       " [1009664, '33566', 'Age of Heroes (2010) #2'],\n",
       " [1009664, '30092', 'Age of Heroes (2010) #3'],\n",
       " [1009664, '30093', 'Age of Heroes (2010) #4']]"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comics_[0:25]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For loop to iterate over the comics_ list\n",
    "for i in comics_:\n",
    "    # Append the comic book name.\n",
    "    name = i[2].rpartition('(')[0]\n",
    "    # Try/Except block, in the event that the text does not have parenthesis with the year, then returns a Null value by default.\n",
    "    i.append(name)\n",
    "    try: \n",
    "        val = re.search(r'\\((.*?)\\)',i[2]).group(1) # Regex expression that extracts the contents on the first set of parentheses.\n",
    "    # Conditional statement to check if the contents inside the parentheses are digits, if not then append NULL as the value.\n",
    "        if val.isdigit():\n",
    "            i.append(val)\n",
    "        else:\n",
    "            i.append(\"NULL\")\n",
    "    except:\n",
    "        i.append(\"NULL\")\n",
    "    val2 = i[2].rpartition(' ')[-1]\n",
    "    if val2[0] == '#':\n",
    "        i.append(val2)\n",
    "    else:\n",
    "        val_r1 = re.sub('[()]', '', val2) # Remove lingering parenthesis from the any non-issue number data. \n",
    "        i.append(val_r1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1009220, '43488', 'A+X (2012) #1', 'A+X ', '2012', '#1'],\n",
       " [1009220, '43501', 'A+X (2012) #4', 'A+X ', '2012', '#4'],\n",
       " [1009220, '43508', 'A+X (2012) #9', 'A+X ', '2012', '#9'],\n",
       " [1009220, '17743', 'A-Next (1998) #2', 'A-Next ', '1998', '#2'],\n",
       " [1009220, '17744', 'A-Next (1998) #3', 'A-Next ', '1998', '#3'],\n",
       " [1009220, '17745', 'A-Next (1998) #4', 'A-Next ', '1998', '#4'],\n",
       " [1009220, '17746', 'A-Next (1998) #5', 'A-Next ', '1998', '#5'],\n",
       " [1009220, '17747', 'A-Next (1998) #6', 'A-Next ', '1998', '#6'],\n",
       " [1009220, '17748', 'A-Next (1998) #7', 'A-Next ', '1998', '#7'],\n",
       " [1009220, '17749', 'A-Next (1998) #8', 'A-Next ', '1998', '#8']]"
      ]
     },
     "execution_count": 206,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A glimpse of the resulting data. \n",
    "comics_[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------\n",
    "Next, we take a look at the series data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1009220, '16450', 'A+X (2012 - 2014)'],\n",
       " [1009220, '3620', 'A-Next (1998 - 1999)'],\n",
       " [1009220, '24227', 'Adventures of Captain America (1991 - 1992)'],\n",
       " [1009220, '15331', 'Age of Apocalypse (2011 - 2013)'],\n",
       " [1009220, '9790', 'Age of Heroes (2010)'],\n",
       " [1009220, '10235', 'AGE OF HEROES TPB (2011)'],\n",
       " [1009220, '13896', 'Age of X: Universe (2011)'],\n",
       " [1009220, '7534', 'All Winners Comics 70th Anniversary Special (2009)'],\n",
       " [1009220, '20682', 'All-New Wolverine (2015 - 2018)'],\n",
       " [1009220, '2114', 'All-Winners Comics (1941 - 1947)'],\n",
       " [1009220, '9865', 'All-Winners Squad: Band of Heroes (2011)'],\n",
       " [1009220, '2116', 'Alpha Flight (1983 - 1994)'],\n",
       " [1009220, '454', 'Amazing Spider-Man (1999 - 2013)'],\n",
       " [1009220, '15540', 'Amazing Spider-Man Annual (2012)'],\n",
       " [1009220, '2984', 'Amazing Spider-Man Annual (1964 - 2018)'],\n",
       " [1009220, '9802', 'Amazing Spider-Man Annual (2010)'],\n",
       " [1009220, '1489', 'AMAZING SPIDER-MAN VOL. 10: NEW AVENGERS TPB (2005)'],\n",
       " [1009220, '14818', 'Annihilators: Earthfall (2011)'],\n",
       " [1009220, '24323', 'Ant-Man and the Wasp Adventures (2018)'],\n",
       " [1009220,\n",
       "  '14696',\n",
       "  'Art of Marvel Movies: The Art of Captain America - The First Avenger (2011 - Present)'],\n",
       " [1009664, '16450', 'A+X (2012 - 2014)'],\n",
       " [1009664, '9790', 'Age of Heroes (2010)'],\n",
       " [1009664, '2116', 'Alpha Flight (1983 - 1994)'],\n",
       " [1009664, '17650', 'Alpha: Big Time (2013)'],\n",
       " [1009664, '454', 'Amazing Spider-Man (1999 - 2013)']]"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series_[0:25]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we want the name and year(s) information. It looks similar to what we just did, however for the year we get either a single digit or we get a range. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 201,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in series_:\n",
    "    # Append the series name.\n",
    "    name = i[2].rpartition('(')[0]\n",
    "    i.append(name)\n",
    "    try:\n",
    "        val = re.search(r'\\((.*?)\\)',i[2]).group(1) # Regex expression that extracts the contents on the first set of parentheses.\n",
    "    # Here we change the conditional statement to identify date ranges by looking for \"-\". If it does not find the \"-\" then it will\n",
    "    # assume the value as both the start and end year of the series with a duration of 0 years. Else, it will split on \"-\", assigning\n",
    "    # the start and end year as well as calculate the duration in years. \n",
    "        if val.find(\"-\") == -1:\n",
    "            start = int(val.strip())\n",
    "            end = int(val.strip())\n",
    "            duration = 0\n",
    "            i.append(start)\n",
    "            i.append(end)\n",
    "            i.append(duration)\n",
    "        else:\n",
    "            start, end = val.split('-')\n",
    "            start = int(start.strip())\n",
    "            end = int(end.strip())\n",
    "            duration = end - start\n",
    "            i.append(start)\n",
    "            i.append(end)\n",
    "            i.append(duration)\n",
    "    except:\n",
    "        test.append(\"NULL\")\n",
    "        test.append(\"NULL\")\n",
    "        test.append(\"NULL\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 202,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[1009220, '16450', 'A+X (2012 - 2014)', 'A+X ', 2012, 2014, 2],\n",
       " [1009220, '3620', 'A-Next (1998 - 1999)', 'A-Next ', 1998, 1999, 1],\n",
       " [1009220,\n",
       "  '24227',\n",
       "  'Adventures of Captain America (1991 - 1992)',\n",
       "  'Adventures of Captain America ',\n",
       "  1991,\n",
       "  1992,\n",
       "  1],\n",
       " [1009220,\n",
       "  '15331',\n",
       "  'Age of Apocalypse (2011 - 2013)',\n",
       "  'Age of Apocalypse ',\n",
       "  2011,\n",
       "  2013,\n",
       "  2],\n",
       " [1009220, '9790', 'Age of Heroes (2010)', 'Age of Heroes ', 2010, 2010, 0],\n",
       " [1009220,\n",
       "  '10235',\n",
       "  'AGE OF HEROES TPB (2011)',\n",
       "  'AGE OF HEROES TPB ',\n",
       "  2011,\n",
       "  2011,\n",
       "  0],\n",
       " [1009220,\n",
       "  '13896',\n",
       "  'Age of X: Universe (2011)',\n",
       "  'Age of X: Universe ',\n",
       "  2011,\n",
       "  2011,\n",
       "  0],\n",
       " [1009220,\n",
       "  '7534',\n",
       "  'All Winners Comics 70th Anniversary Special (2009)',\n",
       "  'All Winners Comics 70th Anniversary Special ',\n",
       "  2009,\n",
       "  2009,\n",
       "  0],\n",
       " [1009220,\n",
       "  '20682',\n",
       "  'All-New Wolverine (2015 - 2018)',\n",
       "  'All-New Wolverine ',\n",
       "  2015,\n",
       "  2018,\n",
       "  3],\n",
       " [1009220,\n",
       "  '2114',\n",
       "  'All-Winners Comics (1941 - 1947)',\n",
       "  'All-Winners Comics ',\n",
       "  1941,\n",
       "  1947,\n",
       "  6]]"
      ]
     },
     "execution_count": 202,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series_[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---------------\n",
    "Now we construct our dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the headers-data.\n",
    "headers = ['char_id', 'char_name','char_description','del_mod_datetime', 'comics_total', 'series_total', 'stories_total', 'events_total',\n",
    "          'url_1','url_2','url_3','record_mod_date','record_mod_time']\n",
    "marvel_df = pd.DataFrame(marvel_, columns=headers) # Create the dataframe\n",
    "marvel_df = marvel_df.drop('del_mod_datetime', axis=1)\n",
    "marvel_df = marvel_df[['char_id', 'char_name','char_description','record_mod_date', 'record_mod_time','comics_total', 'series_total', 'stories_total', 'events_total',\n",
    "          'url_1','url_2','url_3']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>char_name</th>\n",
       "      <th>char_description</th>\n",
       "      <th>record_mod_date</th>\n",
       "      <th>record_mod_time</th>\n",
       "      <th>comics_total</th>\n",
       "      <th>series_total</th>\n",
       "      <th>stories_total</th>\n",
       "      <th>events_total</th>\n",
       "      <th>url_1</th>\n",
       "      <th>url_2</th>\n",
       "      <th>url_3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009664</td>\n",
       "      <td>Thor</td>\n",
       "      <td>As the Norse God of thunder and lightning, Tho...</td>\n",
       "      <td>2019-02-06</td>\n",
       "      <td>18:10:24-0500</td>\n",
       "      <td>1749</td>\n",
       "      <td>501</td>\n",
       "      <td>2657</td>\n",
       "      <td>27</td>\n",
       "      <td>detail: http://marvel.com/characters/1009664/t...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Thor_(Thor_Od...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009368</td>\n",
       "      <td>Iron Man</td>\n",
       "      <td>Wounded, captured and forced to build a weapon...</td>\n",
       "      <td>2016-09-28</td>\n",
       "      <td>12:08:19-0400</td>\n",
       "      <td>2562</td>\n",
       "      <td>627</td>\n",
       "      <td>3880</td>\n",
       "      <td>31</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Iron_Man_(Ant...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009189</td>\n",
       "      <td>Black Widow</td>\n",
       "      <td></td>\n",
       "      <td>2016-01-04</td>\n",
       "      <td>18:09:26-0500</td>\n",
       "      <td>546</td>\n",
       "      <td>206</td>\n",
       "      <td>628</td>\n",
       "      <td>16</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Black_Widow_(...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009351</td>\n",
       "      <td>Hulk</td>\n",
       "      <td>Caught in a gamma bomb explosion while trying ...</td>\n",
       "      <td>2016-06-02</td>\n",
       "      <td>12:38:33-0400</td>\n",
       "      <td>1650</td>\n",
       "      <td>466</td>\n",
       "      <td>2529</td>\n",
       "      <td>26</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Hulk_(Bruce_B...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id        char_name  \\\n",
       "0  1009220  Captain America   \n",
       "1  1009664             Thor   \n",
       "2  1009368         Iron Man   \n",
       "3  1009189      Black Widow   \n",
       "4  1009351             Hulk   \n",
       "\n",
       "                                    char_description record_mod_date  \\\n",
       "0  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "1  As the Norse God of thunder and lightning, Tho...      2019-02-06   \n",
       "2  Wounded, captured and forced to build a weapon...      2016-09-28   \n",
       "3                                                         2016-01-04   \n",
       "4  Caught in a gamma bomb explosion while trying ...      2016-06-02   \n",
       "\n",
       "  record_mod_time  comics_total  series_total  stories_total  events_total  \\\n",
       "0   11:37:19-0400          2350           643           3491            30   \n",
       "1   18:10:24-0500          1749           501           2657            27   \n",
       "2   12:08:19-0400          2562           627           3880            31   \n",
       "3   18:09:26-0500           546           206            628            16   \n",
       "4   12:38:33-0400          1650           466           2529            26   \n",
       "\n",
       "                                               url_1  \\\n",
       "0  detail: http://marvel.com/comics/characters/10...   \n",
       "1  detail: http://marvel.com/characters/1009664/t...   \n",
       "2  detail: http://marvel.com/comics/characters/10...   \n",
       "3  detail: http://marvel.com/comics/characters/10...   \n",
       "4  detail: http://marvel.com/comics/characters/10...   \n",
       "\n",
       "                                               url_2  \\\n",
       "0  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "1  wiki: http://marvel.com/universe/Thor_(Thor_Od...   \n",
       "2  wiki: http://marvel.com/universe/Iron_Man_(Ant...   \n",
       "3  wiki: http://marvel.com/universe/Black_Widow_(...   \n",
       "4  wiki: http://marvel.com/universe/Hulk_(Bruce_B...   \n",
       "\n",
       "                                               url_3  \n",
       "0  comiclink: http://marvel.com/comics/characters...  \n",
       "1  comiclink: http://marvel.com/comics/characters...  \n",
       "2  comiclink: http://marvel.com/comics/characters...  \n",
       "3  comiclink: http://marvel.com/comics/characters...  \n",
       "4  comiclink: http://marvel.com/comics/characters...  "
      ]
     },
     "execution_count": 242,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "marvel_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "________\n",
    "Next we move on to the comics, events, series and stories data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the headers\n",
    "headers_comics = ['char_id', 'comic_id', 'comic_composite', 'comic_name', 'comic_year', 'comic_issue']\n",
    "headers_series = ['char_id', 'series_id', 'series_composite','series_name', 'series_start_year', 'series_end_year', 'series_duration']\n",
    "headers_stories = ['char_id', 'story_id', 'story_name', 'story_type']\n",
    "headers_events = ['char_id', 'event_id', 'event_name']\n",
    "# Create the data frames.\n",
    "comics_df = pd.DataFrame(comics_, columns=headers_comics)\n",
    "series_df = pd.DataFrame(series_, columns=headers_series)\n",
    "stories_df = pd.DataFrame(stories_, columns=headers_stories)\n",
    "events_df = pd.DataFrame(event_, columns=headers_events)\n",
    "# drop the composites.\n",
    "comics_df = comics_df.drop('comic_composite', axis=1)\n",
    "series_df = series_df.drop('series_composite', axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we inspect the dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 221,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>comic_id</th>\n",
       "      <th>comic_name</th>\n",
       "      <th>comic_year</th>\n",
       "      <th>comic_issue</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43488</td>\n",
       "      <td>A+X</td>\n",
       "      <td>2012</td>\n",
       "      <td>#1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43501</td>\n",
       "      <td>A+X</td>\n",
       "      <td>2012</td>\n",
       "      <td>#4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43508</td>\n",
       "      <td>A+X</td>\n",
       "      <td>2012</td>\n",
       "      <td>#9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>17743</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>1998</td>\n",
       "      <td>#2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>17744</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>1998</td>\n",
       "      <td>#3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id comic_id comic_name comic_year comic_issue\n",
       "0  1009220    43488       A+X        2012          #1\n",
       "1  1009220    43501       A+X        2012          #4\n",
       "2  1009220    43508       A+X        2012          #9\n",
       "3  1009220    17743    A-Next        1998          #2\n",
       "4  1009220    17744    A-Next        1998          #3"
      ]
     },
     "execution_count": 221,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comics_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 214,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>series_id</th>\n",
       "      <th>series_name</th>\n",
       "      <th>series_start_year</th>\n",
       "      <th>series_end_year</th>\n",
       "      <th>series_duration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>3620</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1999.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>24227</td>\n",
       "      <td>Adventures of Captain America</td>\n",
       "      <td>1991.0</td>\n",
       "      <td>1992.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>15331</td>\n",
       "      <td>Age of Apocalypse</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>2013.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>9790</td>\n",
       "      <td>Age of Heroes</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id series_id                     series_name  series_start_year  \\\n",
       "0  1009220     16450                            A+X              2012.0   \n",
       "1  1009220      3620                         A-Next              1998.0   \n",
       "2  1009220     24227  Adventures of Captain America              1991.0   \n",
       "3  1009220     15331              Age of Apocalypse              2011.0   \n",
       "4  1009220      9790                  Age of Heroes              2010.0   \n",
       "\n",
       "   series_end_year  series_duration  \n",
       "0           2014.0              2.0  \n",
       "1           1999.0              1.0  \n",
       "2           1992.0              1.0  \n",
       "3           2013.0              2.0  \n",
       "4           2010.0              0.0  "
      ]
     },
     "execution_count": 214,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>story_id</th>\n",
       "      <th>story_name</th>\n",
       "      <th>story_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>670</td>\n",
       "      <td>X-MEN (2004) #186</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>892</td>\n",
       "      <td>THOR (1998) #81</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>960</td>\n",
       "      <td>3 of ?</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>1026</td>\n",
       "      <td>Avengers (1998) #81</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>1041</td>\n",
       "      <td>Avengers (1998) #502</td>\n",
       "      <td>interiorStory</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id story_id            story_name     story_type\n",
       "0  1009220      670     X-MEN (2004) #186          cover\n",
       "1  1009220      892       THOR (1998) #81          cover\n",
       "2  1009220      960                3 of ?          cover\n",
       "3  1009220     1026   Avengers (1998) #81          cover\n",
       "4  1009220     1041  Avengers (1998) #502  interiorStory"
      ]
     },
     "execution_count": 215,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stories_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>event_id</th>\n",
       "      <th>event_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>116</td>\n",
       "      <td>Acts of Vengeance!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>314</td>\n",
       "      <td>Age of Ultron</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>303</td>\n",
       "      <td>Age of X</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>231</td>\n",
       "      <td>Armor Wars</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>234</td>\n",
       "      <td>Avengers Disassembled</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id event_id             event_name\n",
       "0  1009220      116     Acts of Vengeance!\n",
       "1  1009220      314          Age of Ultron\n",
       "2  1009220      303               Age of X\n",
       "3  1009220      231             Armor Wars\n",
       "4  1009220      234  Avengers Disassembled"
      ]
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next - We join Comics data with the Series data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 222,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "char_id         int64\n",
       "comic_id       object\n",
       "comic_name     object\n",
       "comic_year     object\n",
       "comic_issue    object\n",
       "dtype: object"
      ]
     },
     "execution_count": 222,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Checking the data types on comics and series.\n",
    "comics_df.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 220,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "char_id                int64\n",
       "series_id             object\n",
       "series_name           object\n",
       "series_start_year    float64\n",
       "series_end_year      float64\n",
       "series_duration      float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 220,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "series_df.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 225,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Convert the the comic_year into a float, any errors/null values are coerced. \n",
    "comics_df[\"comic_year\"] = pd.to_numeric(comics_df[\"comic_year\"], errors='coerce')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 227,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "char_id          int64\n",
       "comic_id        object\n",
       "comic_name      object\n",
       "comic_year     float64\n",
       "comic_issue     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 227,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comics_df.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Joining the dataframes.\n",
    "comic_series = pd.merge(comics_df, series_df, left_on=  ['char_id', 'comic_name', 'comic_year'], \n",
    "                        right_on= ['char_id','series_name', 'series_start_year'], \n",
    "                        how = 'left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>comic_id</th>\n",
       "      <th>series_id</th>\n",
       "      <th>comic_name</th>\n",
       "      <th>comic_issue</th>\n",
       "      <th>comic_year</th>\n",
       "      <th>series_start_year</th>\n",
       "      <th>series_end_year</th>\n",
       "      <th>series_duration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43488</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#1</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43501</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#4</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>43508</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#9</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>17743</td>\n",
       "      <td>3620</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>#2</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1999.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>17744</td>\n",
       "      <td>3620</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>#3</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1999.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id comic_id series_id comic_name comic_issue  comic_year  \\\n",
       "0  1009220    43488     16450       A+X           #1      2012.0   \n",
       "1  1009220    43501     16450       A+X           #4      2012.0   \n",
       "2  1009220    43508     16450       A+X           #9      2012.0   \n",
       "3  1009220    17743      3620    A-Next           #2      1998.0   \n",
       "4  1009220    17744      3620    A-Next           #3      1998.0   \n",
       "\n",
       "   series_start_year  series_end_year  series_duration  \n",
       "0             2012.0           2014.0              2.0  \n",
       "1             2012.0           2014.0              2.0  \n",
       "2             2012.0           2014.0              2.0  \n",
       "3             1998.0           1999.0              1.0  \n",
       "4             1998.0           1999.0              1.0  "
      ]
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "comic_series = comic_series[['char_id', 'comic_id', 'series_id', 'comic_name', 'comic_issue', 'comic_year', 'series_start_year',\n",
    "                'series_end_year', 'series_duration']]\n",
    "\n",
    "comic_series.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__________\n",
    "Finally, let's join the character data with the merged comic/series data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {},
   "outputs": [],
   "source": [
    "merge_marvel = pd.merge(marvel_df, comic_series, on=['char_id'], how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>char_name</th>\n",
       "      <th>char_description</th>\n",
       "      <th>record_mod_date</th>\n",
       "      <th>record_mod_time</th>\n",
       "      <th>comics_total</th>\n",
       "      <th>series_total</th>\n",
       "      <th>stories_total</th>\n",
       "      <th>events_total</th>\n",
       "      <th>url_1</th>\n",
       "      <th>url_2</th>\n",
       "      <th>url_3</th>\n",
       "      <th>comic_id</th>\n",
       "      <th>series_id</th>\n",
       "      <th>comic_name</th>\n",
       "      <th>comic_issue</th>\n",
       "      <th>comic_year</th>\n",
       "      <th>series_start_year</th>\n",
       "      <th>series_end_year</th>\n",
       "      <th>series_duration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>43488</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#1</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>43501</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#4</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>43508</td>\n",
       "      <td>16450</td>\n",
       "      <td>A+X</td>\n",
       "      <td>#9</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2012.0</td>\n",
       "      <td>2014.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>17743</td>\n",
       "      <td>3620</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>#2</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1999.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>17744</td>\n",
       "      <td>3620</td>\n",
       "      <td>A-Next</td>\n",
       "      <td>#3</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1998.0</td>\n",
       "      <td>1999.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id        char_name  \\\n",
       "0  1009220  Captain America   \n",
       "1  1009220  Captain America   \n",
       "2  1009220  Captain America   \n",
       "3  1009220  Captain America   \n",
       "4  1009220  Captain America   \n",
       "\n",
       "                                    char_description record_mod_date  \\\n",
       "0  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "1  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "2  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "3  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "4  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "\n",
       "  record_mod_time  comics_total  series_total  stories_total  events_total  \\\n",
       "0   11:37:19-0400          2350           643           3491            30   \n",
       "1   11:37:19-0400          2350           643           3491            30   \n",
       "2   11:37:19-0400          2350           643           3491            30   \n",
       "3   11:37:19-0400          2350           643           3491            30   \n",
       "4   11:37:19-0400          2350           643           3491            30   \n",
       "\n",
       "                                               url_1  \\\n",
       "0  detail: http://marvel.com/comics/characters/10...   \n",
       "1  detail: http://marvel.com/comics/characters/10...   \n",
       "2  detail: http://marvel.com/comics/characters/10...   \n",
       "3  detail: http://marvel.com/comics/characters/10...   \n",
       "4  detail: http://marvel.com/comics/characters/10...   \n",
       "\n",
       "                                               url_2  \\\n",
       "0  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "1  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "2  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "3  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "4  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "\n",
       "                                               url_3 comic_id series_id  \\\n",
       "0  comiclink: http://marvel.com/comics/characters...    43488     16450   \n",
       "1  comiclink: http://marvel.com/comics/characters...    43501     16450   \n",
       "2  comiclink: http://marvel.com/comics/characters...    43508     16450   \n",
       "3  comiclink: http://marvel.com/comics/characters...    17743      3620   \n",
       "4  comiclink: http://marvel.com/comics/characters...    17744      3620   \n",
       "\n",
       "  comic_name comic_issue  comic_year  series_start_year  series_end_year  \\\n",
       "0       A+X           #1      2012.0             2012.0           2014.0   \n",
       "1       A+X           #4      2012.0             2012.0           2014.0   \n",
       "2       A+X           #9      2012.0             2012.0           2014.0   \n",
       "3    A-Next           #2      1998.0             1998.0           1999.0   \n",
       "4    A-Next           #3      1998.0             1998.0           1999.0   \n",
       "\n",
       "   series_duration  \n",
       "0              2.0  \n",
       "1              2.0  \n",
       "2              2.0  \n",
       "3              1.0  \n",
       "4              1.0  "
      ]
     },
     "execution_count": 240,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merge_marvel.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also do the same with the stories and events data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 243,
   "metadata": {},
   "outputs": [],
   "source": [
    "merge_marvel_stories = pd.merge(marvel_df, stories_df, on=['char_id'], how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>char_name</th>\n",
       "      <th>char_description</th>\n",
       "      <th>record_mod_date</th>\n",
       "      <th>record_mod_time</th>\n",
       "      <th>comics_total</th>\n",
       "      <th>series_total</th>\n",
       "      <th>stories_total</th>\n",
       "      <th>events_total</th>\n",
       "      <th>url_1</th>\n",
       "      <th>url_2</th>\n",
       "      <th>url_3</th>\n",
       "      <th>story_id</th>\n",
       "      <th>story_name</th>\n",
       "      <th>story_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>670</td>\n",
       "      <td>X-MEN (2004) #186</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>892</td>\n",
       "      <td>THOR (1998) #81</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>960</td>\n",
       "      <td>3 of ?</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>1026</td>\n",
       "      <td>Avengers (1998) #81</td>\n",
       "      <td>cover</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>1041</td>\n",
       "      <td>Avengers (1998) #502</td>\n",
       "      <td>interiorStory</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id        char_name  \\\n",
       "0  1009220  Captain America   \n",
       "1  1009220  Captain America   \n",
       "2  1009220  Captain America   \n",
       "3  1009220  Captain America   \n",
       "4  1009220  Captain America   \n",
       "\n",
       "                                    char_description record_mod_date  \\\n",
       "0  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "1  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "2  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "3  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "4  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "\n",
       "  record_mod_time  comics_total  series_total  stories_total  events_total  \\\n",
       "0   11:37:19-0400          2350           643           3491            30   \n",
       "1   11:37:19-0400          2350           643           3491            30   \n",
       "2   11:37:19-0400          2350           643           3491            30   \n",
       "3   11:37:19-0400          2350           643           3491            30   \n",
       "4   11:37:19-0400          2350           643           3491            30   \n",
       "\n",
       "                                               url_1  \\\n",
       "0  detail: http://marvel.com/comics/characters/10...   \n",
       "1  detail: http://marvel.com/comics/characters/10...   \n",
       "2  detail: http://marvel.com/comics/characters/10...   \n",
       "3  detail: http://marvel.com/comics/characters/10...   \n",
       "4  detail: http://marvel.com/comics/characters/10...   \n",
       "\n",
       "                                               url_2  \\\n",
       "0  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "1  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "2  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "3  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "4  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "\n",
       "                                               url_3 story_id  \\\n",
       "0  comiclink: http://marvel.com/comics/characters...      670   \n",
       "1  comiclink: http://marvel.com/comics/characters...      892   \n",
       "2  comiclink: http://marvel.com/comics/characters...      960   \n",
       "3  comiclink: http://marvel.com/comics/characters...     1026   \n",
       "4  comiclink: http://marvel.com/comics/characters...     1041   \n",
       "\n",
       "             story_name     story_type  \n",
       "0     X-MEN (2004) #186          cover  \n",
       "1       THOR (1998) #81          cover  \n",
       "2                3 of ?          cover  \n",
       "3   Avengers (1998) #81          cover  \n",
       "4  Avengers (1998) #502  interiorStory  "
      ]
     },
     "execution_count": 244,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merge_marvel_stories.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {},
   "outputs": [],
   "source": [
    "merge_marvel_events = pd.merge(marvel_df, events_df, on=['char_id'], how='left')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>char_id</th>\n",
       "      <th>char_name</th>\n",
       "      <th>char_description</th>\n",
       "      <th>record_mod_date</th>\n",
       "      <th>record_mod_time</th>\n",
       "      <th>comics_total</th>\n",
       "      <th>series_total</th>\n",
       "      <th>stories_total</th>\n",
       "      <th>events_total</th>\n",
       "      <th>url_1</th>\n",
       "      <th>url_2</th>\n",
       "      <th>url_3</th>\n",
       "      <th>event_id</th>\n",
       "      <th>event_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>116</td>\n",
       "      <td>Acts of Vengeance!</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>314</td>\n",
       "      <td>Age of Ultron</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>303</td>\n",
       "      <td>Age of X</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>231</td>\n",
       "      <td>Armor Wars</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1009220</td>\n",
       "      <td>Captain America</td>\n",
       "      <td>Vowing to serve his country any way he could, ...</td>\n",
       "      <td>2016-09-06</td>\n",
       "      <td>11:37:19-0400</td>\n",
       "      <td>2350</td>\n",
       "      <td>643</td>\n",
       "      <td>3491</td>\n",
       "      <td>30</td>\n",
       "      <td>detail: http://marvel.com/comics/characters/10...</td>\n",
       "      <td>wiki: http://marvel.com/universe/Captain_Ameri...</td>\n",
       "      <td>comiclink: http://marvel.com/comics/characters...</td>\n",
       "      <td>234</td>\n",
       "      <td>Avengers Disassembled</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   char_id        char_name  \\\n",
       "0  1009220  Captain America   \n",
       "1  1009220  Captain America   \n",
       "2  1009220  Captain America   \n",
       "3  1009220  Captain America   \n",
       "4  1009220  Captain America   \n",
       "\n",
       "                                    char_description record_mod_date  \\\n",
       "0  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "1  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "2  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "3  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "4  Vowing to serve his country any way he could, ...      2016-09-06   \n",
       "\n",
       "  record_mod_time  comics_total  series_total  stories_total  events_total  \\\n",
       "0   11:37:19-0400          2350           643           3491            30   \n",
       "1   11:37:19-0400          2350           643           3491            30   \n",
       "2   11:37:19-0400          2350           643           3491            30   \n",
       "3   11:37:19-0400          2350           643           3491            30   \n",
       "4   11:37:19-0400          2350           643           3491            30   \n",
       "\n",
       "                                               url_1  \\\n",
       "0  detail: http://marvel.com/comics/characters/10...   \n",
       "1  detail: http://marvel.com/comics/characters/10...   \n",
       "2  detail: http://marvel.com/comics/characters/10...   \n",
       "3  detail: http://marvel.com/comics/characters/10...   \n",
       "4  detail: http://marvel.com/comics/characters/10...   \n",
       "\n",
       "                                               url_2  \\\n",
       "0  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "1  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "2  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "3  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "4  wiki: http://marvel.com/universe/Captain_Ameri...   \n",
       "\n",
       "                                               url_3 event_id  \\\n",
       "0  comiclink: http://marvel.com/comics/characters...      116   \n",
       "1  comiclink: http://marvel.com/comics/characters...      314   \n",
       "2  comiclink: http://marvel.com/comics/characters...      303   \n",
       "3  comiclink: http://marvel.com/comics/characters...      231   \n",
       "4  comiclink: http://marvel.com/comics/characters...      234   \n",
       "\n",
       "              event_name  \n",
       "0     Acts of Vengeance!  \n",
       "1          Age of Ultron  \n",
       "2               Age of X  \n",
       "3             Armor Wars  \n",
       "4  Avengers Disassembled  "
      ]
     },
     "execution_count": 246,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merge_marvel_events.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-------------\n",
    "## Conclusion - Final Thoughts.\n",
    "This was my attempt at navigating the Marvel API. Some things I will like to try next round is to use the extracted Comic, Event, Stories and Series ID's to potentially requery the API and get more information. The API is pretty generous in terms of to how many times calls you can make (3000 as of the time of this writting), however and following the old adage that \"only fools rush in\", I probablt want to figure out what I want to get out of it. Once I do, there will be a part two for this.  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}