Regime Forecasting - I.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regime Forecasting I\n",
    "\n",
    "## Table of Contents:\n",
    "&nbsp;&nbsp;1. [Problem Description and Related Work](#1)\n",
    "\n",
    "&nbsp;&nbsp;2. [Understand Data](#2)   \n",
    "\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.0 [Set Up Environment](#2.0)\n",
    "\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.1 [Read Data and Description of the Variables](#2.1)\n",
    "\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2.2 [Explanatory Data Anlaysis](#2.2)\n",
    "\n",
    "&nbsp;&nbsp;3. [References](#3) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Problem Description and Related Work <a id=\"1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Business cycle describes the rise and fall in the growth of the economy that occurs over time. Each business cyle has two turning points trough (or bottom) and peak. Expansion is measured from the trough of the previous business cycle to the peak of the current cycle, while recession is measured from the peak to the trough. A representation of business cycles is shown in __[figure](https://courses.lumenlearning.com/baycollege-introbusiness/chapter/reading-the-business-cycle-definition-and-phases/)__ below. In the United States (US), the Business Cycle Dating Committee of the National Bureau of Economic Research (NBER) determines the dates for business cycles. The turning points are determined by considering monthly growth indicators of the economy such as industrial production, employment, real income. The main focus of business cycle analysis is to analyze why economy goes through contraction and expansion periods. It is a well-studied topic in the literature and still an active reserach area. In our work, we are going to predict recessions in US economy with leading macroeconomic indicators using machine learning algorithms."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predicting business cycle turning points, especially economic recessions, is of great importance to investors, households, businesses in the economy.\n",
    "Starting from &nbsp;[Mitchell and Burns (1938)](#a) pioneering work,  analyzing indicators of business cycles have become a core research area in business cycle analysis and there has been many work done in that field since then. Much of the work in the literature has indicated a wide range of economic and financial variables contain predictive information about future recessions in the economy.&nbsp;[Stock and Watson (1989)](#b) established coincedent and leading economic indicators for recession forecasting. &nbsp;[Estrella and Mishkin (1998)](#c) have documented predictive power of the slope of the term structure of Treasury yields and stock market for US recessions. \n",
    "&nbsp;[Liu and Moench (2016)](#f) have also showed the Treasury term spread has the highest predictive power with lagged abservations as well. Also, they indicated that balances in broker-dealer margin accounts have significant effect on recession predictions, especially in longer horizons.\n",
    "&nbsp;[Chionis et al. (2009)](#d) worked on forecasting recessions in Europe with European Union (EU) data. They found out that the yield curve augmented with the composite stock index has significant predictive\n",
    "power in terms of the EU real output.\n",
    "&nbsp;[Ng (2014)](#e) worked on the problem through exploring effectiveness of boosting and found out interest rate measures and certain employment variables have predictive power for the recessions.\n",
    "Recently, &nbsp;[Huang et al. (2018)](#g) have worked on predictive power of news sentiment analysis on recession forecasting in US economy. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Understand Data <a id=\"2\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.0 Set Up Environment <a id=\"2.0\"></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# load libraries\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt \n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Read Data and Description of the Variables <a id=\"2.1\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our analysis we will use a large macroeconomic database from FRED St. Louis designed by [McCracken and Ng (2015)](#i). It involves 129 macroeconomic monthly time series over the period 1959-2018. The data is organized into 8 categories (1)output and income, (2)labor market, (3)housing, (4)consumption, orders and inventories, (5)money and credit, (6)interest and exchange rates, (7)prices and (8)stock market. Detail description of the variables under each category can be found in __[appendix]( https://s3.amazonaws.com/files.fred.stlouisfed.org/fred-md/Appendix_Tables_Update.pdf)__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>RPI</th>\n",
       "      <th>W875RX1</th>\n",
       "      <th>DPCERA3M086SBEA</th>\n",
       "      <th>CMRMTSPLx</th>\n",
       "      <th>RETAILx</th>\n",
       "      <th>INDPRO</th>\n",
       "      <th>IPFPNSS</th>\n",
       "      <th>IPFINAL</th>\n",
       "      <th>IPCONGD</th>\n",
       "      <th>...</th>\n",
       "      <th>DSERRG3M086SBEA</th>\n",
       "      <th>CES0600000008</th>\n",
       "      <th>CES2000000008</th>\n",
       "      <th>CES3000000008</th>\n",
       "      <th>UMCSENTx</th>\n",
       "      <th>MZMSL</th>\n",
       "      <th>DTCOLNVHFNM</th>\n",
       "      <th>DTCTHFNM</th>\n",
       "      <th>INVEST</th>\n",
       "      <th>VXOCLSx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1/1/59</td>\n",
       "      <td>2437.296</td>\n",
       "      <td>2288.8</td>\n",
       "      <td>17.302</td>\n",
       "      <td>292258.8329</td>\n",
       "      <td>18235.77392</td>\n",
       "      <td>22.6248</td>\n",
       "      <td>23.4555</td>\n",
       "      <td>22.1893</td>\n",
       "      <td>32.4027</td>\n",
       "      <td>...</td>\n",
       "      <td>11.358</td>\n",
       "      <td>2.13</td>\n",
       "      <td>2.45</td>\n",
       "      <td>2.04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>274.9</td>\n",
       "      <td>6476.0</td>\n",
       "      <td>12298.0</td>\n",
       "      <td>84.2043</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2/1/59</td>\n",
       "      <td>2446.902</td>\n",
       "      <td>2297.0</td>\n",
       "      <td>17.482</td>\n",
       "      <td>294429.5453</td>\n",
       "      <td>18369.56308</td>\n",
       "      <td>23.0679</td>\n",
       "      <td>23.7720</td>\n",
       "      <td>22.3816</td>\n",
       "      <td>32.6404</td>\n",
       "      <td>...</td>\n",
       "      <td>11.375</td>\n",
       "      <td>2.14</td>\n",
       "      <td>2.46</td>\n",
       "      <td>2.05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>276.0</td>\n",
       "      <td>6476.0</td>\n",
       "      <td>12298.0</td>\n",
       "      <td>83.5280</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3/1/59</td>\n",
       "      <td>2462.689</td>\n",
       "      <td>2314.0</td>\n",
       "      <td>17.647</td>\n",
       "      <td>293425.3813</td>\n",
       "      <td>18523.05762</td>\n",
       "      <td>23.4002</td>\n",
       "      <td>23.9159</td>\n",
       "      <td>22.4914</td>\n",
       "      <td>32.6404</td>\n",
       "      <td>...</td>\n",
       "      <td>11.395</td>\n",
       "      <td>2.15</td>\n",
       "      <td>2.45</td>\n",
       "      <td>2.07</td>\n",
       "      <td>NaN</td>\n",
       "      <td>277.4</td>\n",
       "      <td>6508.0</td>\n",
       "      <td>12349.0</td>\n",
       "      <td>81.6405</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4/1/59</td>\n",
       "      <td>2478.744</td>\n",
       "      <td>2330.3</td>\n",
       "      <td>17.584</td>\n",
       "      <td>299331.6505</td>\n",
       "      <td>18534.46600</td>\n",
       "      <td>23.8987</td>\n",
       "      <td>24.2613</td>\n",
       "      <td>22.8210</td>\n",
       "      <td>33.1553</td>\n",
       "      <td>...</td>\n",
       "      <td>11.436</td>\n",
       "      <td>2.16</td>\n",
       "      <td>2.47</td>\n",
       "      <td>2.08</td>\n",
       "      <td>NaN</td>\n",
       "      <td>278.1</td>\n",
       "      <td>6620.0</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>81.8099</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5/1/59</td>\n",
       "      <td>2493.228</td>\n",
       "      <td>2345.8</td>\n",
       "      <td>17.796</td>\n",
       "      <td>301372.9597</td>\n",
       "      <td>18679.66354</td>\n",
       "      <td>24.2587</td>\n",
       "      <td>24.4628</td>\n",
       "      <td>23.0407</td>\n",
       "      <td>33.3137</td>\n",
       "      <td>...</td>\n",
       "      <td>11.454</td>\n",
       "      <td>2.17</td>\n",
       "      <td>2.48</td>\n",
       "      <td>2.08</td>\n",
       "      <td>95.3</td>\n",
       "      <td>280.1</td>\n",
       "      <td>6753.0</td>\n",
       "      <td>12646.0</td>\n",
       "      <td>80.7315</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 129 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Date       RPI  W875RX1  DPCERA3M086SBEA    CMRMTSPLx      RETAILx  \\\n",
       "0  1/1/59  2437.296   2288.8           17.302  292258.8329  18235.77392   \n",
       "1  2/1/59  2446.902   2297.0           17.482  294429.5453  18369.56308   \n",
       "2  3/1/59  2462.689   2314.0           17.647  293425.3813  18523.05762   \n",
       "3  4/1/59  2478.744   2330.3           17.584  299331.6505  18534.46600   \n",
       "4  5/1/59  2493.228   2345.8           17.796  301372.9597  18679.66354   \n",
       "\n",
       "    INDPRO  IPFPNSS  IPFINAL  IPCONGD  ...  DSERRG3M086SBEA  CES0600000008  \\\n",
       "0  22.6248  23.4555  22.1893  32.4027  ...           11.358           2.13   \n",
       "1  23.0679  23.7720  22.3816  32.6404  ...           11.375           2.14   \n",
       "2  23.4002  23.9159  22.4914  32.6404  ...           11.395           2.15   \n",
       "3  23.8987  24.2613  22.8210  33.1553  ...           11.436           2.16   \n",
       "4  24.2587  24.4628  23.0407  33.3137  ...           11.454           2.17   \n",
       "\n",
       "   CES2000000008  CES3000000008  UMCSENTx  MZMSL  DTCOLNVHFNM  DTCTHFNM  \\\n",
       "0           2.45           2.04       NaN  274.9       6476.0   12298.0   \n",
       "1           2.46           2.05       NaN  276.0       6476.0   12298.0   \n",
       "2           2.45           2.07       NaN  277.4       6508.0   12349.0   \n",
       "3           2.47           2.08       NaN  278.1       6620.0   12484.0   \n",
       "4           2.48           2.08      95.3  280.1       6753.0   12646.0   \n",
       "\n",
       "    INVEST  VXOCLSx  \n",
       "0  84.2043      NaN  \n",
       "1  83.5280      NaN  \n",
       "2  81.6405      NaN  \n",
       "3  81.8099      NaN  \n",
       "4  80.7315      NaN  \n",
       "\n",
       "[5 rows x 129 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bigmacro=pd.read_csv(\"Macroeconomic_Variables.csv\")\n",
    "bigmacro=bigmacro.rename(columns={'sasdate':'Date'})\n",
    "bigmacro.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consistent with the previous works in the literature, we use __[business cycle dating chronology provided by NBER](http://www.nber.org/cycles.html)__  which involves dates when recession began and ended in US economy. According to NBER's statistics we have 8 recession periods in our dataset where duration is changing from 6 to 18 months. We represent regimes as \"Normal\" and \"Recession\" in our dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Regime</th>\n",
       "      <th>RPI</th>\n",
       "      <th>W875RX1</th>\n",
       "      <th>DPCERA3M086SBEA</th>\n",
       "      <th>CMRMTSPLx</th>\n",
       "      <th>RETAILx</th>\n",
       "      <th>INDPRO</th>\n",
       "      <th>IPFPNSS</th>\n",
       "      <th>IPFINAL</th>\n",
       "      <th>...</th>\n",
       "      <th>DSERRG3M086SBEA</th>\n",
       "      <th>CES0600000008</th>\n",
       "      <th>CES2000000008</th>\n",
       "      <th>CES3000000008</th>\n",
       "      <th>UMCSENTx</th>\n",
       "      <th>MZMSL</th>\n",
       "      <th>DTCOLNVHFNM</th>\n",
       "      <th>DTCTHFNM</th>\n",
       "      <th>INVEST</th>\n",
       "      <th>VXOCLSx</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1/1/59</td>\n",
       "      <td>Normal</td>\n",
       "      <td>2437.296</td>\n",
       "      <td>2288.8</td>\n",
       "      <td>17.302</td>\n",
       "      <td>292258.8329</td>\n",
       "      <td>18235.77392</td>\n",
       "      <td>22.6248</td>\n",
       "      <td>23.4555</td>\n",
       "      <td>22.1893</td>\n",
       "      <td>...</td>\n",
       "      <td>11.358</td>\n",
       "      <td>2.13</td>\n",
       "      <td>2.45</td>\n",
       "      <td>2.04</td>\n",
       "      <td>NaN</td>\n",
       "      <td>274.9</td>\n",
       "      <td>6476.0</td>\n",
       "      <td>12298.0</td>\n",
       "      <td>84.2043</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2/1/59</td>\n",
       "      <td>Normal</td>\n",
       "      <td>2446.902</td>\n",
       "      <td>2297.0</td>\n",
       "      <td>17.482</td>\n",
       "      <td>294429.5453</td>\n",
       "      <td>18369.56308</td>\n",
       "      <td>23.0679</td>\n",
       "      <td>23.7720</td>\n",
       "      <td>22.3816</td>\n",
       "      <td>...</td>\n",
       "      <td>11.375</td>\n",
       "      <td>2.14</td>\n",
       "      <td>2.46</td>\n",
       "      <td>2.05</td>\n",
       "      <td>NaN</td>\n",
       "      <td>276.0</td>\n",
       "      <td>6476.0</td>\n",
       "      <td>12298.0</td>\n",
       "      <td>83.5280</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3/1/59</td>\n",
       "      <td>Normal</td>\n",
       "      <td>2462.689</td>\n",
       "      <td>2314.0</td>\n",
       "      <td>17.647</td>\n",
       "      <td>293425.3813</td>\n",
       "      <td>18523.05762</td>\n",
       "      <td>23.4002</td>\n",
       "      <td>23.9159</td>\n",
       "      <td>22.4914</td>\n",
       "      <td>...</td>\n",
       "      <td>11.395</td>\n",
       "      <td>2.15</td>\n",
       "      <td>2.45</td>\n",
       "      <td>2.07</td>\n",
       "      <td>NaN</td>\n",
       "      <td>277.4</td>\n",
       "      <td>6508.0</td>\n",
       "      <td>12349.0</td>\n",
       "      <td>81.6405</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4/1/59</td>\n",
       "      <td>Normal</td>\n",
       "      <td>2478.744</td>\n",
       "      <td>2330.3</td>\n",
       "      <td>17.584</td>\n",
       "      <td>299331.6505</td>\n",
       "      <td>18534.46600</td>\n",
       "      <td>23.8987</td>\n",
       "      <td>24.2613</td>\n",
       "      <td>22.8210</td>\n",
       "      <td>...</td>\n",
       "      <td>11.436</td>\n",
       "      <td>2.16</td>\n",
       "      <td>2.47</td>\n",
       "      <td>2.08</td>\n",
       "      <td>NaN</td>\n",
       "      <td>278.1</td>\n",
       "      <td>6620.0</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>81.8099</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5/1/59</td>\n",
       "      <td>Normal</td>\n",
       "      <td>2493.228</td>\n",
       "      <td>2345.8</td>\n",
       "      <td>17.796</td>\n",
       "      <td>301372.9597</td>\n",
       "      <td>18679.66354</td>\n",
       "      <td>24.2587</td>\n",
       "      <td>24.4628</td>\n",
       "      <td>23.0407</td>\n",
       "      <td>...</td>\n",
       "      <td>11.454</td>\n",
       "      <td>2.17</td>\n",
       "      <td>2.48</td>\n",
       "      <td>2.08</td>\n",
       "      <td>95.3</td>\n",
       "      <td>280.1</td>\n",
       "      <td>6753.0</td>\n",
       "      <td>12646.0</td>\n",
       "      <td>80.7315</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 130 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Date  Regime       RPI  W875RX1  DPCERA3M086SBEA    CMRMTSPLx  \\\n",
       "0  1/1/59  Normal  2437.296   2288.8           17.302  292258.8329   \n",
       "1  2/1/59  Normal  2446.902   2297.0           17.482  294429.5453   \n",
       "2  3/1/59  Normal  2462.689   2314.0           17.647  293425.3813   \n",
       "3  4/1/59  Normal  2478.744   2330.3           17.584  299331.6505   \n",
       "4  5/1/59  Normal  2493.228   2345.8           17.796  301372.9597   \n",
       "\n",
       "       RETAILx   INDPRO  IPFPNSS  IPFINAL  ...  DSERRG3M086SBEA  \\\n",
       "0  18235.77392  22.6248  23.4555  22.1893  ...           11.358   \n",
       "1  18369.56308  23.0679  23.7720  22.3816  ...           11.375   \n",
       "2  18523.05762  23.4002  23.9159  22.4914  ...           11.395   \n",
       "3  18534.46600  23.8987  24.2613  22.8210  ...           11.436   \n",
       "4  18679.66354  24.2587  24.4628  23.0407  ...           11.454   \n",
       "\n",
       "   CES0600000008  CES2000000008  CES3000000008  UMCSENTx  MZMSL  DTCOLNVHFNM  \\\n",
       "0           2.13           2.45           2.04       NaN  274.9       6476.0   \n",
       "1           2.14           2.46           2.05       NaN  276.0       6476.0   \n",
       "2           2.15           2.45           2.07       NaN  277.4       6508.0   \n",
       "3           2.16           2.47           2.08       NaN  278.1       6620.0   \n",
       "4           2.17           2.48           2.08      95.3  280.1       6753.0   \n",
       "\n",
       "   DTCTHFNM   INVEST  VXOCLSx  \n",
       "0   12298.0  84.2043      NaN  \n",
       "1   12298.0  83.5280      NaN  \n",
       "2   12349.0  81.6405      NaN  \n",
       "3   12484.0  81.8099      NaN  \n",
       "4   12646.0  80.7315      NaN  \n",
       "\n",
       "[5 rows x 130 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Recession_periods=pd.read_csv('Recession_Periods.csv')\n",
    "bigmacro.insert(loc=1,column=\"Regime\", value=Recession_periods['Regime'].values)\n",
    "bigmacro.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Regime</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Normal</th>\n",
       "      <td>628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Recession</th>\n",
       "      <td>93</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           Date\n",
       "Regime         \n",
       "Normal      628\n",
       "Recession    93"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bigmacro[[\"Date\",\"Regime\"]].groupby(\"Regime\").count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Explanatory Data Analysis <a id=\"2.2\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at performance of stock market and economy over normal and recession periods. Below you can see the performance of SP500 and GDP over the period 1959-2018. Gray areas indicate recession periods in our dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. References <a id=\"3\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. **W. Mitchell and A. Burns.** \"Statistical Indicators of Cyclical Revivals\", _National Bureau of Economic Research,_ 1938. <a class=\"anchor\" id=\"a\"></a>\n",
    "\n",
    "2. **J. H. Stock and M. W. Watson.** \"New Indexes of Coincedent and Leading Economic Indicators\", _NBER Macroeconomics Annual,_ 1989. <a class=\"anchor\" id=\"b\"></a> \n",
    "\n",
    "3. **A. Estrella and F. S. Mishkin.** \"Predicting U.S. Recessions: Financial Variables as Leading Indicators\", _Review of Economics and Statistics,_ 1998. <a class=\"anchor\" id=\"c\"></a>\n",
    "\n",
    "4. **D. Chionis, P. Gogas and I. Pragidis.** \"Predicting Euroepan Union Recessions in the Euro Era: the Yield Curve as a Forecasting Tool of Economic Activity\", _International Advances in Economic Research,_ 2009. <a class=\"anchor\" id=\"d\"></a>\n",
    "\n",
    "5. **S. Ng.** \"Viewpoint: Boosting Recessions\", _Canadian Journal of Economics,_ 2014. <a class=\"anchor\" id=\"e\"></a>\n",
    "\n",
    "6. **W. Liu and E. Moench** \"What Predicts US Recessions?\", _International Journal of Forecasting,_ 2016. <a class=\"anchor\" id=\"f\"></a>\n",
    "\n",
    "7. **M. Y. Huang, R. R. Rojas and P. D. Convery.** \" News Sentiment as Leading Indicators for Recessions\", _arXiv,_ 2018. <a id=\"g\"></a>\n",
    "\n",
    "8. **M. McCracken and S. Ng** \"__[FRED-MD: A Monthly Database for Macroeconomic Research](https://research.stlouisfed.org/econ/mccracken/fred-databases/)__\", _Working Paper,_ 2015.  <a class=\"anchor\" id=\"i\"></a>\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}