SubmissionEDA_predictionCatboost.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](amex.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AmExpert 2018 (Machine Learning Hackathon)\n",
    "**-Analytics Vidhya**\n",
    "\n",
    "## Problem Statement\n",
    "\n",
    "Recent years have witnessed a surge in the number of internet savvy users. Companies in the financial services domain leverage this huge internet traffic arriving at their interface by strategically placing ads/promotions for cross selling of various financial products on a plethora of web pages. The digital analytics unit of Best Cards Company uses cutting edge data science and machine learning for successful promotion of its valuable card products. They believe that a predictive model that forecasts whether a session involves a click on the ad/promotion would help them extract the maximum out of the huge clickstream data that they have collected. You are hired as a consultant to build an efficient model to predict whether a user will click on an ad or not, given the following features:\n",
    "\n",
    "* **Clickstream data/train data for duration: (2nd July 2017 – 7th July 2017)**\n",
    "* **Test data for duration: (8th July 2017 – 9th July 2017)**\n",
    "* **User features (demographics, user behaviour/activity, buying power etc.)**\n",
    "* **Historical transactional data of the previous month with timestamp info (28th May 2017– 1st July 2017). This data contains * actions (views/interest registered) taken by the user historically on the product page via an ad or other sources**\n",
    "* **Ad features (product category, webpage, campaign for ad etc.)**\n",
    "* **Date time features (exact timestamp of the user session)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from matplotlib.pylab import rcParams\n",
    "rcParams['figure.figsize'] = 12, 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 249,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>session_id</th>\n",
       "      <th>DateTime</th>\n",
       "      <th>user_id</th>\n",
       "      <th>product</th>\n",
       "      <th>campaign_id</th>\n",
       "      <th>webpage_id</th>\n",
       "      <th>product_category_1</th>\n",
       "      <th>product_category_2</th>\n",
       "      <th>user_group_id</th>\n",
       "      <th>gender</th>\n",
       "      <th>age_level</th>\n",
       "      <th>user_depth</th>\n",
       "      <th>city_development_index</th>\n",
       "      <th>var_1</th>\n",
       "      <th>is_click</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>140690</td>\n",
       "      <td>2017-07-02 00:00</td>\n",
       "      <td>858557</td>\n",
       "      <td>C</td>\n",
       "      <td>359520</td>\n",
       "      <td>13787</td>\n",
       "      <td>4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.0</td>\n",
       "      <td>Female</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>333291</td>\n",
       "      <td>2017-07-02 00:00</td>\n",
       "      <td>243253</td>\n",
       "      <td>C</td>\n",
       "      <td>105960</td>\n",
       "      <td>11085</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Female</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>129781</td>\n",
       "      <td>2017-07-02 00:00</td>\n",
       "      <td>243253</td>\n",
       "      <td>C</td>\n",
       "      <td>359520</td>\n",
       "      <td>13787</td>\n",
       "      <td>4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Female</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>464848</td>\n",
       "      <td>2017-07-02 00:00</td>\n",
       "      <td>1097446</td>\n",
       "      <td>I</td>\n",
       "      <td>359520</td>\n",
       "      <td>13787</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>Male</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>90569</td>\n",
       "      <td>2017-07-02 00:01</td>\n",
       "      <td>663656</td>\n",
       "      <td>C</td>\n",
       "      <td>405490</td>\n",
       "      <td>60305</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Male</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   session_id          DateTime  user_id product  campaign_id  webpage_id  \\\n",
       "0      140690  2017-07-02 00:00   858557       C       359520       13787   \n",
       "1      333291  2017-07-02 00:00   243253       C       105960       11085   \n",
       "2      129781  2017-07-02 00:00   243253       C       359520       13787   \n",
       "3      464848  2017-07-02 00:00  1097446       I       359520       13787   \n",
       "4       90569  2017-07-02 00:01   663656       C       405490       60305   \n",
       "\n",
       "   product_category_1  product_category_2  user_group_id  gender  age_level  \\\n",
       "0                   4                 NaN           10.0  Female        4.0   \n",
       "1                   5                 NaN            8.0  Female        2.0   \n",
       "2                   4                 NaN            8.0  Female        2.0   \n",
       "3                   3                 NaN            3.0    Male        3.0   \n",
       "4                   3                 NaN            2.0    Male        2.0   \n",
       "\n",
       "   user_depth  city_development_index  var_1  is_click  \n",
       "0         3.0                     3.0      0         0  \n",
       "1         2.0                     NaN      0         0  \n",
       "2         2.0                     NaN      0         0  \n",
       "3         3.0                     2.0      1         0  \n",
       "4         3.0                     2.0      1         0  "
      ]
     },
     "execution_count": 249,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train=pd.read_csv('train.csv')\n",
    "test=pd.read_csv('test.csv')\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets look at the historical data given and see what can we do with it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Historical Log Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>DateTime</th>\n",
       "      <th>user_id</th>\n",
       "      <th>product</th>\n",
       "      <th>action</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-05-28 15:44</td>\n",
       "      <td>704</td>\n",
       "      <td>B</td>\n",
       "      <td>view</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-05-29 07:08</td>\n",
       "      <td>499679</td>\n",
       "      <td>F</td>\n",
       "      <td>view</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-05-29 07:10</td>\n",
       "      <td>499679</td>\n",
       "      <td>G</td>\n",
       "      <td>view</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-05-29 07:10</td>\n",
       "      <td>499679</td>\n",
       "      <td>G</td>\n",
       "      <td>view</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-05-29 07:10</td>\n",
       "      <td>499679</td>\n",
       "      <td>G</td>\n",
       "      <td>view</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           DateTime  user_id product action\n",
       "0  2017-05-28 15:44      704       B   view\n",
       "1  2017-05-29 07:08   499679       F   view\n",
       "2  2017-05-29 07:10   499679       G   view\n",
       "3  2017-05-29 07:10   499679       G   view\n",
       "4  2017-05-29 07:10   499679       G   view"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log=pd.read_csv('historical_user_logs.csv')\n",
    "log.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grouping the historical log data using and extracting features\n",
    "\n",
    "* user and product view count\n",
    "* user and product interest count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>product</th>\n",
       "      <th>action</th>\n",
       "      <th>count_view</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>A</td>\n",
       "      <td>view</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>B</td>\n",
       "      <td>view</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>H</td>\n",
       "      <td>view</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>I</td>\n",
       "      <td>view</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19</td>\n",
       "      <td>A</td>\n",
       "      <td>view</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id product action  count_view\n",
       "0        4       A   view           3\n",
       "1        4       B   view           1\n",
       "2        4       H   view           1\n",
       "3        4       I   view           2\n",
       "4       19       A   view          14"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_view=log[log.action=='view'].groupby(by=['user_id','product','action']).count().rename(columns={'DateTime':'count_view','action':'action_view'}).reset_index()\n",
    "user_view.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>product</th>\n",
       "      <th>action</th>\n",
       "      <th>count_interest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19</td>\n",
       "      <td>D</td>\n",
       "      <td>interest</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19</td>\n",
       "      <td>F</td>\n",
       "      <td>interest</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>19</td>\n",
       "      <td>G</td>\n",
       "      <td>interest</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>26</td>\n",
       "      <td>H</td>\n",
       "      <td>interest</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>30</td>\n",
       "      <td>H</td>\n",
       "      <td>interest</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id product    action  count_interest\n",
       "0       19       D  interest               1\n",
       "1       19       F  interest               2\n",
       "2       19       G  interest               2\n",
       "3       26       H  interest               1\n",
       "4       30       H  interest               1"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "user_interest=log[log.action=='interest'].groupby(by=['user_id','product','action']).count().rename(columns={'DateTime':'count_interest','action':'action_interest'}).reset_index()\n",
    "user_interest.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combining the dataset on train data therefore getting the count of each datasets\n",
    "\n",
    "**Train Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 463291 entries, 0 to 463290\n",
      "Data columns (total 17 columns):\n",
      "session_id                463291 non-null int64\n",
      "DateTime                  463291 non-null object\n",
      "user_id                   463291 non-null int64\n",
      "product                   463291 non-null object\n",
      "campaign_id               463291 non-null int64\n",
      "webpage_id                463291 non-null int64\n",
      "product_category_1        463291 non-null int64\n",
      "product_category_2        97437 non-null float64\n",
      "user_group_id             445048 non-null float64\n",
      "gender                    445048 non-null object\n",
      "age_level                 445048 non-null float64\n",
      "user_depth                445048 non-null float64\n",
      "city_development_index    338162 non-null float64\n",
      "var_1                     463291 non-null int64\n",
      "is_click                  463291 non-null int64\n",
      "count_view                376828 non-null float64\n",
      "count_interest            81212 non-null float64\n",
      "dtypes: float64(7), int64(7), object(3)\n",
      "memory usage: 63.6+ MB\n"
     ]
    }
   ],
   "source": [
    "train_log=pd.merge(train,user_view,on=['user_id','product'],how='left')\n",
    "train_log=train_log.merge(user_interest,on=['user_id','product'],how='left')\n",
    "train_log.drop(['action_x','action_y'],axis=1,inplace=True)\n",
    "train_log.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Test Data**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 128858 entries, 0 to 128857\n",
      "Data columns (total 16 columns):\n",
      "session_id                128858 non-null int64\n",
      "DateTime                  128858 non-null object\n",
      "user_id                   128858 non-null int64\n",
      "product                   128858 non-null object\n",
      "campaign_id               128858 non-null int64\n",
      "webpage_id                128858 non-null int64\n",
      "product_category_1        128858 non-null int64\n",
      "product_category_2        52687 non-null float64\n",
      "user_group_id             123174 non-null float64\n",
      "gender                    123174 non-null object\n",
      "age_level                 123174 non-null float64\n",
      "user_depth                123174 non-null float64\n",
      "city_development_index    94249 non-null float64\n",
      "var_1                     128858 non-null int64\n",
      "count_view                101482 non-null float64\n",
      "count_interest            21889 non-null float64\n",
      "dtypes: float64(7), int64(6), object(3)\n",
      "memory usage: 16.7+ MB\n"
     ]
    }
   ],
   "source": [
    "test_log=pd.merge(test,user_view,on=['user_id','product'],how='left')\n",
    "test_log=test_log.merge(user_interest,on=['user_id','product'],how='left')\n",
    "test_log.drop(['action_x','action_y'],axis=1,inplace=True)\n",
    "test_log.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EDA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2eecc00d8d0>"
      ]
     },
     "execution_count": 266,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtwAAAHSCAYAAADfSxyEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAE7ZJREFUeJzt3W+Iped53/HfFW2UGkIsx9oYZ1fp\nCrylkQPNn0UW5E2wg7VKStcvYpAptTACQZAhIYVa7hvnTw32mzgYHIOoRORSooi0YOEqEUK2CaWJ\nrXXj2pWFq62SWItMtI5k1SHErpyrL+ZWMhnPv93updlRPh8Y5pzruc+5n4Fl+XJ45pnq7gAAADO+\n66BPAAAAXskENwAADBLcAAAwSHADAMAgwQ0AAIMENwAADBLcAAAwSHADAMAgwQ0AAIMENwAADDpy\n0CdwuV177bV94sSJgz4NAABewT73uc99rbuP7mftKy64T5w4kbNnzx70aQAA8ApWVX+237UuKQEA\ngEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgB\nAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABh056BPgynbirv9y0KcA2/rTD/zsQZ8C\nAOyLT7gBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGC\nGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ\n4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEH7Du6quqqq/riqPrGeX19Vn6mqJ6vqd6rq6jX/nvX8\n3Dp+YtN7vHfNv1xVN2+an16zc1V116b5tnsAAMBhcTGfcP9Ckic2Pf9gkg9198kkzye5fc1vT/J8\nd78hyYfWulTVDUluTfLGJKeT/OaK+KuSfCTJLUluSPKOtXa3PQAA4FDYV3BX1fEkP5vk36/nleTN\nSX53LbkvydvW4zPredbxt6z1Z5Lc393f7O4/SXIuyY3r61x3P9Xd30pyf5Ize+wBAACHwn4/4f6N\nJP8myd+s569N8vXufnE9P5/k2Hp8LMnTSbKOv7DW/+18y2t2mu+2BwAAHAp7BndV/fMkz3b35zaP\nt1naexy7XPPtzvGOqjpbVWcvXLiw3RIAADgQ+/mE+yeT/Iuq+tNsXO7x5mx84n1NVR1Za44neWY9\nPp/kuiRZx1+d5LnN8y2v2Wn+tV32+Hu6++7uPtXdp44ePbqPHwkAAF4eewZ3d7+3u49394ls/NLj\nJ7v7Xyb5VJKfW8tuS/Lx9fjB9Tzr+Ce7u9f81nUXk+uTnEzy2SSPJTm57khy9drjwfWanfYAAIBD\n4f/nPtzvSfJLVXUuG9db37Pm9yR57Zr/UpK7kqS7H0/yQJIvJfn9JHd297fXNdrvTvJwNu6C8sBa\nu9seAABwKBzZe8nf6e5PJ/n0evxUNu4wsnXNXyd5+w6vf3+S928zfyjJQ9vMt90DAAAOC39pEgAA\nBgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYA\ngEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgB\nAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABglu\nAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGC\nGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ\n4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAY\nJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAA\nBgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGDQnsFd\nVf+oqj5bVf+jqh6vql9Z8+ur6jNV9WRV/U5VXb3m37Oen1vHT2x6r/eu+Zer6uZN89Nrdq6q7to0\n33YPAAA4LPbzCfc3k7y5u/9Zkh9NcrqqbkrywSQf6u6TSZ5Pcvtaf3uS57v7DUk+tNalqm5IcmuS\nNyY5neQ3q+qqqroqyUeS3JLkhiTvWGuzyx4AAHAo7BncveEv19PvXl+d5M1JfnfN70vytvX4zHqe\ndfwtVVVrfn93f7O7/yTJuSQ3rq9z3f1Ud38ryf1JzqzX7LQHAAAcCvu6hnt9Ev35JM8meSTJ/07y\n9e5+cS05n+TYenwsydNJso6/kOS1m+dbXrPT/LW77AEAAIfCvoK7u7/d3T+a5Hg2PpH+4e2Wre+1\nw7HLNf8OVXVHVZ2tqrMXLlzYbgkAAByIi7pLSXd/Pcmnk9yU5JqqOrIOHU/yzHp8Psl1SbKOvzrJ\nc5vnW16z0/xru+yx9bzu7u5T3X3q6NGjF/MjAQDAqP3cpeRoVV2zHr8qyU8neSLJp5L83Fp2W5KP\nr8cPrudZxz/Z3b3mt667mFyf5GSSzyZ5LMnJdUeSq7Pxi5UPrtfstAcAABwKR/ZektcnuW/dTeS7\nkjzQ3Z+oqi8lub+q/l2SP05yz1p/T5L/UFXnsvHJ9q1J0t2PV9UDSb6U5MUkd3b3t5Okqt6d5OEk\nVyW5t7sfX+/1nh32AACAQ2HP4O7uLyT5sW3mT2Xjeu6t879O8vYd3uv9Sd6/zfyhJA/tdw8AADgs\n/KVJAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYA\ngEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgB\nAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABglu\nAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGC\nGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ\n4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAY\nJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAA\nBgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYA\ngEF7BndVXVdVn6qqJ6rq8ar6hTX//qp6pKqeXN9fs+ZVVR+uqnNV9YWq+vFN73XbWv9kVd22af4T\nVfXF9ZoPV1XttgcAABwW+/mE+8Uk/7q7fzjJTUnurKobktyV5NHuPpnk0fU8SW5JcnJ93ZHko8lG\nPCd5X5I3Jbkxyfs2BfRH19qXXnd6zXfaAwAADoU9g7u7v9rd/309/kaSJ5IcS3ImyX1r2X1J3rYe\nn0nysd7wR0muqarXJ7k5ySPd/Vx3P5/kkSSn17Hv6+4/7O5O8rEt77XdHgAAcChc1DXcVXUiyY8l\n+UyS13X3V5ONKE/yA2vZsSRPb3rZ+TXbbX5+m3l22QMAAA6FfQd3VX1vkv+U5Be7+//stnSbWV/C\nfN+q6o6qOltVZy9cuHAxLwUAgFH7Cu6q+u5sxPZ/7O7/vMZ/vi4Hyfr+7JqfT3LdppcfT/LMHvPj\n28x32+Pv6e67u/tUd586evTofn4kAAB4WeznLiWV5J4kT3T3r2869GCSl+40cluSj2+av3PdreSm\nJC+sy0EeTvLWqnrN+mXJtyZ5eB37RlXdtPZ655b32m4PAAA4FI7sY81PJvlXSb5YVZ9fs3+b5ANJ\nHqiq25N8Jcnb17GHkvxMknNJ/irJu5Kku5+rql9L8tha96vd/dx6/PNJfivJq5L83vrKLnsAAMCh\nsGdwd/d/zfbXWSfJW7ZZ30nu3OG97k1y7zbzs0l+ZJv5X2y3BwAAHBb+0iQAAAwS3AAAMEhwAwDA\nIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAA\nMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcA\nAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMEN\nAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhw\nAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS\n3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACD\nBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDA\nIMENAACDBDcAAAwS3AAAMEhwAwDAIMENAACDBDcAAAwS3AAAMEhwAwDAoD2Du6rurapnq+p/bpp9\nf1U9UlVPru+vWfOqqg9X1bmq+kJV/fim19y21j9ZVbdtmv9EVX1xvebDVVW77QEAAIfJfj7h/q0k\np7fM7kryaHefTPLoep4ktyQ5ub7uSPLRZCOek7wvyZuS3JjkfZsC+qNr7UuvO73HHgAAcGjsGdzd\n/QdJntsyPpPkvvX4viRv2zT/WG/4oyTXVNXrk9yc5JHufq67n0/ySJLT69j3dfcfdncn+diW99pu\nDwAAODQu9Rru13X3V5Nkff+BNT+W5OlN686v2W7z89vMd9sDAAAOjcv9S5O1zawvYX5xm1bdUVVn\nq+rshQsXLvblAAAw5lKD+8/X5SBZ359d8/NJrtu07niSZ/aYH99mvtse36G77+7uU9196ujRo5f4\nIwEAwOV3qcH9YJKX7jRyW5KPb5q/c92t5KYkL6zLQR5O8taqes36Zcm3Jnl4HftGVd207k7yzi3v\ntd0eAABwaBzZa0FV/XaSn0pybVWdz8bdRj6Q5IGquj3JV5K8fS1/KMnPJDmX5K+SvCtJuvu5qvq1\nJI+tdb/a3S/9IubPZ+NOKK9K8nvrK7vsAQAAh8aewd3d79jh0Fu2WdtJ7tzhfe5Ncu8287NJfmSb\n+V9stwcAABwm/tIkAAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAM\nEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAA\ngwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMA\nwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwA\nADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAgwQ3\nAAAMEtwAADBIcAMAwCDBDQAAgwQ3AAAMEtwAADBIcAMAwCDBDQAAg44c9AkAwCvKL7/6oM8AdvbL\nLxz0GfyD5BNuAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgB\nAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGCQ4AYAgEGCGwAABglu\nAAAYJLgBAGCQ4AYAgEGCGwAABgluAAAYJLgBAGDQFR/cVXW6qr5cVeeq6q6DPh8AALgYV3RwV9VV\nST6S5JYkNyR5R1XdcLBnBQAA+3dFB3eSG5Oc6+6nuvtbSe5PcuaAzwkAAPbtyEGfwB6OJXl60/Pz\nSd60dVFV3ZHkjvX0L6vqyy/DucHFujbJ1w76JF4p6oMHfQbAy8T/nZfTr9RBn8EryT/e78IrPbi3\n+1fR3zHovjvJ3fOnA5euqs5296mDPg+Aw8T/nbwSXOmXlJxPct2m58eTPHNA5wIAABftSg/ux5Kc\nrKrrq+rqJLcmefCAzwkAAPbtir6kpLtfrKp3J3k4yVVJ7u3uxw/4tOBSuewJ4OL5v5NDr7q/45Jo\nAADgMrnSLykBAIBDTXADAMAgwQ0AAIOu6F+ahMOsqv5pNv4y6rFs3D/+mSQPdvcTB3piAMDLyifc\nMKCq3pPk/mz88abPZuMWl5Xkt6vqroM8N4DDqKreddDnAJfKXUpgQFX9ryRv7O7/u2V+dZLHu/vk\nwZwZwOFUVV/p7h866POAS+GSEpjxN0l+MMmfbZm/fh0DYIuq+sJOh5K87uU8F7icBDfM+MUkj1bV\nk0meXrMfSvKGJO8+sLMCuLK9LsnNSZ7fMq8k/+3lPx24PAQ3DOju36+qf5Lkxmz80mQlOZ/kse7+\n9oGeHMCV6xNJvre7P7/1QFV9+uU/Hbg8XMMNAACD3KUEAAAGCW4AABgkuAEAYJDgBgCAQYIbAAAG\n/T9MvLSghsEbWAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train.is_click.value_counts().plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Too much imbalanced data, maybe sampling techniques can be used but the bar is too high for it i think"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Acer\\Anaconda3\\lib\\site-packages\\scipy\\stats\\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtsAAAH+CAYAAABN+cKwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2w3nV95//X2wSIFcVtjDdNSJMO\nbLfxBkojqHVpt1SF7U20i2OoVXdkTJkVO7/fTpel2ymr/HZnF/Un023pDbPwE9NWWGlr45YV3bJq\nu64swZu1kTKNqCUgs+GmFGEDBN6/P86FczieSALXJ1dOeDxmmHNd3+tzfc/7ZODMk2++1/db3R0A\nAGD6njHrAQAA4HAltgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMsnzW\nA0zT8573vF63bt2sxwAA4DB344033tndq55o3WEV2+vWrcv27dtnPQYAAIe5qvrG/qxzGgkAAAwi\ntgEAYBCxDQAAgxxW52wDAHDgHn744ezatSt79uyZ9SiHnBUrVmTNmjU54ogjntT7xTYAwNPcrl27\n8uxnPzvr1q1LVc16nENGd+euu+7Krl27sn79+ie1D6eRAAA8ze3ZsycrV64U2gtUVVauXPmUjviL\nbQAAhPY+PNU/F7ENAACDDD9nu6pOT/LrSZYl+Y/d/e8XvH5Ukg8l+ZEkdyV5U3d/varenORfzFv6\nsiQndfcXR88MAPB0tu78P53q/r7+739qqvtbSoYe2a6qZUkuSXJGkg1JzqqqDQuWnZ3knu4+LsnF\nSS5Kku7+/e4+sbtPTPKWJF8X2gAAh6dXvepVU9nPBz/4wZx77rlJkt/5nd/Jhz70oX2u/dSnPpWf\n/umfnsr33ZfRR7ZPTrKzu29Jkqq6MsmmJF+Zt2ZTkndPHl+d5Derqrq75605K8mHB88KAMCMfPaz\nn536Ps8555yp7/NAjT5ne3WSW+c93zXZtuia7t6b5N4kKxeseVP2EdtVtaWqtlfV9t27d09laAAA\nDq6jjz46SfLNb34zp556ak488cS85CUvyZ//+Z/v8z0f//jHc9JJJ+WEE07Iaaed9h2vv/vd7877\n3//+JMnOnTvzkz/5kznhhBNy0kkn5atf/erj1t5www354R/+4dxyyy1T/KnGH9le7OObfSBrquqU\nJA90918u9g26+9IklybJxo0bF+4bAIAl5A/+4A/yute9Lr/6q7+aRx55JA888MCi63bv3p13vOMd\n+cxnPpP169fn7rvv/q77ffOb35zzzz8/b3jDG7Jnz548+uijufXWuWPCn/3sZ/Oud70rf/Inf5K1\na9dO9ecZHdu7khw77/maJLfvY82uqlqe5Jgk8/+0NscpJAAATwsvf/nL8/a3vz0PP/xwXv/61+fE\nE09cdN3nPve5nHrqqd++2cz3fu/37nOf9913X2677ba84Q1vSDJ3V8jH3HTTTdmyZUs+8YlP5Pu+\n7/um+JPMGX0ayQ1Jjq+q9VV1ZObCeduCNduSvG3y+Mwk1z12vnZVPSPJG5NcOXhOAAAOAaeeemo+\n85nPZPXq1XnLW96yzw84dvd+XwP78R8FfLwXvehFWbFiRb7whS88qXmfyNAj2929t6rOTXJt5i79\nd3l376iqC5Ns7+5tSS5LsrWqdmbuiPbmebs4Ncmuxz5gCQDAeLO8VN83vvGNrF69Ou94xzty//33\n5/Of/3ze+ta3fse6V77ylXnnO9+Zr33ta98+jWRfR7ef85znZM2aNfnoRz+a17/+9XnwwQfzyCOP\nJEme+9zn5rLLLstrX/vaPOtZz8qP//iPT/XnGX6d7e6+Jsk1C7ZdMO/xnswdvV7svZ9K8oqR8wEA\ncOj41Kc+lfe973054ogjcvTRR+/zyPaqVaty6aWX5ud+7ufy6KOP5vnPf34++clP7nO/W7duzS/+\n4i/mggsuyBFHHJGPfOQj337tBS94QT72sY/ljDPOyOWXX55TTjllaj9PfbfD6kvNxo0be/v27bMe\nAwBgSbnpppvyQz/0Q7Me45C12J9PVd3Y3Ruf6L3Dj2wDAByo8847L3fccUde+MIX5r3vfe+sx4En\nTWwDAIecO+64I7fddtusx+AQccopp+TBBx983LatW7fmpS996Ywm2n9iGwCAA7q6x8F2/fXXz+x7\nP9VTrkdf+g8AgEPcihUrctdddz3lsDzcdHfuuuuux12X+0A5sg0A8DS3Zs2a7Nq1K7t37571KIec\nFStWZM2aNU/6/WIbAOBp7ogjjvj2nRiZLrENwCHhby489D/oxMGz9+7vTbI8e+/+hn83eJy1F3x5\n1iMcEOdsAwDAIGIbAAAGEdsAADCI2AYAgEF8QBIAOOQ8b8WjSfZOvsLSJbYBgEPOL7/sb2c9AkyF\n2IaD4Lzzzssdd9yRF77whXnve98763EAgINEbMNBcMcdd+S2226b9RgAwEHmA5IAADCI2AYAgEHE\nNgAADCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYJDlsx7g\ncPQj/+JDsx6BQ8yz77wvy5L8zZ33+feDx7nxfW+d9QgADOTINgAADCK2AQBgELENAACDiG0AABhE\nbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYZPmsB4Cng0ePfNbjvgIA\nTw9iGw6C+49/7axHAABmwGkkAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsA\nAAYZHttVdXpV3VxVO6vq/EVeP6qqrpq8fn1VrZv32suq6n9U1Y6q+nJVrRg9LwAATMvQ2K6qZUku\nSXJGkg1JzqqqDQuWnZ3knu4+LsnFSS6avHd5kt9Lck53vzjJjyd5eOS8AAAwTaOPbJ+cZGd339Ld\nDyW5MsmmBWs2Jbli8vjqJKdVVSV5bZL/1d1fSpLuvqu7Hxk8LwAATM3o2F6d5NZ5z3dNti26prv3\nJrk3ycokfz9JV9W1VfX5qjpv8KwAADBVywfvvxbZ1vu5ZnmSVyd5eZIHkvxZVd3Y3X/2uDdXbUmy\nJUnWrl37lAcGAIBpGX1ke1eSY+c9X5Pk9n2tmZynfUySuyfbP93dd3b3A0muSXLSwm/Q3Zd298bu\n3rhq1aoBPwIAADw5o2P7hiTHV9X6qjoyyeYk2xas2ZbkbZPHZya5rrs7ybVJXlZV3zOJ8B9L8pXB\n8wIAwNQMPY2ku/dW1bmZC+dlSS7v7h1VdWGS7d29LcllSbZW1c7MHdHePHnvPVX1gcwFeye5prv/\ndOS8AAAwTaPP2U53X5O5U0Dmb7tg3uM9Sd64j/f+XuYu/wcAAEuOO0gCAMAgYhsAAAYR2wAAMIjY\nBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAADCK2AQBg\nELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwD\nAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI\n2AYAgEHENgAADCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEA\nYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMMjw2K6q06vq5qraWVXnL/L6UVV11eT166tq3WT7\nuqr6P1X1xck/vzN6VgAAmKblI3deVcuSXJLkNUl2JbmhqrZ191fmLTs7yT3dfVxVbU5yUZI3TV77\nanefOHJGAAAYZfSR7ZOT7OzuW7r7oSRXJtm0YM2mJFdMHl+d5LSqqsFzAQDAcKNje3WSW+c93zXZ\ntuia7t6b5N4kKyevra+qL1TVp6vqHw6eFQAApmroaSRJFjtC3fu55ptJ1nb3XVX1I0k+WlUv7u6/\ne9ybq7Yk2ZIka9euncLIAAAwHaOPbO9Kcuy852uS3L6vNVW1PMkxSe7u7ge7+64k6e4bk3w1yd9f\n+A26+9Lu3tjdG1etWjXgRwAAgCdndGzfkOT4qlpfVUcm2Zxk24I125K8bfL4zCTXdXdX1arJByxT\nVT+Q5PgktwyeFwAApmboaSTdvbeqzk1ybZJlSS7v7h1VdWGS7d29LcllSbZW1c4kd2cuyJPk1CQX\nVtXeJI8kOae77x45LwAATNPoc7bT3dckuWbBtgvmPd6T5I2LvO8Pk/zh6PkAAGAUd5AEAIBBxDYA\nAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOI\nbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAADCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAA\nBhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2\nAAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAADCK2AQBgELENAACD\niG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYJDhsV1Vp1fVzVW1s6rOX+T1o6rq\nqsnr11fVugWvr62qb1XVL4+eFQAApmlobFfVsiSXJDkjyYYkZ1XVhgXLzk5yT3cfl+TiJBcteP3i\nJP9l5JwAADDC6CPbJyfZ2d23dPdDSa5MsmnBmk1Jrpg8vjrJaVVVSVJVr09yS5Idg+cEAICpGx3b\nq5PcOu/5rsm2Rdd0994k9yZZWVXPSvIvk7xn8IwAADDE6NiuRbb1fq55T5KLu/tb3/UbVG2pqu1V\ntX337t1PckwAAJi+5YP3vyvJsfOer0ly+z7W7Kqq5UmOSXJ3klOSnFlV703y3CSPVtWe7v7N+W/u\n7kuTXJokGzduXBjyAAAwM6Nj+4Ykx1fV+iS3Jdmc5OcXrNmW5G1J/keSM5Nc192d5B8+tqCq3p3k\nWwtDGwAADmVDY7u791bVuUmuTbIsyeXdvaOqLkyyvbu3Jbksydaq2pm5I9qbR84EAAAHy+gj2+nu\na5Jcs2DbBfMe70nyxifYx7uHDAcAAAO5gyQAAAwitgEAYBCxDQAAg+x3bFfVukW2vXyawwAAwOHk\nQI5s/1FVffvuj1X1Y0kun/5IAABweDiQ2P7FJB+tqhdW1T9O8utJ/vGYsQAAYOnb70v/dfcNVfVL\nST6RZE+S13S3+6MDAMA+PGFsV9XHksy/Dfr3JLk3yWVVle7+2VHDAQDAUrY/R7bfP3wKAAA4DD1h\nbHf3p5OkqtYn+ebkjo+pqmcmecHY8QAAYOk6kA9IfiTJo/OePzLZBgAALOJAYnt5dz/02JPJ4yOn\nPxIAABweDiS2d1fVtz8MWVWbktw5/ZEAAODwsN+X/ktyTpLfr6rfTFJJbk3y1iFTAQDAYeBArrP9\n1SSvqKqjk1R33zduLAAAWPr25zrbv9Ddv1dV/3zB9iRJd39g0GwAALCk7c+R7WdNvj575CAAAHC4\n2Z/rbP/u5Ot7xo8DAACHj/05jeQ/fLfXu/uXpjcOAAAcPvbnNJIbh08BAACHof05jeSKgzEIAAAc\nbvb7pjZV9cmqeu6853+vqq4dMxYAACx9B3IHyVXd/bePPenue5I8f/ojAQDA4eFAYvuRqlr72JOq\n+v4kPf2RAADg8HAgt2v/1SR/UVWfnjw/NcmW6Y8EAACHhwO5XfvHq+qkJK9IUkn+7+6+87HXq+rF\n3b1jwIwAALAkHciR7Uzi+j/v4+WtSU56yhMBAMBh4kDO2X4iNcV9AQDAkjfN2PZhSQAAmGeasQ0A\nAMwzzdh+aIr7AgCAJe9A7iD5o1X1rMnjX6iqD0yutZ0k6e5XjBgQAACWqgM5sv3bSR6oqhOSnJfk\nG0k+NGQqAAA4DBxIbO/t7k6yKcmvd/evJ3n2mLEAAGDpO5DrbN9XVb+S5BeSnFpVy5IcMWYsAABY\n+g7kyPabkjyY5OzuviPJ6iTvGzIVAAAcBg7kdu13JPnAvOd/E+dsAwDAPj1hbFfVX3T3q6vqvjz+\nxjWVpLv7OcOmAwCAJewJY7u7Xz356sOQAABwANxBEgAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCx\nDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMIrYBAGCQ4bFdVadX1c1VtbOqzl/k\n9aOq6qrJ69dX1brJ9pOr6ouTf75UVW8YPSsAAEzT0NiuqmVJLklyRpINSc6qqg0Llp2d5J7uPi7J\nxUkummz/yyQbu/vEJKcn+d2qWj5yXgAAmKbRR7ZPTrKzu2/p7oeSXJlk04I1m5JcMXl8dZLTqqq6\n+4Hu3jvZviJJD54VAACmanRsr05y67znuybbFl0ziet7k6xMkqo6pap2JPlyknPmxfe3VdWWqtpe\nVdt379494EcAAIAnZ3Rs1yLbFh6h3uea7r6+u1+c5OVJfqWqVnzHwu5Lu3tjd29ctWrVUx4YAACm\nZXRs70py7Lzna5Lcvq81k3Oyj0ly9/wF3X1TkvuTvGTYpAAAMGWjY/uGJMdX1fqqOjLJ5iTbFqzZ\nluRtk8dnJrmuu3vynuVJUlXfn+QHk3x98LwAADA1Q6/u0d17q+rcJNcmWZbk8u7eUVUXJtne3duS\nXJZka1XtzNwR7c2Tt786yflV9XCSR5P8s+6+c+S8AAAwTcMvpdfd1yS5ZsG2C+Y93pPkjYu8b2uS\nraPnAwCAUdxBEgAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR\n2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAA\nDCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4ht\nAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAG\nEdsAADCI2AYAgEHENgAADCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBhsd2\nVZ1eVTdX1c6qOn+R14+qqqsmr19fVesm219TVTdW1ZcnX39i9KwAADBNQ2O7qpYluSTJGUk2JDmr\nqjYsWHZ2knu6+7gkFye5aLL9ziQ/090vTfK2JFtHzgoAANM2+sj2yUl2dvct3f1QkiuTbFqwZlOS\nKyaPr05yWlVVd3+hu2+fbN+RZEVVHTV4XgAAmJrRsb06ya3znu+abFt0TXfvTXJvkpUL1vyTJF/o\n7gcHzQkAAFO3fPD+a5FtfSBrqurFmTu15LWLfoOqLUm2JMnatWuf3JQAADDA6CPbu5IcO+/5miS3\n72tNVS1PckySuyfP1yT54yRv7e6vLvYNuvvS7t7Y3RtXrVo15fEBAODJGx3bNyQ5vqrWV9WRSTYn\n2bZgzbbMfQAySc5Mcl13d1U9N8mfJvmV7v7vg+cEAICpGxrbk3Owz01ybZKbkvyn7t5RVRdW1c9O\nll2WZGVV7Uzyz5M8dnnAc5Mcl+TXquqLk3+eP3JeAACYptHnbKe7r0lyzYJtF8x7vCfJGxd5379J\n8m9GzwcAAKO4gyQAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAM\nIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAADCK2AQBgELENAACDiG0A\nABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR\n2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAA\nDCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAgwyP\n7ao6vapurqqdVXX+Iq8fVVVXTV6/vqrWTbavrKr/VlXfqqrfHD0nAABM29DYrqplSS5JckaSDUnO\nqqoNC5adneSe7j4uycVJLpps35Pk15L88sgZAQBglNFHtk9OsrO7b+nuh5JcmWTTgjWbklwxeXx1\nktOqqrr7/u7+i8xFNwAALDmjY3t1klvnPd812bbomu7em+TeJCsHzwUAAMONju1aZFs/iTX7/gZV\nW6pqe1Vt37179wENBwAAI42O7V1Jjp33fE2S2/e1pqqWJzkmyd37+w26+9Lu3tjdG1etWvUUxwUA\ngOkZHds3JDm+qtZX1ZFJNifZtmDNtiRvmzw+M8l13b3fR7YBAOBQtXzkzrt7b1Wdm+TaJMuSXN7d\nO6rqwiTbu3tbksuSbK2qnZk7or35sfdX1deTPCfJkVX1+iSv7e6vjJwZAACmZWhsJ0l3X5PkmgXb\nLpj3eE+SN+7jveuGDgcAAAO5gyQAAAwitgEAYBCxDQAAg4htAAAYRGwDAMAgYhsAAAYR2wAAMIjY\nBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI2AYAgEHENgAADCK2AQBg\nELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEAYBCxDQAAg4htAAAYRGwD\nAMAgYhsAAAYR2wAAMIjYBgCAQcQ2AAAMIrYBAGAQsQ0AAIOIbQAAGERsAwDAIGIbAAAGEdsAADCI\n2AYAgEHENgAADCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABhHbAAAwiNgGAIBBxDYAAAwitgEA\nYBCxDQAAg4htAAAYZHhsV9XpVXVzVe2sqvMXef2oqrpq8vr1VbVu3mu/Mtl+c1W9bvSsAAAwTUNj\nu6qWJbkkyRlJNiQ5q6o2LFh2dpJ7uvu4JBcnuWjy3g1JNid5cZLTk/zWZH8AALAkjD6yfXKSnd19\nS3c/lOTKJJsWrNmU5IrJ46uTnFZVNdl+ZXc/2N1fS7Jzsj8AAFgSlg/e/+okt857vivJKfta0917\nq+reJCsn2z+34L2rF36DqtqSZMvk6beq6ubpjA5T97wkd856CA4t9f63zXoEOJT5vcl3+tc16wke\n8/37s2h0bC/2p9H7uWZ/3pvuvjTJpQc+GhxcVbW9uzfOeg6ApcLvTQ4Ho08j2ZXk2HnP1yS5fV9r\nqmp5kmOS3L2f7wUAgEPW6Ni+IcnxVbW+qo7M3Acety1Ysy3JY3+PemaS67q7J9s3T65Wsj7J8Un+\n5+B5AQBgaoaeRjI5B/vcJNcmWZbk8u7eUVUXJtne3duSXJZka1XtzNwR7c2T9+6oqv+U5CtJ9iZ5\nZ3c/MnJeGMzpTgAHxu9NlryaO4gMAABMmztIAgDAIGIbAAAGEdsAADCI2IaDoKqeNesZAJaKqnpm\nVf3grOeAaRDbMFBVvaqqvpLkpsnzE6rqt2Y8FsAhq6p+JskXk3x88vzEqlp42WBYMsQ2jHVxktcl\nuStJuvtLSU6d6UQAh7Z3Jzk5yd8mSXd/Mcm6Gc4DT4nYhsG6+9YFm1wvHmDf9nb3vbMeAqZl6E1t\ngNxaVa9K0pO7qP5SJqeUALCov6yqn0+yrKqOz9zvzc/OeCZ40hzZhrHOSfLOJKuT7Epy4uQ5AIt7\nV5IXJ3kwyYeT/F2S/2umE8FT4A6SAAAwiNNIYICq+o0k+/w/2e7+pYM4DsAhr6o+lu/+e/NnD+I4\nMDViG8bYPusBAJaY9896ABjBaSQAADCII9swUFWtSvIvk2xIsuKx7d39EzMbCuAQNrkCyb/Ld/7e\n/IGZDQVPgauRwFi/n7lL/a1P8p4kX09ywywHAjjE/X9JfjvJ3iT/KMmHkmyd6UTwFIhtGGtld1+W\n5OHu/nR3vz3JK2Y9FMAh7Jnd/WeZO9X1G9397iT+NpAly2kkMNbDk6/frKqfSnJ7kjUznAfgULen\nqp6R5K+r6twktyV5/oxngifNByRhoKr66SR/nuTYJL+R5DlJ3tPd22Y6GMAhqqpenrnT756b5P9J\nckyS93b352Y6GDxJYhsAAAZxGgkMVFXrM3fr4XWZ99+bmzMAPF5Vfde/8fN7k6VKbMNYH01yWZKP\nJXl0xrMAHMpemeTWJB9Ocn2Smu04MB1OI4GBqur67j5l1nMAHOqqalmS1yQ5K8nLkvxpkg93946Z\nDgZPkdiGgarq55Mcn+QTSR58bHt3f35mQwEc4qrqqMxF9/uSXNjdvzHjkeBJcxoJjPXSJG/J3DVi\nHzuNpOOasQDfYRLZP5W50F6X5D8k+aNZzgRPlSPbMFBV/VWSl3X3Q7OeBeBQVlVXJHlJkv+S5Mru\n/ssZjwRTIbZhoKq6Ksm7uvt/z3oWgENZVT2a5P7J0/lxUkm6u59z8KeCp85pJDDWC5L8VVXdkMef\ns+0SVgDzdPczZj0DjCC2Yax/PesBAIDZcRoJDFZV35/k+O7+r1X1PUmWdfd9s54LABjPX9nAQFX1\njiRXJ/ndyabVmbvRDQDwNCC2Yax3JvnRJH+XJN3910meP9OJAICDRmzDWA/Ov+xfVS3P4z9lDwAc\nxsQ2jPXpqvpXSZ5ZVa9J8pEkH5vxTADAQeIDkjBQVT0jydlJXpu5a8Vem+Q/tv/wAOBpQWzDAFW1\ntrv/ZtZzAACz5TQSGOPbVxypqj+c5SAAwOyIbRij5j3+gZlNAQDMlNiGMXofjwGApxHnbMMAVfVI\nkvszd4T7mUkeeOylJN3dz5nVbADAwSO2AQBgEKeRAADAIGIbAAAGEdsAfIeq+mBVnTnrOQCWOrEN\nwFNWVctnPQPAocgvR4Alrqp+Lcmbk9ya5M4kNyb54ySXJFmVuavhvKO7/6qqPpjk75JsTPLCJOd1\n99VVVUl+I8lPJPla5l0rvqp+JMkHkhw92f8/7e5vVtWnknw2yY8m2Zbk/x3+wwIsMWIbYAmrqo1J\n/kmSH87c7/TPZy62L01yTndssd+gAAABlUlEQVT/dVWdkuS3MhfSSfKiJK9O8g8yF8lXJ3lDkh9M\n8tIkL0jylSSXV9URmYvwTd29u6relOTfJnn7ZF/P7e4fG/6DAixRYhtgaXt1kj/p7v+TJFX1sSQr\nkrwqyUfmDlgnSY6a956PdvejSb5SVS+YbDs1yYe7+5Ekt1fVdZPtP5jkJUk+OdnXsiTfnLevq6b/\nIwEcPsQ2wNJWi2x7RpK/7e4T9/GeB/fx/sVuvFBJdnT3K/exr/ufeESApy8fkARY2v4iyc9U1Yqq\nOjrJT2XuHO2vVdUbk6TmnPAE+/lMks1VtayqXpTkH02235xkVVW9crKvI6rqxUN+EoDDkNgGWMK6\n+4bMnXf9pSR/lGR7knsz94HJs6vqS0l2JNn0BLv64yR/neTLSX47yacn+38oyZlJLprs64uZO0UF\ngP3gdu0AS1xVHd3d36qq78ncEeot3f35Wc8FgHO2AQ4Hl1bVhsx9MPIKoQ1w6HBkGwAABnHONgAA\nDCK2AQBgELENAACDiG0AABhEbAMAwCBiGwAABvn/AfQldfr+2rWMAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train[['gender','is_click']].groupby('gender').mean().plot.bar()\n",
    "sns.barplot('gender', 'is_click', data = train)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 268,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Acer\\Anaconda3\\lib\\site-packages\\scipy\\stats\\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtsAAAHqCAYAAADVvoI9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3X2YZmV9J/jvT1ok4guKPbbSEpiB\nzIhRkLT4lkWvmChmjKiDKya+JDESrxV1NjPTQ9YZNU5mZu1xdXR1JrKDRnEdNWTVdiViNl5qYpR0\no0ZFhqTFF7qxkuZFgyLvv/2jHkxRVEs31N2nq+rzua66nnPuc5/n/OpcVae/fdf9nFPdHQAAYPnd\nY+oCAABgtRK2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGWTd1Acvp\nQQ96UB911FFTlwEAwCp30UUXXdnd6++s36oK20cddVS2b98+dRkAAKxyVfWtvelnGgkAAAwibAMA\nwCDCNgAADLKq5mwDALDvbrrppuzcuTPXX3/91KUccA455JBs3Lgx97znPe/S/sI2AMAat3Pnztz3\nvvfNUUcdlaqaupwDRnfnqquuys6dO3P00UffpfcwjQQAYI27/vrrc/jhhwvai1RVDj/88Ls14i9s\nAwAgaO/B3T0vwjYAAAxizjYAALdz1FkfW9b3++b//k+X9f1WEiPbAABM7glPeMKyvM/v//7v58wz\nz0yS/N7v/V7e85737LHvpz71qTzjGc9YluPuiZFtAAAm9+d//ufL/p4ve9nLlv0995WRbQAAJnef\n+9wnSfKd73wnJ598ck444YT89E//dP70T/90j/t8/OMfz4knnpjjjz8+T3nKU+6w/XWve13e+MY3\nJkl27NiRn//5n8/xxx+fE088MV//+tdv13fbtm159KMfncsuu2wZvysj2wAAHEDe97735WlPe1pe\n/epX55Zbbsl11123ZL/du3fnpS99aT7zmc/k6KOPztVXX/1j3/dXfuVXctZZZ+XZz352rr/++tx6\n6625/PLLk8yPqr/iFa/IRz7ykRx55JHL+v0I2wAAHDAe85jH5Nd//ddz00035VnPelZOOOGEJft9\n/vOfz8knn/yjh8088IEP3ON7Xnvttdm1a1ee/exnJ5l/KuRtLrnkkpxxxhn5xCc+kYc+9KHL+J3M\nM40EAIADxsknn5zPfOYzOeKII/LCF75wjx9w7O69vgd2d+9x20Me8pAccsgh+eIXv3iX6r0zRrYB\nALidKW/V961vfStHHHFEXvrSl+YHP/hBvvCFL+RFL3rRHfo9/vGPz8tf/vJ84xvf+NE0kj2Nbt/v\nfvfLxo0b8+EPfzjPetazcsMNN+SWW25Jkhx22GE555xz8tSnPjWHHnponvzkJy/r92NkGwCAA8an\nPvWpnHDCCXn0ox+dP/zDP8yrXvWqJfutX78+Z599dp7znOfk+OOPz/Oe97wf+77nnntu3vrWt+ZR\nj3pUnvCEJ2Rubu5H2x784Afnox/9aF7+8pfnwgsvXNbvp37csPpKs2nTpt6+ffvUZQAArCiXXHJJ\nHv7wh09dxgFrqfNTVRd196Y729c0kgPE5s2bMzc3lw0bNmTLli1TlwMAwDIQtg8Qc3Nz2bVr19Rl\nAAAccB772MfmhhtuuF3bueeem0c+8pETVbT3hG0AAPbp7h7723LPo94Xd3fKtQ9IAgCscYccckiu\nuuqqux0sV5vuzlVXXXW7+3LvKyPbAABr3MaNG7Nz587s3r176lIOOIccckg2btx4l/cXtgEA1rh7\n3vOeP3oSI8vLNBIAABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAA\nGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhkeNiuqlOq6tKq2lFVZy2x/eSq+kJV3VxV\npy1oP6GqPldVF1fVl6vqeaNrBQCA5TQ0bFfVQUnenuTpSY5L8vyqOm5Rt28n+dUk71vUfl2SF3X3\nI5KckuQ/V9VhI+sFAIDltG7w+5+UZEd3X5YkVfX+JKcm+dptHbr7m7Ntty7csbv/asHyFVX1t0nW\nJ/nu4JoBAGBZjJ5GckSSyxes75y17ZOqOinJwUm+vkx1AQDAcKPDdi3R1vv0BlUPSXJukl/r7luX\n2H5GVW2vqu27d+++i2UCAMDyGx22dyZ52IL1jUmu2Nudq+p+ST6W5N909+eX6tPdZ3f3pu7etH79\n+rtVLAAALKfRYXtbkmOr6uiqOjjJ6Um27s2Os/4fSvKe7v6DgTUCAMAQQ8N2d9+c5MwkFyS5JMkH\nu/viqnp9VT0zSarqMVW1M8lzk7yjqi6e7f4/Jzk5ya9W1ZdmXyeMrBcAAJbT6LuRpLvPT3L+orbX\nLFjelvnpJYv3e2+S946uDwAARvEESQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAA\nGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgG\nAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCA\nQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGE\nbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGGR42K6qU6rq0qraUVVnLbH95Kr6\nQlXdXFWnLdr24qr669nXi0fXCgAAy2lo2K6qg5K8PcnTkxyX5PlVddyibt9O8qtJ3rdo3wcmeW2S\nxyY5Kclrq+oBI+sFAIDlNHpk+6QkO7r7su6+Mcn7k5y6sEN3f7O7v5zk1kX7Pi3JH3f31d19TZI/\nTnLK4HoBAGDZjA7bRyS5fMH6zlnbsu1bVWdU1faq2r579+67XCgAACy30WG7lmjr5dy3u8/u7k3d\nvWn9+vX7VBwAAIw0OmzvTPKwBesbk1yxH/YFAIDJjQ7b25IcW1VHV9XBSU5PsnUv970gyVOr6gGz\nD0Y+ddYGAAArwtCw3d03Jzkz8yH5kiQf7O6Lq+r1VfXMJKmqx1TVziTPTfKOqrp4tu/VSf5d5gP7\ntiSvn7UBAMCKsG70Abr7/CTnL2p7zYLlbZmfIrLUvu9M8s6hBQIAwCCeIAkAAIMI2wAAMIiwDQAA\ngwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI\n2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADDIuqkLAABgbdm8eXPm5uayYcOGbNmyZepyhhK2AQDY\nr+bm5rJr166py9gvTCMBAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgG\nAIBBhG0AABhE2AYAgEGEbQAAGGTd1AVM4Wf+1XumLuEO7nvltTkoybevvPaAq++i//SiqUsAAFiR\njGwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAga/I+2wAAa8Ul//6TU5dwBzde\n/cMfvR5o9T381T+3rO9nZBsAAAYRtgEAYBBhGwAABhketqvqlKq6tKp2VNVZS2y/V1V9YLb9wqo6\natZ+z6p6d1V9paouqarfHl0rAAAsp6Fhu6oOSvL2JE9PclyS51fVcYu6vSTJNd19TJI3J3nDrP25\nSe7V3Y9M8jNJfvO2IA4AACvB6JHtk5Ls6O7LuvvGJO9PcuqiPqcmefds+bwkT6mqStJJDq2qdUl+\nIsmNSf5ucL0AALBsRoftI5JcvmB956xtyT7dfXOS7yU5PPPB+wdJvpPk20ne2N1XD64XAACWzeiw\nXUu09V72OSnJLUkemuToJP+iqv7hHQ5QdUZVba+q7bt377679QIAwLIZHbZ3JnnYgvWNSa7YU5/Z\nlJH7J7k6yS8n+Xh339Tdf5vks0k2LT5Ad5/d3Zu6e9P69esHfAsAAHDXjA7b25IcW1VHV9XBSU5P\nsnVRn61JXjxbPi3JJ7u7Mz915Odq3qFJHpfkfwyuFwAAls3QsD2bg31mkguSXJLkg919cVW9vqqe\nOet2TpLDq2pHkt9KctvtAd+e5D5Jvpr50P6u7v7yyHoBAGA5rRt9gO4+P8n5i9pes2D5+szf5m/x\nft9fqh0AAFYKT5AEAIBBhG0AABhk+DQSWAk2b96cubm5bNiwIVu2bJm6HABglRC2Icnc3Fx27do1\ndRkA7CcGWdhfhG0AYM0xyML+Ys42AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIG79x373\n7dc/cuoS7uDmqx+YZF1uvvpbB1x9R77mK1OXAADcRUa2AQBgECPbAMBQ//4Fp01dwh1c/bffm3+d\n+84BWd+r33ve1CWwTIxsAwDAIHsdtqvqqCXaHrOcxQAAwGqyLyPb/09VHXHbSlU9Kck7l78kAABY\nHfYlbP9mkg9X1Yaq+sUkb0nyi2PKAgCAlW+vPyDZ3duq6pVJPpHk+iS/0N27h1UGrAmbN2/O3Nxc\nNmzYkC1btkxdDgAsqzsN21X10SS9oOneSb6X5JyqSnc/c1RxsL886JBbk9w8e2V/mpuby65du6Yu\nA4D96PBD7n+719Vsb0a23zi8CpjYv3zUd6cuAQDWjDMf/ctTl7Df3GnY7u5PJ0lVHZ3kO919/Wz9\nJ5I8eGx5AACwcu3LByT/IMnCv7HfMmsDAACWsC9PkFzX3TfettLdN1bVwQNqAgZ54v/5xKlLuIOD\nv3tw7pF75PLvXn5A1vfZV3x26hKAAQ456B63e4VR9iVs766qZ3b31iSpqlOTXDmmLACAcR59+H2n\nLoE1Yl/C9suS/N9V9bYkleTyJC8aUhUAAKwC+3Kf7a8neVxV3SdJdfe148oCAICVb2/us/2C7n5v\nVf3WovYkSXe/aVBtAACwou3NyPahs1eTm4Bl1/fu3Jpb0/fuO+8MACvM3txn+x2z198ZXw6w1tz0\nxJumLgEAhtmbaSRv/XHbu/uVy1cOAACsHnszjeSi4VUAAMAqtDfTSN69PwoBAIDVZq8fm1RVf1xV\nhy1Yf0BVXTCmLAAAWPn25Rml67v7u7etdPc1Sf7B8pcEAACrw76E7Vuq6sjbVqrqJ5O4VxcAAOzB\nvjyu/dVJ/qyqPj1bPznJGctfEgAArA778rj2j1fViUkel6SS/K/dfeVt26vqEd198YAaAQBgRdqX\nke3MwvX/u4fN5yY58W5XBAAAq8Q+he07Ucv4XgDsB5s3b87c3Fw2bNiQLVu2TF0OwKqznGHbhyUB\nVpi5ubns2rVr6jIAVq19uRsJAACwD5YzbN+4jO8FAAAr3r48QfKJVXXobPkFVfWm2b22kyTd/bgR\nBQIAwEq1L3O2/2uS46vq+CSbk5yT5D1JnjSiMIDV5tMnH3iXyx+uOyipyg937jzg6nvSZz59550A\nDnD7Mo3k5u7uJKcmeUt3vyXJfceUBQAAK9++jGxfW1W/neQFSU6uqoOS3HNMWQAAsPLty8j285Lc\nkOQl3T2X5Igk/2lIVQAAsArsy+Pa55K8acH6tzM/ZxuAFeqw7tu9ArC87jRsV9WfdffPVtW1uf2D\naypJd/f9hlUHwFAvuOXWqUsAWNXuNGx398/OXn0YEgAA9oEnSAIAwCDCNgAADCJsAwDAIMPDdlWd\nUlWXVtWOqjprie33qqoPzLZfWFVHLdj2qKr6XFVdXFVfqapDRtcLAADLZV8earPPZg++eXuSX0iy\nM8m2qtra3V9b0O0lSa7p7mOq6vQkb0jyvKpal+S9SV7Y3X9ZVYcnuWlkvQCwP23evDlzc3PZsGFD\ntmzZMnU5wACjR7ZPSrKjuy/r7huTvD/zj3tf6NQk754tn5fkKVVVSZ6a5Mvd/ZdJ0t1Xdfctg+sF\ngP1mbm4uu3btytzc3NSlAIOMDttHJLl8wfrOWduSfbr75iTfS3J4kp9K0lV1QVV9oao2D64VAACW\n1dBpJJl/8M1iix9Ttqc+65L8bJLHJLkuyZ9U1UXd/Se327nqjCRnJMmRRx55twsGAIDlMnpke2eS\nhy1Y35jkij31mc3Tvn+Sq2ftn+7uK7v7uiTnJzlx8QG6++zu3tTdm9avXz/gWwAAgLtmdNjeluTY\nqjq6qg5OcnqSrYv6bE3y4tnyaUk+2d2d5IIkj6qqe89C+JOSfC0AALBCDJ1G0t03V9WZmQ/OByV5\nZ3dfXFWvT7K9u7cmOSfJuVW1I/Mj2qfP9r2mqt6U+cDeSc7v7o+NrBcAAJbT6Dnb6e7zMz8FZGHb\naxYsX5/kuXvY972Zv/0fANwtb/sXH526hDv47pU/+NHrgVbfmf/HL01dAqwKniAJAACDCNsAADCI\nsA0AAIMI2wAAMIiwDQAAgwy/GwkAsLRDD77f7V6B1UfYBoCJPPEfPWfqEoDBTCMBAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgG\nAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCA\nQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGE\nbQAAGGTd1AUw79aDD73dKwAAK5+wfYD4wbFPnboEAACWmWkkAAAwiLANAACDCNsAADCIsA0AAIMI\n2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwwP21V1SlVdWlU7quqs\nJbbfq6o+MNt+YVUdtWj7kVX1/ar6l6NrBQCA5TQ0bFfVQUnenuTpSY5L8vyqOm5Rt5ckuaa7j0ny\n5iRvWLT9zUn+aGSdAAAwwuiR7ZOS7Ojuy7r7xiTvT3Lqoj6nJnn3bPm8JE+pqkqSqnpWksuSXDy4\nTgAAWHajw/YRSS5fsL5z1rZkn+6+Ocn3khxeVYcm+ddJfufHHaCqzqiq7VW1fffu3ctWOAAA3F2j\nw3Yt0dZ72ed3kry5u7//4w7Q3Wd396bu3rR+/fq7WCYAACy/dYPff2eShy1Y35jkij302VlV65Lc\nP8nVSR6b5LSq2pLksCS3VtX13f22wTUDAMCyGB22tyU5tqqOTrIryelJfnlRn61JXpzkc0lOS/LJ\n7u4k/9NtHarqdUm+L2gDALCSDA3b3X1zVZ2Z5IIkByV5Z3dfXFWvT7K9u7cmOSfJuVW1I/Mj2qeP\nrAkAAPaX0SPb6e7zk5y/qO01C5avT/LcO3mP1w0pDgAABvIESQAAGETYBgCAQYRtAAAYRNgGAIBB\nhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRt\nAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAA\nGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgG\nAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGGh+2qOqWqLq2q\nHVV11hLb71VVH5htv7Cqjpq1/0JVXVRVX5m9/tzoWgEAYDkNDdtVdVCStyd5epLjkjy/qo5b1O0l\nSa7p7mOSvDnJG2btVyb5pe5+ZJIXJzl3ZK0AALDcRo9sn5RkR3df1t03Jnl/klMX9Tk1ybtny+cl\neUpVVXd/sbuvmLVfnOSQqrrX4HoBAGDZjA7bRyS5fMH6zlnbkn26++Yk30ty+KI+/yzJF7v7hkF1\nAgDAsls3+P1ribbelz5V9YjMTy156pIHqDojyRlJcuSRR961KgEAYIDRI9s7kzxswfrGJFfsqU9V\nrUty/yRXz9Y3JvlQkhd199eXOkB3n93dm7p70/r165e5fAAAuOtGh+1tSY6tqqOr6uAkpyfZuqjP\n1sx/ADJJTkvyye7uqjosyceS/HZ3f3ZwnQAAsOyGhu3ZHOwzk1yQ5JIkH+zui6vq9VX1zFm3c5Ic\nXlU7kvxWkttuD3hmkmOS/Nuq+tLs6x+MrBcAAJbT6Dnb6e7zk5y/qO01C5avT/LcJfb73SS/O7o+\nAAAYxRMkAQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsA\nAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAG\nEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2\nAQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEA\nYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQ\nYRsAAAYRtgEAYBBhGwAABhG2AQBgkOFhu6pOqapLq2pHVZ21xPZ7VdUHZtsvrKqjFmz77Vn7pVX1\ntNG1AgDAchoatqvqoCRvT/L0JMcleX5VHbeo20uSXNPdxyR5c5I3zPY9LsnpSR6R5JQk/2X2fgAA\nsCKMHtk+KcmO7r6su29M8v4kpy7qc2qSd8+Wz0vylKqqWfv7u/uG7v5Gkh2z9wMAgBWhunvcm1ed\nluSU7v6N2foLkzy2u89c0Oersz47Z+tfT/LYJK9L8vnufu+s/Zwkf9Td5y06xhlJzpit/uMklw77\nhsZ7UJIrpy5iDXP+p+PcT8v5n5bzPx3nflor/fz/ZHevv7NO6wYXUUu0LU73e+qzN/umu89Ocva+\nl3bgqart3b1p6jrWKud/Os79tJz/aTn/03Hup7VWzv/oaSQ7kzxswfrGJFfsqU9VrUty/yRX7+W+\nAABwwBodtrclObaqjq6qgzP/gceti/psTfLi2fJpST7Z83NbtiY5fXa3kqOTHJvkLwbXCwAAy2bo\nNJLuvrmqzkxyQZKDkryzuy+uqtcn2d7dW5Ock+TcqtqR+RHt02f7XlxVH0zytSQ3J3l5d98yst4D\nwKqYDrOCOf/Tce6n5fxPy/mfjnM/rTVx/od+QBIAANYyT5AEAIBBhG0AABhE2AYAgEFG32ebvVBV\nD0zS3X3N1LXA/lJVD05yRObvn39Fd//NxCWtOa4903Hup+PaM621eP59QHIiVXVkki1JnpLku5l/\niM/9knwyyVnd/c3pqls71uIv/dSq6oQkv5f5e+rvmjVvzPzvwf/S3V+Yqra1wLVnOs79tFx7prWW\nz7+wPZGq+lyS/5zkvNtuaVhVByV5bpJ/3t2Pm7K+1W4t/9JPraq+lOQ3u/vCRe2PS/KO7j5+msrW\nBtee6Tj303LtmdZaPv/C9kSq6q+7+9h93cbyWMu/9FO7k5/9Hd19zP6uaS1x7ZmOcz8t155preXz\nb872dC6qqv+S5N1JLp+1PSzzT9P84mRVrR2HLg7aSdLdn6+qQ6coaA35o6r6WJL35PY/+y9K8vHJ\nqlo7XHum49xPy7VnWmv2/BvZnsjs8fUvSXJq5ucMV+Z/+D6a5JzuvmHC8la9qnprkn+UpX/pv9Hd\nZ05V21pQVU/P7X/2dybZ2t3nT1rYGuDaMx3nfnquPdNaq+df2GbNWqu/9ADA/uM+2wegqnrG1DWs\nBd39R939su7+pe5+xmxZ0J5QVZ0xdQ1rmWvPdJz7abn2TGu1n39h+8D0mKkLWMtW+y/9Aa6mLmCN\nc+2ZjnM/Ldeeaa3q8+8DkhOqqn+Sv5/G0EmuyPw0htdOWhir+pf+QDD72T8iyYXd/f0Fm741UUlr\nSlWdlPkHqmyrquOSnJLkf7j27H9V9Z7ufpFzv/9V1c8mOSnJV7v7HVPXs9pV1WOTXNLdf1dVP5Hk\nrCQnJvlakv8waXGDGdmeSFX96yTvz3yw+4sk22bL/72qzpqyNnLj1AWsZlX1yiQfSfKKJF+tqlMX\nbF7VF9wDQVW9Nslbk/zXqvqPSd6W5D5JzqqqV09a3CpXVVsXfX00yXNuW5+6vtWuqv5iwfJLM/+z\nf98kr/Xv7n7xziTXzZbfkvnnXLxh1vauqYraH3xAciJV9VdJHtHdNy1qPzjJxe63Op2q+nZ3Hzl1\nHatVVX0lyeO7+/tVdVSS85Kc291vqaovdvejJy1wlZud/xOS3CvJXJKNC0aaLuzuR01a4CpWVV/I\n/Cjef8v8XzMryX9PcnqSdPenp6tu9Vt4famqbUl+sbt3z273+vnufuS0Fa5uVXVJdz98tvyF7j5x\nwbYvdfcJ01U3lmkk07k1yUNzxz+bP2S2jYGq6st72pTkwfuzljXooNumjnT3N6vqyUnOq6qfjCk8\n+8PNs6cXXldVX+/uv0uS7v5hVbn2jLUpyauSvDrJv+ruL1XVD4Xs/eYeVfWAzP9Vv7p7d5J09w+q\n6uZpS1sTvlpVv9bd70ryl1W1qbu3V9VPJbnpznZeyYTt6fzzJH9SVX+dv7/P85FJjkniHs/jPTjJ\n05Jcs6i9kvz5/i9nTZmrqhO6+0tJMhvhfkbm/8RoZGm8G6vq3t19XZKfua2xqu4f/9EfqrtvTfLm\nqvqD2evfxL/D+9P9k1yU+etro8buAAACwklEQVR8V9WG7p6rqvvEf/T3h99I8paq+jdJrkzyuaq6\nPPMZ6DcmrWww00gmVFX3yPyHMxbe53nbbNSJgarqnCTv6u4/W2Lb+7r7lycoa02oqo2ZH12dW2Lb\nE7v7sxOUtWZU1b2WenhKVT0oyUO6+ysTlLUmVdU/TfLE7v7fpq5lLauqeyd5cHd/Y+pa1oKqum+S\nf5j5/2ju7O6/mbik4YRtAAAYxN1IAABgEGEbAAAGEbYBAGAQYRtgjauqX62qtx3o7wmwEgnbAAAw\niLANsEJU1Yer6qKquriqzpi1vaSq/qqqPlVV/9dto8lVtb6q/rCqts2+nriXx7jDflV1j6r6ZlUd\ntqDfjqp68F09DsBa4Wb6ACvHr3f31bNHq2+rqo8l+bdJTkxybZJPJvnLWd+3JHlzd/9ZVR2Z5IIk\nD9+LY9xhv+5+eFV9JMmzk7yrqh6b5Jvd/TdV9b67eByANUHYBlg5XllVz54tPyzJC5N8uruvTpLZ\nkwl/arb955McV/WjB+Pdr6ru293X3skxltwvyQeSvCbJu5KcPlv/cf0BiLANsCJU1ZMzH2wf393X\nVdWnklyaPY8i32PW94f7eKgl96uqzyU5pqrWJ3lWkt+9k/77eFiA1cmcbYCV4f5JrpkF7X+S5HFJ\n7p3kSVX1gKpal+SfLej/iSRn3rZSVSfs5XGW3K/nHzf8oSRvSnJJd191N48DsCYI2wArw8eTrKuq\nLyf5d0k+n2RXkv+Q5MIk/1+SryX53qz/K5NsqqovV9XXkrxsL4/z4/b7QJIX5O+nkNyd4wCsCTU/\nWAHASlRV9+nu789Gtj+U5J3d/aGp6wJgnpFtgJXtdVX1pSRfTfKNJB+euB4AFjCyDbBGVNWvJXnV\noubPdvfLp6gHYC0QtgEAYBDTSAAAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGCQ/x+5DvSzLPe96wAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train[['age_level','is_click']].groupby('age_level').mean().plot.bar()\n",
    "sns.barplot('age_level', 'is_click', data = train)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Acer\\Anaconda3\\lib\\site-packages\\scipy\\stats\\stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtsAAAHwCAYAAAB386PAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu0ZWV5J+rfKwgoIChUQCkJdGvS\nYhQkCMbYtIk3TFQ0aovGWzSSnBaTnHSPamz7GGO607GOIyYOTSecYFSMwVuMqEQ0MWhMFMFLVEQS\nxAtVuO0S8IIKCLznj7UgVdsqqnaxvlq1F88zRo251jfn/Nb7Va296rfn+uac1d0BAABm707zLgAA\nABaVsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMMie8y5glg4++OA+\n4ogj5l0GAAAL7hOf+MQ3unvN9rZbqLB9xBFH5KKLLpp3GQAALLiq+sqObGcaCQAADCJsAwDAIMI2\nAAAMslBztgEAWLkf/OAH2bBhQ6677rp5l7Lb2WeffbJ27drc+c533qn9hW0AgDu4DRs2ZP/9988R\nRxyRqpp3ObuN7s5VV12VDRs25Mgjj9ypPkwjAQC4g7vuuuty0EEHCdrLVFUOOuig23XEX9gGAEDQ\n3obb+/cibAMAwCDmbAMAsIUjTn/vTPv78u/9/Ez7W00c2QYAYO4e+tCHzqSf17/+9TnttNOSJH/8\nx3+cN77xjdvc9vzzz8/jHve4mbzutjiyDQDA3P3jP/7jzPv81V/91Zn3uVKObAMAMHf77bdfkuRr\nX/taTjzxxBxzzDH5iZ/4ifz93//9Nvd53/vel2OPPTZHH310HvGIR/zQ+pe97GV55StfmSS57LLL\n8shHPjJHH310jj322Hzxi1/cYtsLL7wwD3rQg3L55ZfPcFSObAMAsBt585vfnMc85jF5yUtekptu\nuinf+973trrdpk2b8oIXvCAf/vCHc+SRR+bqq6++zX5/8Rd/Maeffnqe9KQn5brrrsvNN9+cK664\nIsnkqPqLXvSivOtd78rhhx8+0/EI2wAA7DYe/OAH53nPe15+8IMf5IlPfGKOOeaYrW73sY99LCee\neOKtN5u5xz3usc0+v/Od72Tjxo150pOelGRyV8hbXHLJJTn11FPz/ve/P/e6171mOJIJ00gAANht\nnHjiifnwhz+cww47LM961rO2eYJjd+/wNbC7e5vr7nnPe2afffbJpz71qZ2qd3sc2QYAYAvzvFTf\nV77ylRx22GF5wQtekO9+97v55Cc/mWc/+9k/tN1P/dRP5YUvfGG+9KUv3TqNZFtHt+92t7tl7dq1\n+au/+qs88YlPzPXXX5+bbropSXLggQfmzDPPzKMf/ejsu+++efjDHz7T8TiyDQDAbuP888/PMccc\nkwc96EF5xzvekV//9V/f6nZr1qzJGWeckV/4hV/I0Ucfnac97Wm32e9ZZ52VV7/61XngAx+Yhz70\noVlaWrp13SGHHJJ3v/vdeeELX5gLLrhgpuOp2zqsvtocd9xxfdFFFw19jXXr1mVpaSmHHnpo1q9f\nP/S1AAB2hUsuuST3u9/95l3Gbmtrfz9V9YnuPm57+5pGskJLS0vZuHHjvMsAAGAVELYBANitnXDC\nCbn++uu3aDvrrLPygAc8YE4V7ThhGwCAFV3dY1eb9Tzqlbi9U66dIAkAcAe3zz775KqrrrrdwXLR\ndHeuuuqqLa7LvVKObAMA3MGtXbs2GzZsyKZNm+Zdym5nn332ydq1a3d6f2EbAOAO7s53vvOtd2Jk\ntkwjAQCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE\n2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgG\nAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYaH7ao6qaourarLqur0raw/sao+WVU3VtVTNms/pqo+\nWlUXV9Vnquppo2sFAIBZGhq2q2qPJK9N8tgkRyV5elUdtWyzryZ5bpI3L2v/XpJnd/f9k5yU5A+q\n6sCR9QIAwCztObj/45Nc1t2XJ0lVnZ3k5CSfv2WD7v7ydN3Nm+/Y3f+82eMrq+r/JFmT5JuDawYA\ngJkYPY3ksCRXbPZ8w7RtRarq+CR7JfniVtadWlUXVdVFmzZt2ulCAQBg1kaH7dpKW6+og6p7Jjkr\nyS91983L13f3Gd19XHcft2bNmp0sEwAAZm902N6Q5N6bPV+b5Mod3bmq7pbkvUn+e3d/bMa1AQDA\nUKPD9oVJ7ltVR1bVXklOSXLOjuw43f6dSd7Y3W8bWCMAAAwxNGx3941JTktyXpJLkry1uy+uqpdX\n1ROSpKoeXFUbkjw1yZ9U1cXT3f9jkhOTPLeqPj39c8zIegEAYJZGX40k3X1uknOXtb10s8cXZjK9\nZPl+b0ryptH1AQDAKO4gCQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAw\niLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiw\nDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0A\nAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAg+w57wIAAGAW1q1b\nl6WlpRx66KFZv379vMtJImwDALAglpaWsnHjxnmXsQXTSAAAYBBhGwAABhG2AQBgEGEbAAAGEbYB\nAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBg\nEGEbAAAGGR62q+qkqrq0qi6rqtO3sv7EqvpkVd1YVU9Ztu45VfUv0z/PGV0rAADM0tCwXVV7JHlt\nkscmOSrJ06vqqGWbfTXJc5O8edm+90jyW0lOSHJ8kt+qqruPrBcAAGZp9JHt45Nc1t2Xd/cNSc5O\ncvLmG3T3l7v7M0luXrbvY5J8oLuv7u5rknwgyUmD6wUAgJkZHbYPS3LFZs83TNtmtm9VnVpVF1XV\nRZs2bdrpQgEAYNZGh+3aSlvPct/uPqO7j+vu49asWbOi4gAAYKTRYXtDkntv9nxtkit3wb4AADB3\no8P2hUnuW1VHVtVeSU5Jcs4O7ntekkdX1d2nJ0Y+etoGAACrwtCw3d03Jjktk5B8SZK3dvfFVfXy\nqnpCklTVg6tqQ5KnJvmTqrp4uu/VSX4nk8B+YZKXT9sAAGBV2HP0C3T3uUnOXdb20s0eX5jJFJGt\n7fu6JK8bWiAAAAziDpIAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI\n2wAAMIiwDQAAgwjbAAAwiLANAACDCNsAADCIsA0AAIMI2wAAMIiwDQAAgwjbAAAwiLANAACDCNsA\nADCIsA0AAIMI2wAAMMie8y4AgPlZt25dlpaWcuihh2b9+vXzLgdg4QjbAHdgS0tL2bhx47zLAFhY\nppEAAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAM4tJ/AADMzddf/ZGZ9XXTN6+7dTnLfg/5\ntYft9L6ObAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2\nAAAMImwDAMAge867gF1h0/9+08z6uulb37l1Oct+1/xfz5xZXwAA7B4c2QYAgEGEbQAAGETYBgCA\nQYRtAAAYRNgGAIBBhG0AABhE2AYAgEHuENfZBgBg69atW5elpaUceuihWb9+/bzLWTjCNgDAHdjS\n0lI2btw47zIWlmkkAAAwiLANAACDCNsAADDI8LBdVSdV1aVVdVlVnb6V9XtX1Vum6y+oqiOm7Xeu\nqjdU1Wer6pKqevHoWgEAYJaGhu2q2iPJa5M8NslRSZ5eVUct2+z5Sa7p7vskeVWSV0zbn5pk7+5+\nQJKfTPIrtwRxAABYDUYf2T4+yWXdfXl335Dk7CQnL9vm5CRvmD5+e5JHVFUl6ST7VtWeSe6S5IYk\n3x5cLwAAzMzosH1Ykis2e75h2rbVbbr7xiTfSnJQJsH7u0m+luSrSV7Z3Vcvf4GqOrWqLqqqizZt\n2jT7EQAAwE4aHbZrK229g9scn+SmJPdKcmSS/1xV/+aHNuw+o7uP6+7j1qxZc3vrBQCAmRkdtjck\nufdmz9cmuXJb20ynjByQ5Ookz0jyvu7+QXf/nyT/kOS4wfUCAMDMjA7bFya5b1UdWVV7JTklyTnL\ntjknyXOmj5+S5IPd3ZlMHfnZmtg3yUOSfGFwvQAAMDNDw/Z0DvZpSc5LckmSt3b3xVX18qp6wnSz\nM5McVFWXJfnNJLdcHvC1SfZL8rlMQvufdfdnRtYLAACztOfoF+juc5Ocu6ztpZs9vi6Ty/wt3+/a\nrbUDwLasW7cuS0tLOfTQQ7N+/fp5lwMwPmwDMFs//47/b2Z9XX/t5IqqV1777Zn2+94nv2Bmfa3E\n0tJSNm7cOJfXBtgaYfsOzlEgAIBxhO07OEeBAADGGX01EgAAuMNyZBtghUy/AmBHCdsAK2T6FTBP\nX1v/tZn2d9M1N926nFXf91x3z5n0swhMIwEAgEGEbQAAGETYBgCAQYRtAAAYxAmSq8yG1zxvpv3d\n+M2v37qcVd9rT3vdTPoBAFjtHNkGAIBBhG0AABhE2AYAgEGEbQAAGGSHw3ZVHbGVtgfPshgAAFgk\nK7kayV9W1eO7e2OSVNV/SPKaJA8YUhnAjPzcX/23mfZ3w3evSpJc+d2rZtb3uU/83Zn0A8DuZSVh\n+1eS/FVVPT7JsUl+N8nPDakKdsK6deuytLSUQw89NOvXr593OQAAOx62u/vCqvq1JO9Pcl2SR3X3\npmGVwQotLS1l48aN8y4DAOBW2w3bVfXuJL1Z012TfCvJmVWV7n7CqOIY7+B999xiCQCwWq2564Fb\nLHcHO5KwXjm8CuZm3U8fMu8SAABm4sUPffa8S/gh2w3b3f2hJKmqI5N8rbuvmz6/SxJJDQAAtmEl\ncwfeluShmz2/adrm8n/stL/705+fWV/f//b10+WVM+33Z375vTPrCwB2Nwff5eAtlszWSsL2nt19\nwy1PuvuGqtprQE0AAOwipx93+rxLWGgruYPkpqq69WTIqjo5yTdmXxIAACyGlRzZ/tUkf15Vr0lS\nSa5IsvvNQgeABeDeAbAYVnKd7S8meUhV7Zekuvs748oCgDs29w6AxbAj19l+Zne/qap+c1l7kqS7\nf39QbbAiB+xbWywBAOZtR45s7ztd7j+yELi9fvFnnK/LLrL/XqnpEgBuy45cZ/tPpsvfHl8OwO5v\nryfeb94lALBK7Mg0klff1vru/rXZlQMAAItjR6aRfGJ4FQAAsIB2ZBrJG3ZFIQAAsGh2+KY2VfWB\nqjpws+d3r6rzxpQFAACr30puarOmu795y5PuvqaqfmRATQDAgnGTHu6oVhK2b6qqw7v7q0lSVT+a\npMeUBQAsEjfp4Y5qJWH7JUk+UlUfmj4/Mcmpsy8JAAAWw0pu1/6+qjo2yUOSVJL/u7u/ccv6qrp/\nd188oEYAAFiVVnJkO9Nw/Z5trD4rybG3uyIgifmNALAIVhS2t6Nm2Bfc4ZnfyK5Q+991iyUAszXL\nsO1kSYBVZq8n/My8SwBYaDt8nW0AAGBlZhm2b5hhXwAAsOrt8DSSqvrpJJ/u7u9W1TMzORnyD7v7\nK0nS3Q8ZVCOsCq9/w6Nn2t+3v33TdLlxZn0/9znvn0k/MCuPf/s7Z9rf96+9Nkly5bXXzqzvdz/l\nSTPpB7hjWsmR7f+d5HtVdXSSdUm+kuSNQ6oCAIAFsJKwfWN3d5KTMzmi/YdJ9h9TFgAArH4ruRrJ\nd6rqxUmemeTEqtojyZ3HlAUAq8+T3/HxmfX17WuvS5J87drrZtrvO558/Mz6ArZvJWH7aUmekeT5\n3b1UVYcn+X/HlAUAzNPvv3Nppv1989qbbl3Oqu/ffNKhM+kHRlrJ7dqXkvz+Zs+/GnO2YZj99tty\nCQCsPtsN21X1ke5+WFV9J1veuKaSdHffbVh1cAf2yEftMe8SAIDbabthu7sfNl06GRIAAFbAHSQB\nAGCQlZwgCbBT1q1bl6WlpRx66KFZv379vMsBgF1G2AaGW1paysaNG+ddBgDscsOnkVTVSVV1aVVd\nVlWnb2X93lX1lun6C6rqiM3WPbCqPlpVF1fVZ6tqn9H1AgDArAwN29Mb37w2yWOTHJXk6VV11LLN\nnp/kmu6+T5JXJXnFdN89k7wpya929/2TPDzJD0bWCwAAszR6GsnxSS7r7suTpKrOzuR275/fbJuT\nk7xs+vjtSV5TVZXk0Uk+093/lCTdfdXgWgEAtst5KKzE6GkkhyW5YrPnG6ZtW92mu29M8q0kByX5\nsSRdVedV1Serat3gWgEAtuuW81CWlmZ7l00W0+gj27WVtt7BbfZM8rAkD07yvSR/W1Wf6O6/3WLn\nqlOTnJokhx9++O0uGACYvbvc7eAtlnBHMTpsb0hy782er01y5Ta22TCdp31Akqun7R/q7m8kSVWd\nm+TYJFuE7e4+I8kZSXLcccctD/IAwG7ghJNfPO8SYC5Gh+0Lk9y3qo5MsjHJKUmesWybc5I8J8lH\nkzwlyQe7u6vqvCTrququSW5I8h8yOYES2AVe8raTZtbXVdf+YLrcONN+/+dT3zezvgBghKFhu7tv\nrKrTkpyXZI8kr+vui6vq5Uku6u5zkpyZ5KyquiyTI9qnTPe9pqp+P5PA3knO7e73jqwXAABmafhN\nbbr73CTnLmt76WaPr0vy1G3s+6ZMLv8HAACrzvCb2gAAwB2VsA0AAIMMn0YCAKzcnfa/+xZLYHUS\ntgFgN7TfE54/7xKAGRC2geH23r+S9HQJAHccwjYw3P0e56MGgDsm/wMCAAvv/Ddtmllf3//OTbcu\nZ9nvw5+5ZmZ9sftwNRIAABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGE\nbQAAGMQdJAFYGLX/3bZYAsybsA3Awtjn8SfPuwSALQjbAAArcOB+a7ZYwm0RtgEAVuAXH/OSeZfA\nKuIESQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBB\nhG0AABhE2AYAgEGEbQAAGETYBgCAQYRtAAAYRNgGAIBBhG0AABhE2AYAgEGEbQAAGETYBgCAQYRt\nAAAYZM95F7DarLnrflssAQBgW4TtFXrJiY+ZdwkAAKwSppEAAMAgwjYAAAwibAMAwCDCNgAADCJs\nAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMMjxsV9VJVXVpVV1WVadv\nZf3eVfWW6foLquqIZesPr6prq+q/jK4VAABmaWjYrqo9krw2yWOTHJXk6VV11LLNnp/kmu6+T5JX\nJXnFsvWvSvLXI+sEAIARRh/ZPj7JZd19eXffkOTsJCcv2+bkJG+YPn57kkdUVSVJVT0xyeVJLh5c\nJwAAzNzosH1Ykis2e75h2rbVbbr7xiTfSnJQVe2b5L8m+e3beoGqOrWqLqqqizZt2jSzwgEA4PYa\nHbZrK229g9v8dpJXdfe1t/UC3X1Gdx/X3cetWbNmJ8sEAIDZ23Nw/xuS3Huz52uTXLmNbTZU1Z5J\nDkhydZITkjylqtYnOTDJzVV1XXe/ZnDNAAAwE6PD9oVJ7ltVRybZmOSUJM9Yts05SZ6T5KNJnpLk\ng93dSf79LRtU1cuSXCtoAwCwmgwN2919Y1WdluS8JHskeV13X1xVL09yUXefk+TMJGdV1WWZHNE+\nZWRNAACwq4w+sp3uPjfJucvaXrrZ4+uSPHU7fbxsSHEAADCQO0gCAMAgwjYAAAwibAMAwCDCNgAA\nDCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwi\nbAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMImwD\nAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDA\nIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDC\nNgAADCJsAwDAIMI2AAAMImwDAMAgwjYAAAwibAMAwCDCNgAADCJsAwDAIMI2AAAMMjxsV9VJVXVp\nVV1WVadvZf3eVfWW6foLquqIafujquoTVfXZ6fJnR9cKAACzNDRsV9UeSV6b5LFJjkry9Ko6atlm\nz09yTXffJ8mrkrxi2v6NJI/v7gckeU6Ss0bWCgAAszb6yPbxSS7r7su7+4YkZyc5edk2Jyd5w/Tx\n25M8oqqquz/V3VdO2y9Osk9V7T24XgAAmJnRYfuwJFds9nzDtG2r23T3jUm+leSgZds8Ocmnuvv6\n5S9QVadW1UVVddGmTZtmVjgAANxeo8N2baWtV7JNVd0/k6klv7K1F+juM7r7uO4+bs2aNTtdKAAA\nzNrosL0hyb03e742yZXb2qaq9kxyQJKrp8/XJnlnkmd39xcH1woAADM1OmxfmOS+VXVkVe2V5JQk\n5yzb5pxMToBMkqck+WB3d1UdmOS9SV7c3f8wuE4AAJi5oWF7Ogf7tCTnJbkkyVu7++KqenlVPWG6\n2ZlJDqqqy5L8ZpJbLg94WpL7JPl/qurT0z8/MrJeAACYpT1Hv0B3n5vk3GVtL93s8XVJnrqV/f5H\nkv8xuj4AABjFHSQBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEA\nYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQ\nYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEb\nAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAA\nBhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYRtgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGAQYRsAAAYR\ntgEAYBBhGwAABhG2AQBgEGEbAAAGEbYBAGCQ4WG7qk6qqkur6rKqOn0r6/euqrdM119QVUdstu7F\n0/ZLq+oxo2sFAIBZGhq2q2qPJK9N8tgkRyV5elUdtWyz5ye5prvvk+RVSV4x3feoJKckuX+Sk5L8\n0bQ/AABYFUYf2T4+yWXdfXl335Dk7CQnL9vm5CRvmD5+e5JHVFVN28/u7uu7+0tJLpv2BwAAq0J1\n97jOq56S5KTu/uXp82clOaG7T9tsm89Nt9kwff7FJCckeVmSj3X3m6btZyb56+5++7LXODXJqdOn\nP57k0mED+lcHJ/nGLnidXWWRxrNIY0kWazyLNJZkscazSGNJFms8izSWZLHGs0hjSRZrPLtqLD/a\n3Wu2t9Geg4uorbQtT/fb2mZH9k13n5HkjJWXtvOq6qLuPm5XvuZIizSeRRpLsljjWaSxJIs1nkUa\nS7JY41mksSSLNZ5FGkuyWOPZ3cYyehrJhiT33uz52iRXbmubqtozyQFJrt7BfQEAYLc1OmxfmOS+\nVXVkVe2VyQmP5yzb5pwkz5k+fkqSD/Zkbss5SU6ZXq3kyCT3TfLxwfUCAMDMDJ1G0t03VtVpSc5L\nskeS13X3xVX18iQXdfc5Sc5MclZVXZbJEe1TpvteXFVvTfL5JDcmeWF33zSy3hXYpdNWdoFFGs8i\njSVZrPEs0liSxRrPIo0lWazxLNJYksUazyKNJVms8exWYxl6giQAANyRuYMkAAAMImwDAMAgwjYA\nAAwy+jrbC6Wq7pGku/uaedfCv6qqQ5Iclsl12K/s7q/PuaTbZdHeZ4syngV8ny3MeBZpLLfwcwOL\nwwmS21FVhydZn+QRSb6Zyc127pbkg0lO7+4vz6+6nbcIH4BVdUySP87k2uwbp81rM/l3+k/d/cl5\n1bZSi/Y+W6TxLNL7LFms8SzSWBI/N7CohO3tqKqPJvmDJG+/5dKDVbVHkqcm+Y3ufsg861upRfoA\nrKpPJ/mV7r5gWftDkvxJdx89n8pWbgHfZwsznkV6nyWLNZ5FGkvi52Z3V1UHJDkpmx2oSnJed39z\nroXtpKr6d0lOzpbjOae7L5lrYTthdx+LOdvbd3B3v2Xza3x3903dfXaSg+ZY1856fZJf7+77dfcj\np3/+XZLfSPJn8y1txfZd/kGeJN39sST7zqGe22PR3meLNJ5Fep8lizWeRRpL4udmt1VVz07yySQP\nT3LXTMbwM0k+MV23qlTVf01ydibfnnw8k5sQVpK/qKrT51nbSq2GsTiyvR1VdXYmN9t5Q5Irps33\nzuSulwd393+cV207o6r+pbvvu411l3X3fXZ1TTurql6d5N8meWO2/Ld5dpIvdfdp86ptpRbwfbYw\n41mk91myWONZpLEkfm52Z1V1aZITlh/Frqq7J7mgu39sPpXtnKr65yT37+4fLGvfK8nF28oJu6PV\nMBZhezum/1jPz79+PVGZfHC8O8mZ3X39HMtbsQX8AHxstvy32ZDJV0fnzrWwFVrA99mijWch3me3\nWKTxLNhY/NzspqaB7sHd/a1l7QdkckfsuQe6laiqLyR5THd/ZVn7jyZ5f3f/+HwqW7nVMBZh+w5o\nkT4AAWC0qnpOkpcmeX/+9UDV4UkeleR3uvv1cyptp1TVSUlek+RfsuV47pPktO5+37xqW6nVMBZh\n+3aoqsd193vmXQc/rKpO7e4z5l3HLCza+2yRxrNI77NkscazSGNJ/NzsDqZTRh6TLQ9UnbdaL89Y\nVXdKcny2HM+Fm58zsFrs7mNGoQLAAAAJqElEQVRxne3b58FJFuLDL1m9H4DbUPMuYIYW6n2WxRrP\nIr3PksUazyKNJfFzM3fTUH32vOuYle6+OcnH5l3HLOzuY3Fkewfs7peUmZWq+pXu/pN517ES03+b\nwzI5QeXazdpP2h2+OlqJqjo+k5tYXFhVR2VyiakvLMr0nqp6Y3evurP2l6uqh2VyBOVz3f3+edez\nUlV1QpJLuvvbVXWXJKcnOTbJ55P87vI5qbuzqvq1JO/s7iu2u/EqMJ2zfUom9z74m6p6RpKHJrkk\nyRnLTwDb3VXVv03ypEzOC7oxk6/5/2I1vcd2RFWd0d2nzruOWamq93T34+ZdxyzsLmMRtrdjekmZ\np2fy2+yGafPaTD4Qz+7u35tXbbNWVb/U3avm8n/T/2hfmMl/RMdkcknDd03XfbK7j51nfStRVb+V\n5LGZfNv0gSQnJDk/ySMz+Zryf86vupWrqnOWN2VymawPJkl3P2GXF7WTqurj3X389PELMnnPvTPJ\no5O8e7V9BlTVxUmO7u4bq+qMJN9L8vZMbqRydHf/wlwLXIGq+laS7yb5YpK/SPK27t4036p2XlX9\neSafAXfN5N4H+yX5y0z+bdLdz51bcSs0/Xx+fJIPJfm5JJ9Ock0m4fs/dff586tutqrqJ7v7E/Ou\nY1aq6p7d/bV51zELu8tYhO3tWA2XlJmVqvpqdx8+7zp2VFV9NslPdfe1VXVEJoHhrO7+w6r6VHc/\naK4FrsB0LMck2TvJUpK1mx15vKC7HzjXAleoqj6ZyZHSP83k26DKJAydkiTd/aH5Vbcym7+XqurC\nJD/X3Zuqat8kH+vuB8y3wpWpqku6+37Tx1v8UlpVn+7uY+ZX3cpU1aeS/GQmv5Q+LckTknwik/fa\nX3b3d+ZY3opV1We6+4FVtWcmNx27V3ffVFWV5J9W0+fALZ9p0/rvmuTc7n54Te6S+a7V9PkMt5eb\n2mzfzUnutZX2e07XrSpV9Zlt/PlskkPmXd8K7XHL1JGe3Mb44UkeW1W/n9U3J/DG6c0rvpfki939\n7STp7u9nFb7PkhyXSeh5SZJvTY9ifb+7P7SagvbUnarq7lV1UCYHKDYlSXd/N5Ovxlebz1XVL00f\n/1NVHZckVfVjSVbVNIVMpl3d3N3v7+7nZ/JZ/UeZTMG6fL6l7ZQ7TQ/k7J/J0e0Dpu17J7nz3Kra\nebecF7Z3JmNKd381q3AsVXVAVf1eVX2hqq6a/rlk2nbgvOubpar663nXsBJVdbeq+l9VddZ06tXm\n6/5oXnVtzgmS2/cbSf62qrZ6SZm5VbXzDsnkbOrlZ09Xkn/c9eXcLktVdUx3fzpJpke4H5fkdUlW\n1dHGJDdU1V2nYfsnb2mcXsN11YXt6ckqr6qqt02XX8/q/bw5IJNfHCpJV9Wh3b1UVftl9f1SlyS/\nnOQPq+q/J/lGko9W1RWZfL798lwrW7kt/v6n30Cek+Sc6bdCq82ZSb6QZI9MflF9W1VdnuQhWX0n\n5v1pkgur6mNJTkzyiiSpqjWZ3LhntXlrJtPgHt7dS0lSVYdmcsOht2VyCcBVo6q2Nc2yMvmWdTX5\ns0zOB3hHkudV1ZOTPGN6XfqHzLWyKdNIdsDufkmZlaiqM5P8WXd/ZCvr3tzdz9jKbrulqlqbyRHh\npa2s++nu/oc5lLVTqmrvrd2woqoOTnLP7v7sHMqamar6+SQ/3d3/bd61zMr0q/FDuvtL865lZ1TV\n/kn+TSa/BG3o7q/PuaQVq6of6+5/nncds1RV90qS7r5yesT0kUm+2t0fn29lK1dV909yv0xOJv7C\nvOu5Parq0m3dHOW21u2uquqmTObTb+2AwUO6e9X8srp8+ltVvSST8wSekOQDu8P5W8I2AMBtqKr3\nJ/mbJG+45RfTqjokyXOTPKq7HznH8lasqj6X5End/S9bWXdFd997DmXtlKq6JJNz627erO05SdYl\n2a+7f3RuxU2Zsw0AcNueluSgJB+qqqur6upMrhh1jyRPnWdhO+ll2XYGfNEurGMW3p3kZzdv6O43\nJPnPSW6YS0XLOLINALCTVttlc7dnkcazu4xF2AYA2Emr7bK527NI49ldxrJarw4AALBLVNVntrUq\nq++yuQs1ntUwFmEbAOC2LdJlc5PFGs9uPxZhGwDgtr0nkytbfHr5iqo6f9eXc7st0nh2+7GYsw0A\nAIO49B8AAAwibAMAwCDCNgAADCJsA7DLVNVWrw5QVa+vqqfs6noARhO2ARZEVd2uK0xV1R6zqmVb\nuvuho18DYHcibAPMSVUdUVWf2+z5f6mql1XVr1XV56vqM1V19nTdvlX1uqq6sKo+VVUnT9ufW1Vv\nq6p3J3n/Nl7nTlX1R1V1cVW9p6rOveUoclV9uapeWlUfSfLUqjqmqj42fe13VtXdp9udX1XHTR8f\nXFVf3uz131VV76uqS6vqt7Yz5muny6qq10zH+d4kP3L7/jYBdk+usw2w+zk9yZHdfX1VHThte0mS\nD3b386ZtH6+qv5mu+6kkD+zuq7fR3y8kOSLJAzIJtZcked1m66/r7oclt96N7UXd/aGqenmS30ry\nG9up9/gkP5Hke0kurKr3dvdF29nnSUl+fFrTIUk+v6wmgIXgyDbA7uczSf68qp6Z5MZp26OTnF5V\nn05yfpJ9khw+XfeB2wjaSfKwJG/r7pu7eynJ3y1b/5YkqaoDkhzY3R+atr8hyYk7UO8Huvuq7v5+\nkr+cvt72nJjkL7r7pu6+MskHd2AfgFVH2AaYnxuz5efwPtPlzyd5bZKfTPKJ6VzsSvLk7j5m+ufw\n7r5kuv13t/M6tZ3129t/ea37LFu3/O5oO3q3NHdVAxaesA0wP19P8iNVdVBV7Z3kcZl8Lt+7u/8u\nybokBybZL8l5SV5UVZUkVfWgFbzOR5I8eTp3+5AkD9/aRt39rSTXVNW/nzY9K8ktR7m/nEn4T5Ll\nVw15VFXdo6rukuSJSf5hB2r6cJJTqmqPqrpnkp/Z0cEArCbmbAPMSXf/YDov+oIkX0ryhSR7JHnT\ndEpHJXlVd3+zqn4nyR8k+cw0cH85k3C+I96R5BFJPpfkn6ev961tbPucJH9cVXdNcnmSX5q2vzLJ\nW6vqWfnhKR8fSXJWkvskefMOzNdOkncm+dkkn53W9KHb3hxgdapu3+IBLLqq2q+7r62qg5J8PMlP\nT+dv395+n5vkuO4+7fb2BbCIHNkGuGN4z/QqJnsl+Z1ZBG0Ats+RbYAFUVUPyGQ6x+au7+4TdnEd\nByX5262sekR3X7UrawGYN2EbAAAGcTUSAAAYRNgGAIBBhG0AABhE2AYAgEH+f4B0ixqwD042AAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train[['user_group_id','is_click']].groupby('user_group_id').mean().plot.bar()\n",
    "sns.barplot('user_group_id', 'is_click', data = train)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from the value counts for the is_click, taking other features into consideration really helped which are pretty much balanced"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing for Training and Test data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracting Datetime features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df=train_log.copy()\n",
    "df.DateTime=pd.to_datetime(df.DateTime)\n",
    "df['date']=df.DateTime.dt.day\n",
    "df['hour']=df.DateTime.dt.hour\n",
    "df['minute']=df.DateTime.dt.minute\n",
    "df.drop('DateTime',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dftest=test_log.copy()\n",
    "dftest.DateTime=pd.to_datetime(dftest.DateTime)\n",
    "dftest['date']=dftest.DateTime.dt.day\n",
    "dftest['hour']=dftest.DateTime.dt.hour\n",
    "dftest['minute']=dftest.DateTime.dt.minute\n",
    "dftest.drop('DateTime',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Converting **webpage_id, campaign_id** and **user_id** as Categorical variable\n",
    "* Converting if the value of **product_category_2** is null or not and dropping it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.campaign_id=df.campaign_id.astype(np.object)\n",
    "df.webpage_id=df.webpage_id.astype(np.object)\n",
    "df.user_id=df.user_id.astype(np.object)\n",
    "import math\n",
    "def nanRet(val):\n",
    "    if math.isnan(val):\n",
    "        return 0\n",
    "    else:\n",
    "        return 1\n",
    "df['is_prod2']=df.product_category_2.apply(nanRet)\n",
    "#or change into buckets\n",
    "df.drop('product_category_2',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dftest.campaign_id=dftest.campaign_id.astype(np.object)\n",
    "dftest.webpage_id=dftest.webpage_id.astype(np.object)\n",
    "dftest.user_id=dftest.user_id.astype(np.object)\n",
    "import math\n",
    "def nanRet(val):\n",
    "    if math.isnan(val):\n",
    "        return 0\n",
    "    else:\n",
    "        return 1\n",
    "dftest['is_prod2']=dftest.product_category_2.apply(nanRet)\n",
    "#or change into buckets\n",
    "dftest.drop('product_category_2',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* **user_group_id, gender, age_level and user_depth** contains similar features as the NaN values in them are also in sync\n",
    "* Therefore Imputing values in them which is not present earlier\n",
    "* Creating a new feature combining age_level & user_depth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.user_group_id.fillna(13.0,inplace=True)\n",
    "df.age_level.fillna(7.0,inplace=True)\n",
    "df.user_depth.fillna(4.0,inplace=True)\n",
    "df.city_development_index.fillna(5.0,inplace=True)\n",
    "df['agelevel_userdepth']=df.age_level*df.user_depth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dftest.user_group_id.fillna(13.0,inplace=True)\n",
    "dftest.age_level.fillna(7.0,inplace=True)\n",
    "dftest.user_depth.fillna(4.0,inplace=True)\n",
    "dftest.city_development_index.fillna(5.0,inplace=True)\n",
    "dftest['agelevel_userdepth']=dftest.age_level*df.user_depth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Imputing **gender** values as 'Others'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.gender.fillna('Others',inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dftest.gender.fillna('Others',inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filling the NaN values for **count_view** and **count_interest** as 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 142,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df.count_view.fillna(0,inplace=True)\n",
    "df.count_interest.fillna(0,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 143,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dftest.count_view.fillna(0,inplace=True)\n",
    "dftest.count_interest.fillna(0,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 463291 entries, 0 to 463290\n",
      "Data columns (total 20 columns):\n",
      "session_id                463291 non-null int64\n",
      "user_id                   463291 non-null object\n",
      "product                   463291 non-null object\n",
      "campaign_id               463291 non-null object\n",
      "webpage_id                463291 non-null object\n",
      "product_category_1        463291 non-null int64\n",
      "user_group_id             463291 non-null float64\n",
      "gender                    463291 non-null object\n",
      "age_level                 463291 non-null float64\n",
      "user_depth                463291 non-null float64\n",
      "city_development_index    463291 non-null float64\n",
      "var_1                     463291 non-null int64\n",
      "is_click                  463291 non-null int64\n",
      "count_view                463291 non-null float64\n",
      "count_interest            463291 non-null float64\n",
      "date                      463291 non-null int64\n",
      "hour                      463291 non-null int64\n",
      "minute                    463291 non-null int64\n",
      "is_prod2                  463291 non-null int64\n",
      "agelevel_userdepth        463291 non-null float64\n",
      "dtypes: float64(7), int64(8), object(5)\n",
      "memory usage: 94.2+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 145,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 128858 entries, 0 to 128857\n",
      "Data columns (total 19 columns):\n",
      "session_id                128858 non-null int64\n",
      "user_id                   128858 non-null object\n",
      "product                   128858 non-null object\n",
      "campaign_id               128858 non-null object\n",
      "webpage_id                128858 non-null object\n",
      "product_category_1        128858 non-null int64\n",
      "user_group_id             128858 non-null float64\n",
      "gender                    128858 non-null object\n",
      "age_level                 128858 non-null float64\n",
      "user_depth                128858 non-null float64\n",
      "city_development_index    128858 non-null float64\n",
      "var_1                     128858 non-null int64\n",
      "count_view                128858 non-null float64\n",
      "count_interest            128858 non-null float64\n",
      "date                      128858 non-null int64\n",
      "hour                      128858 non-null int64\n",
      "minute                    128858 non-null int64\n",
      "is_prod2                  128858 non-null int64\n",
      "agelevel_userdepth        128858 non-null float64\n",
      "dtypes: float64(7), int64(7), object(5)\n",
      "memory usage: 24.7+ MB\n"
     ]
    }
   ],
   "source": [
    "dftest.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x2eea41de9e8>"
      ]
     },
     "execution_count": 272,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvQAAAJFCAYAAACySmPGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xu8HXV97//XewfCRRC8H1QQinhB\nsChUxAuioj9srUhFxSqKeJraFqltqUePreXosUetRxQvtbEHg4iCgCAqioJclIsSboEgKAWqESoq\nFwlIIMnn98eaxLW2+5Zk78y6vJ485rFnfec73/nM2iH5rM/6zkyqCkmSJEmDaaztACRJkiStPxN6\nSZIkaYCZ0EuSJEkDzIRekiRJGmAm9JIkSdIAM6GXJEmSBpgJvSRJkrSRJDkuye1Jrp1ke5Icm+TG\nJEuSPHO6MU3oJUmSpI1nEXDAFNtfBuzSLAuAf51uQBN6SZIkaSOpqguBO6bociDwueq4FNg2yXZT\njWlCL0mSJPWPxwE/7Xq9rGmb1CZzGo4GXl7y+Go7hjV+9pWL2w6hx68fuKvtENZaWSvbDqHHlps8\npO0Qeoylf2oXD6xa0XYIPTabt3nbIfS4/Tf/1XYIa/XbezN/bH7bIfTot//P++nvwbseuLPtEHps\nMW+LtkPo8dRtfz9txwBzlOOc87M/pzNNZo2FVbVwHUeZ6P2ZMlYTekmSJGkWNMn7uibw4y0Dtu96\n/Xjg1ql26J+ylSRJkrSxJLO/zI4zgTc2d7t5NnB3Vd021Q5W6CVJkjR6WiprJ/kisB/wyCTLgH8C\nNgWoqk8DZwF/CNwI3Ae8eboxTeglSZKkjaSqXjfN9gL+al3GNKGXJEnS6Jm9KTKtcw69JEmSNMCs\n0EuSJGn0DE+B3gq9JEmSNMis0EuSJGn0DNEcehN6SZIkjZ4hmqcyRKciSZIkjR4T+j6S5LFJTp3l\nMc9Ksu0E7UcnOWo2jyVJkjQw+vdJsevMKTd9pKpuBQ6e5TH/cDbHkyRJUn+xQr+ekjwkydeTXJ3k\n2iSvTbJnkguSXJ7k7CTbNX2PTHJdkiVJTmraXpDkqma5MsnWSXZMcm2zffMkn01yTbP9hU37YUm+\nnOSbSX6c5EPTxHlLkkc26+9OckOSc4AnT7HPgiSLkyxm2b2z9I5JkiT1kczB0hIr9OvvAODWqvoj\ngCTbAN8ADqyqXyR5LfB+4HDgncBOVbWia/rLUcBfVdVFSbYC7h83/l8BVNXuSZ4CfCvJk5ptewDP\nAFYANyT5eFX9dKpgk+wJHNLstwlwBXD5RH2raiGwECAveXzN7O2QJEkaIGPDc5cbK/Tr7xpg/yQf\nTPJ8YHtgN+DbSa4C/gF4fNN3CXBikjcAK5u2i4CPJDkS2LaqVvYOz/OAEwCq6nrgP4E1Cf25VXV3\nVd0PXAc8YQbxPh84varuq6pfA2eu+ylLkiSp35jQr6eq+hGwJ53E/v8ArwKWVtUezbJ7Vb206f5H\nwCeb/pcn2aSqPgD8d2AL4NKmCt9tqo+NK7rWVzHzb1qstkuSJMFQTbkxoV9PSR4L3FdVnwc+DOwN\nPCrJPs32TZM8LckYsH1VnQe8A9gW2CrJzlV1TVV9EFgMjE/oLwRe34z1JGAH4IYNCPlC4KAkWyTZ\nGvjjDRhLkiRJfcI59Otvd+BfkqwGHgT+gs50mmOb+fSbAB8FfgR8vmkLcExV3ZXkfc2FrqvoTJv5\nBrBd1/ifAj6d5Jpm3MOaOfjrFWxVXZHkZOAqOtN3vrteA0mSJA2DIXpSbKqchaHJ9dNFsT/7ysVt\nh9Dj1w/c1XYIa638nUsw2rXlJg9pO4QeY+mfLyMfWLVi+k4b0WbzNm87hB63/+a/2g5hrX57b+aP\nzW87hB799v95P/09eNcDd7YdQo8t5m3Rdgg9nrrt7/dFJp2Df2/Wc5w69aZWzq1//pWTJEmStM6c\ncjMkknwf2Gxc86FVdU0b8UiSJPW1IbptpQn9kKiqvduOQZIkSRufCb0kSZJGz/AU6J1DL0mSJA0y\nK/SSJEkaPUN020oTekmSJI0eL4rVqOine78/7sDntB1Cj6WnfK3tENa6f9X9bYfQ46GbbtN2CD02\nGdu07RDWWv7gPW2H0GPzPrs/dT/Fs2kf/bkB+Nm9y9oOocfuj9ij7RB63Pvg8rZD6FuralXbIWiO\nmdBLkiRp9AxPgd6LYiVJkqRBZoVekiRJo8eLYiVJkqQBNjz5vFNuJEmSpEFmhV6SJEmjZ4huW2mF\nXpIkSRpgVuglSZI0eoanQG9CL0mSpBE0RHe5ccqNJEmSNMCs0EuSJGn0DFFZe4hORZIkSRo9Q5PQ\nJ9kxybXrue9+SZ4zy/H8z9kcbzYk2TfJFUlWJjm47XgkSZJak8z+0pK+T+iTzNsIh9kPmNWEHpjz\nhD7Juk6Z+glwGPCF2Y9GkiRJbWg1oW+q6tcnOT7JkiSnJtkyyS1J3pPke8Crk+yR5NKmz+lJHtbs\nv2eSq5NcAvxV17iHJflE1+uvJdmvWT+gqVJfneTcJDsCbwX+JslVSZ4/SayPaY59dbM8p2k/I8nl\nSZYmWdC0fQDYohnvxKbtDUl+0LT925oPKknekuRHSc5P8pk1cSd5QhPfkubnDk37oiQfSXIe8C9J\nfpzkUc22sSQ3JnnkROdQVbdU1RJg9TS/lwVJFidZ/PnjzP0lSdIQyhwsLemHi2KfDLylqi5Kchzw\nl037/VX1PIAkS4C3VdUFSd4L/BPwduCzXe3/Mt2BmsT3M8C+VXVzkodX1R1JPg0sr6oPT7H7scAF\nVXVQk4xv1bQf3oyxBXBZktOq6p1JjqiqPZrjPhV4LfDcqnowyaeA1yc5B/hH4JnAPcB3gKubcT8B\nfK6qjk9yeHP8VzbbngTsX1WrktwFvB74KLA/cHVV/XK692IqVbUQWAhw630/qQ0ZS5IkqS9528pZ\n9dOquqhZ/zzwvGb9ZIAk2wDbVtUFTfvxwL4TtJ8wg2M9G7iwqm4GqKo71iHOFwH/2uy3qqrubtqP\nTHI1cCmwPbDLBPu+GNiTTsJ/VfP694Bn0fmQcEdVPQic0rXPPvx2aswJ/PZ9ATilqlY168cBb2zW\nD6fzIUeSJEkjoh8q9OMrwGte3zvNfplg3zVW0vthZfMZ7LPOmmk8+wP7VNV9Sc7vOlZPV+D4qnrX\nuP0PWofDdce99r2pqp8m+XmSFwF706nWS5IkaSr9UNaeJf1wKjsk2adZfx3wve6NTSX8zq657YfS\nqWrfBdydZE3lujuRvQXYo5lTvj2dSjjAJcALkuwEkOThTfs9wNbTxHku8BfNfvOSPBTYBrizSeaf\nQucbgDUeTLJp174HJ3n0muMmeQLwgyaehzUXuL6qa/+LgUO6zq3nfRnn3+l8u/Glrsq9JEmSRkA/\nJPQ/BN7UzJN/OM20lnHeROcC0CXAHsB7m/Y3A59sLor9TVf/i4CbgWuADwNXAFTVL4AFwJebaTIn\nN/2/Chw01UWxwF8DL0xyDXA58DTgm8AmTVzvozPtZo2FwJIkJ1bVdcA/AN9q+n4b2K6qfgb8M/B9\n4BzgOmDtVB7gzU3/Q5vjT+ZMOnP6p5xuk+QPkiwDXg38W5KlU/WXJEkaWkN028pUtXfNY3OHma9V\n1W6tBdGyJFtV1fKmQn86cFxVnb6OY+wFHFNVk30YWW/9dFHs4w6c7TuLbpilp3yt7RDWum/lfW2H\n0OPRmz+m7RB6bDK26fSdNpI7VmzQNeuz7qGbbtN2CD3ufuCutkNYa9M++nMD8LN7l7UdQo/dH7FH\n2yH0uPfB5W2HsNadD6zLJXpzb/7Y/LZD6LHbw57ZF1ej5i+fNus5Tn1qaSvn1g8V+lF3dHOh7LV0\nvlU4Y112TvJO4DTgXdP1lSRJ0vBp9aLYqroF6KvqfJJ305mS0u2Uqnr/XByvqo7awP0/AHygu21j\nn4MkSdLAGeuLLwpmRT/c5aavNEnvQCe+w3AOkiRJmhkTekmSJI2eIXqwlAm9JEmSRs/w5PNeFCtJ\nkiQNMiv0kiRJGjlxyo1Gxa/76J7Q/XTfd4CnvfrlbYew1sOfvUPbIfR4/yGvazuEHjtvs1PbIay1\ny7ZPbjuEHqv67OHSm83brO0Q1kqffR+/8za7tB1Cj+UP3tN2CH1r2/kPazuEHnt/+PC2Q+jxi/91\nUdshDB0TekmSJI0cK/SSJEnSABuifN6LYiVJkqRBZoVekiRJI2dsiEr0VuglSZKkAWaFXpIkSSNn\nmC6KtUIvSZIkDTAr9JIkSRo5w1ShN6GXJEnSyBmmhN4pN5IkSdIAs0IvSZKkkTNEBXor9JIkSdLG\nkuSAJDckuTHJOyfYvkOS85JcmWRJkj+cbkwT+iGX5OJJ2hclOXhjxyNJktQPksz6MoNjzgM+CbwM\n2BV4XZJdx3X7B+BLVfUM4BDgU9ON65SbDZBkk6pauQH7z6uqVbMZ03hV9Zy5HF+SJGkQtXRR7LOA\nG6vqpiaGk4ADgeu6+hTw0GZ9G+DW6QYdqQp9kh2TXNv1+qgkRyc5Msl1zdcaJzXbHpLkuCSXNV95\nHNi0H5bklCRfBb41yXHGknwqydIkX0ty1ppqeJJbkrwnyfeAVyfZI8mlzbFPT/Kwpt/5SfZq1h+Z\n5Jau438lyTebr2v+aZpzXt78TJJPNOf5deDRU+yzIMniJIu/tOjUmb69kiRJmtrjgJ92vV7WtHU7\nGnhDkmXAWcDbphvUCn3HO4GdqmpFkm2btncD36mqw5u2HyQ5p9m2D/D0qrpjkvH+BNgR2J1O4vxD\n4Liu7fdX1fMAkiwB3lZVFyR5L/BPwNunifdZwG7AfcBlSb5eVYun2ecg4MlNTI+h80nwuIk6VtVC\nYCHA9XctqWnGlSRJGjhh9iv0SRYAC7qaFjZ51W8P+7vG51qvAxZV1f9Nsg9wQpLdqmr1ZMc1oe9Y\nApyY5AzgjKbtpcArkhzVvN4c2KFZ//YUyTzA84BTmjf+v5KcN277yQBJtgG2raoLmvbjgVNmEO+3\nq+pXzRhfbo43XUK/L/DFZorPrUm+M4PjSJIkaYa6i6KTWAZs3/X68fzulJq3AAc0412SZHPgkcDt\nkw06UlNugJX0nvPmzc8/onOBwp7A5Uk2ofMJ6lVVtUez7FBVP2z63zvNcab7yDfd/uNj3XzctvGf\n5GZaRbfaLkmSRDsXxQKXAbsk2SnJfDoXvZ45rs9PgBc3MT6VTh74i6kGHbWE/ufAo5M8IslmwMvp\nvAfbV9V5wDuAbYGtgLOBt6X57SR5xjoc53vAq5q59I8B9puoU1XdDdyZ5PlN06HAmmr9LXQ+YACM\nvxvNS5I8PMkWwCuBi2YQ04XAIUnmJdkOeOFMT0aSJGnYJLO/TKe5mcoRdPLMH9K5m83SJO9N8oqm\n298Bf5bkauCLwGFVNWVRdqSm3FTVg8089e8DNwPXA/OAzzfTXwIcU1V3JXkf8FFgSZPU30LnA8BM\nnEbnk9W1wI+a4909Sd83AZ9OsiVwE/Dmpv3DwJeSHAqMnx7zPeAE4InAF2Ywfx7gdOBFwDVNTBdM\n3V2SJEmzrarOonOxa3fbe7rWrwOeuy5jjlRCD1BVxwLHzqDfb4A/n6B9EbBomn1XJzmqqpYneQTw\nAzqJNFW147i+VwHPnmCM64GndzX9Q9f67VV1xHTn0IyzVfOz6HwilCRJGnljQ/So2JFL6DeirzV3\nx5kPvK+q/qvtgCRJkjR8TOg3QJLd6Ux96baiqvauqv3m4pgTfUPQfAtw7gTdX7zmbjiSJEn6rZYe\nLDUnTOg3QFVdA+zRB3H8qh/ikCRJ0sZnQi9JkqSRY4VekiRJGmBDlM+P3H3oJUmSpKFihV6SJEkj\nxyk3Ghkra2XbIax1/6r72w6hx8OfvUPbIax1x6U/aTuEHite/WDbIfTYev7WbYew1q8fmOwZc+14\n6Pxt2g6hx+pa3XYIaxVTPphxo9ukNm07BM1Qv/3Z+eXS29oOQXPMhF6SJEkjxwq9JEmSNMCGKaH3\nolhJkiRpgFmhlyRJ0sixQi9JkiSpL1ihlyRJ0sgZogK9Cb0kSZJGj1NuJEmSJPUFK/SSJEkaOVbo\nJUmSJPUFK/SSJEkaOWNW6CVJkiT1AxP6AZDksCSf6PcxJUmSBkUy+0tbnHIjSZKkkeNFsZpQkjOS\nXJ5kaZIFTdtbkvwoyflJPrOmKp7kUUlOS3JZszx3hsf4nf2SjCW5Jcm2Xf1uTPKY9TlOkgVJFidZ\nfMqiL6/v2yFJkqSNwAr97Dq8qu5IsgVwWZKvA/8IPBO4B/gOcHXT92PAMVX1vSQ7AGcDT53BMX5n\nv6p6apKvAAcBn02yN3BLVf08yRfW9ThVtRBYCHDtnVfUur0FkiRJ/S8MT4XehH52HZnkoGZ9e+BQ\n4IKqugMgySnAk5rt+wO7dn3d89AkW1fVPdMcY8L9gJOB9wCfBQ5pXk/VX5IkSUPAhH6WJNmPTvK8\nT1Xdl+R84AYmr4aPNX1/s46HmnC/JJcAT0zyKOCVwP+epv86HlaSJGl4DFMu5Bz62bMNcGeTzD8F\neDawJfCCJA9Lsgnwqq7+3wKOWPMiyR4zPM6E+1VVAacDHwF+WFW/2sDjSJIkDa0ks760xYR+9nwT\n2CTJEuB9wKXAz4B/Br4PnANcB9zd9D8S2CvJkiTXAW+d4XGm2u9k4A38drrNhhxHkiRJA8ApN7Ok\nqlYALxvfnmRxVS1sKvSn06mYU1W/BF47w7EXAYum26+qFkPvFR6T9e8eU5IkadQM0YwbK/QbwdFJ\nrgKuBW4Gzmg5HkmSJA0RK/RzrKqOmmnfJG8G/npc80VV9VezG5UkSdJoG6aLYk3o+0hVfZbObScl\nSZI0h4YpoXfKjSRJkjTArNBLkiRp5FihlyRJktQXrNBLkiRp5AxRgd6EXlPbcpOHtB3CWg/ddJu2\nQ+jx/kNe13YIa6149YNth9Dj7X/3kbZD6LH8Gz9sO4S1/uPXN7QdQo/VtbrtEHpsOm9+2yFohkJ/\nZUNFtR1C39r34H3aDkFzzIRekiRJI2eY5tCb0EuSJGnkDFNC70WxkiRJ0gCzQi9JkqSRY4VekiRJ\nUl+wQi9JkqSRM0QFehN6SZIkjR6n3EiSJEnqC1boJUmSNHKs0EuSJEnqC1boJUmSNHKGqUJvQi9J\nkqSRM0T5vFNu+l2SRUkOXs9990vynNkYS5IkSf3JCn1LkmxSVSvn+DD7AcuBi+f4OJIkSQNlmKbc\nWKGfoSQ7Jrm26/VRSY5OcmSS65IsSXJSs+0hSY5LclmSK5Mc2LQfluSUJF8FvjXJcZLkE82YXwce\n3bVtzyQXJLk8ydlJtmvaz0/y0SQXJ7k2ybOS7Ai8FfibJFcleX4zzL5Nv5smq9YnWZBkcZLFX/zs\nyRv83kmSJGnuWKHfcO8EdqqqFUm2bdreDXynqg5v2n6Q5Jxm2z7A06vqjknGOwh4MrA78BjgOuC4\nJJsCHwcOrKpfJHkt8H7g8Ga/h1TVc5LsCxxXVbsl+TSwvKo+DJDkLcB2wPOApwBnAqeOD6CqFgIL\nAW6654Zaz/dFkiSpfw1Rhd6EfsMtAU5McgZwRtP2UuAVSY5qXm8O7NCsf3uKZB5gX+CLVbUKuDXJ\nd5r2JwO7Ad9uviKaB9zWtd8XAarqwiQP7fpwMd4ZVbUauC7JY2Z8lpIkSepLJvQzt5LeKUqbNz//\niE4S/grgH5M8DQjwqqq6oXuAJHsD987gWBNVxQMsrap9ZrjPZJX1FePGlCRJGjnOoR9NPwceneQR\nSTYDXk7n/du+qs4D3gFsC2wFnA28Lc2flCTPWIfjXAgckmReM0f+hU37DcCjkuzTjLlp8+Fhjdc2\n7c8D7q6qu4F7gK3X73QlSZKGVzL7S1us0M9QVT2Y5L3A94GbgevpTHv5fJJt6FS7j6mqu5K8D/go\nsKRJ6m+h8wFgJk4HXgRcA/wIuKA5/gPNRazHNsfbpDnG0ma/O5NcDDyU386r/ypwanNR7tvW++Ql\nSZLUt0zo10FVHQscO4N+vwH+fIL2RcCiafYt4IhJtl1FZ3rPRE6rqneN6/8j4OldTd8dt32rqWKR\nJEkaVk65kSRJktQXrNC3JMnuwAnjmldU1d7rOlZV7TcrQUmSJI2IYarQm9C3pKquAfZoOw5JkqRR\nNEwJvVNuJEmSpAFmhV6SJEkjZ4gK9FboJUmSpEFmQi9JkqSRk2TWlxke94AkNyS5Mck7J+nzmiTX\nJVma5AvTjemUG01pLP3zmW+TsU3bDqHHztvs1HYIa209v78eCLz8Gz9sO4QeW73sqW2HsNZvvvmj\ntkPocdt9y9oOoUfon+/AV9WqtkPoscm8/vone+XqlW2H0LfmZV7bIfT48Iv/ou0Q+lIbF8UmmQd8\nEngJsAy4LMmZVXVdV59dgHcBz62qO5M8erpx+ydbkyRJkobbs4Abq+qmqnoAOAk4cFyfPwM+WVV3\nAlTV7dMNakIvSZKkkTMXU26SLEiyuGtZMO6wjwN+2vV6WdPW7UnAk5JclOTSJAdMdy799f2dJEmS\nNKCqaiGwcIouE83zqXGvNwF2AfYDHg98N8luVXXXZIOa0EuSJGnktPRgqWXA9l2vHw/cOkGfS6vq\nQeDmJDfQSfAvm2xQp9xIkiRJG8dlwC5JdkoyHzgEOHNcnzOAFwIkeSSdKTg3TTWoFXpJkiSNnDYK\n9FW1MskRwNnAPOC4qlqa5L3A4qo6s9n20iTXAauAv6+qX001rgm9JEmSRk5LU26oqrOAs8a1vadr\nvYC/bZYZccqNJEmSNMCs0EuSJGnktFWhnwtW6CVJkqQBZoVekiRJI2eYKvQm9JIkSRo5Q5TPr9+U\nmyRvTfLGZv2wJI/d0ECacT6xoePM9ZgbKsmOSf50mj57JTl2Hcc9OslRGxadJEmSBs16Veir6tNd\nLw8DruV3n3Klie0I/Cnwhck6VNViYPHGCkiSJGnUDNOUmxlV6JO8McmSJFcnOWFNNTjJwcBewIlJ\nrkryR0lO79rvJUm+PMW4b07yoyQXAM/tan9UktOSXNYsz00yluSWJNt29bsxyWMm6j/BsZ6Q5Nzm\nPM5NskPTvijJp5N8t4nl5U37YUnOSPLVJDcnOSLJ3ya5MsmlSR7e9Ns5yTeTXN6M8ZSucY9NcnGS\nm5r3CuADwPOb9+tvJnlf9kvytWb96CTHJTm/GefIrn7vTnJDknOAJ3e1TxbTV7q+WfnzJCdOcvwF\nSRYnWfyF406a7NcnSZKkPjBthT7J04B3A8+tql82ieyRAFV1avO0q6OqanE6H3X+b5JHVdUvgDcD\nn51k3O2A/wXsCdwNnAdc2Wz+GHBMVX2vSbzPrqqnJvkKcBDw2SR7A7dU1c+TfGF8f+Cp4w75CeBz\nVXV8ksOBY4FXNtt2BF4A7Aycl+SJTftuwDOAzYEbgf9RVc9IcgzwRuCjwELgrVX14yamTwEvavbf\nDnge8BQ6j/U9FXhn8369fLr3vstT6DwCeGvghiT/CjydzuOCn0Hn93gFcHnTf7KYFgAXJbkZ+Dvg\n2RMdrKoWNmNwy/If1zrEKUmSNBiGqEI/kyk3LwJOrapfAlTVHZN9RVFVleQE4A1JPgvsQyfxncje\nwPlN4k+Sk4EnNdv2B3btOs5Dk2wNnAy8h86HhEOa11P177YP8CfN+gnAh7q2famqVgM/TnITnQQa\n4Lyquge4J8ndwFeb9muApyfZCngOcErXsTfrGveMZtzrkjxmkvdhJr5eVSuAFUluBx4DPB84varu\nA0hyZvNz0piaDz/vofPh6aCqumMDYpIkSRpYwzTlZiYJfYB1qdJ+lk7iez9wSlWtnKLvZOOOAftU\n1W96AkkuAZ6Y5FF0quv/e5r+U8VZk6x3v17R1ba66/VqOu/dGHBXVe0xyTG699+QPzXd46zit7+3\nid6/6WLaHfgVsMEXMkuSJKl9M5lDfy7wmiSPAFgzd7zLPXSmggBQVbfSuUD2H4BFU4z7fWC/JI9I\nsinw6q5t3wKOWPMiyR7N2AWcDnwE+GFV/Wqq/uNcTKeqD/B64Htd217dzNHfGfg94IYp4l6rqn4N\n3Jzk1c1xk+T3p9mt5/3aABcCByXZovk24o+niynJs4CX0Zmmc1SSnWYhDkmSpIEzltlfWjuX6TpU\n1VLg/cAFSa6mk0x3WwR8urnIc4um7UTgp1V13RTj3gYcDVwCnENnDvgaRwJ7NRewXge8tWvbycAb\n+O10m+n6d/d5c5IlwKHAX3dtuwG4APgGnbnn908W9wReD7yleW+WAgdO038JsDKdC4wnvCh2Jqrq\nCjrvwVXAacB3p4opyWbAZ4DDmw9dfwccl2H6vkmSJGkEpVP0nuVBO/d+v7Kq/t+sDz7LkiwCvlZV\np7YdSz/qp4ti549tNn2njWjpHUvaDmGtrefPxpc+s2f3hz+z7RB6bPWy8dfIt+c33/xR2yH0uO2+\nZW2H0LdW1aq2Q+ix2bz++jtw5eqpZtRufLVOs4Pn1ljW6zE/c+YXv/l52yH0+INHPa8vion7n/am\nWf9Dc86rjm/l3Gb9SbFJLgfupVMBliRJkjSHZj2hr6o9x7cl+T69d38BOLSqrpnt46+rqjqsrWMn\n+f+AD45rvrmqDmojHkmSpFExNkSzjmc9oZ9IVe29MY4zaKrqbDr3zJckSdJGNEyXEfbXJC9JkiRJ\n62SjVOglSZKkfjJMVe1hOhdJkiRp5FihlyRJ0sjxoliNjAdWrWg7hLWWP3hP2yH02GXbJ7cdwlq/\nfuDutkPo8R+/ntHDljeafrr3+xYHPKntEHr8+LRvtx1Cj/lj89sOYa1NxzZtO4QeK1b3z9/HAGN+\nyT+peZnXdgg9tthky7ZD6EteFCtJkiSpL1ihlyRJ0sgZpik3VuglSZKkAWaFXpIkSSNnmObQm9BL\nkiRp5AzTNJVhOhdJkiRp5FihlyRJ0sjxolhJkiRJfcEKvSRJkkbOMF0Ua4VekiRJGmBW6CVJkjRy\nhmkOvQm9JEmSRs7wpPNOuRkZSd6f5KdJlrcdiyRJkmaPCf0QScdkv9OvAs/amPFIkiT1q7Fk1pfW\nzqW1I2tSST6Y5C+7Xh+d5J+SnJvkiiTXJDmw2bZjkh8m+RRwBbD9RGNW1aVVddsMj78gyeIki09e\ndMpsnJIkSZLmiHPo+9NJwEd2gF5tAAAgAElEQVSBTzWvXwMcABxTVb9O8kjg0iRnNtufDLy5qv7y\nd4dad1W1EFgI8KO7r63ZGFOSJKmfeFGs5lRVXZnk0UkeCzwKuBO4DTgmyb7AauBxwGOaXf6zqi5t\nJ1pJkqTBM0z3oTeh71+nAgcD/41Oxf71dJL7PavqwSS3AJs3fe9tJUJJkiS1zoS+f50EfAZ4JPAC\nOtNubm+S+RcCT2gzOEmSpEE2TFNuvCi2T1XVUmBr4GfNxawnAnslWUynWn/9uoyX5ENJlgFbJlmW\n5OjZjlmSJEkbnxX6PlZVu3et/xLYZ5Kuu81grHcA75il0CRJkgba8NTnTeglSZI0goZpyo0J/ZBJ\n8n1gs3HNh1bVNW3EI0mSpLllQj9kqmrvtmOQJEnqd8NUofeiWEmSJGmAWaGXJEnSyPHBUpIkSdIA\nc8qNJEmSpL5ghV5T2mze5m2HsNbm87ZoO4Qeq2pV2yGs9dD527QdQo/VtbrtEHrcdt+ytkNY68en\nfbvtEHrs8qqXtB1Cj356f8bSXzWv+WPz2w6hx8rVK9sOoW/1078PAA/ZZKu2Q+hLw1Oft0IvSZIk\nDTQr9JIkSRo5zqGXJEmS1Bes0EuSJGnkDFOF3oRekiRJI2eY7kPvlBtJkiRpgFmhlyRJ0sgZpqr2\nMJ2LJEmSNHKs0EuSJGnkDNMcehN6SZIkjZxhusuNU24kSZKkAWaFXpIkSSPHCr02WJKLZ2mcw5J8\noll/a5I3TtF3vyRfm43jSpIkqT+Y0Lekqp4zB2N+uqo+N9vjSpIkDZsks77M8LgHJLkhyY1J3jlF\nv4OTVJK9phvThL4lSZY3P7dLcmGSq5Jcm+T5U+xzQJIrklyd5NwJth+d5Khm/YlJzmn6XpFk53F9\n/yDJlUl+b4JxFiRZnGTxF447acNPVpIkqc+MkVlfppNkHvBJ4GXArsDrkuw6Qb+tgSOB78/kXJxD\n374/Bc6uqvc3v+QtJ+qU5FHAZ4B9q+rmJA+fZtwTgQ9U1elJNqfz4W37ZqznAB8HDqyqn4zfsaoW\nAgsB/nP5jbWe5yVJkqRezwJurKqbAJKcBBwIXDeu3/uADwFHzWRQK/Ttuwx4c5Kjgd2r6p5J+j0b\nuLCqbgaoqjsmG7D5VPe4qjq96Xt/Vd3XbH4qnWT9jydK5iVJkkbBXEy56Z7l0CwLxh32ccBPu14v\na9q643oGsH1Vzfi6RxP6llXVhcC+wM+AE6a4qDXATKvlU33ncxtwP/CMGQcpSZKkaVXVwqraq2tZ\nOK7LRDna2vwuyRhwDPB363JcE/qWJXkCcHtVfQb4f8AzJ+l6CfCCJDs1+0065aaqfg0sS/LKpu9m\nSdZM5bkL+CPgn5PsNztnIUmSNFjGkllfZmAZzRToxuOBW7tebw3sBpyf5BY6MzTOnO7CWBP69u0H\nXJXkSuBVwMcm6lRVvwAWAF9OcjVw8jTjHgocmWQJcDHw37rG+jnwx8Ank+y9wWcgSZKkmbgM2CXJ\nTknmA4cAZ67ZWFV3V9Ujq2rHqtoRuBR4RVUtnmpQL4ptSVVt1fw8Hjh+hvt8A/jGuLZFwKJm/eiu\n9h8DLxo3xE3A+c32nwBPW4/QJUmSBl5mcFea2VZVK5McAZwNzAOOq6qlSd4LLK6qM6ceYWIm9JIk\nSRo5M71v/GyrqrOAs8a1vWeSvvvNZEwT+j6U5PvAZuOaD62qa9qIR5IkSf3LhL4PVZXz2iVJkubQ\nDC9iHQheFCtJkiQNMCv0kiRJGjkZorq2Cb0kSZJGjlNuJEmSJPUFK/Sa0u2/+a+2Q1hr83lbtB1C\nj83mjb8RUXtW1+q2Q+ix6bz5bYfQo417DU9m/lh/vTc/Pu3bbYfQY5dXvaTtENZ6zd+8vO0Qehyz\n37vaDqHH6vTX3zuralXbIaxVVW2H0OOB1SvaDqEvtXXbyrlghV6SJEkaYFboJUmSNHL66dvbDWVC\nL0mSpJHjRbGSJEmS+oIVekmSJI0cL4qVJEmS1Bes0EuSJGnkjA1RXXt4zkSSJEkaQVboJUmSNHKG\naQ69Cb0kSZJGzjAl9E65kSRJkgaYFXpJkiSNnLEhelKsFfo+kOTtSbZcz30vnu14JEmSNDhM6PvD\n24H1Suir6jmzHIskSdLQSzLrS1tM6GcoyRuTLElydZITkjwhyblN27lJdmj6LUpycNd+y5uf+yU5\nP8mpSa5PcmI6jgQeC5yX5LxJjv0XST7U9fqwJB/vHr9Z//sklzUx/a+m7R3NMUhyTJLvNOsvTvL5\nSY63IMniJItP/9yZG/bGSZIk9aGxZNaX1s6ltSMPkCRPA94NvKiqfh/4a+ATwOeq6unAicCxMxjq\nGXSq8bsCvwc8t6qOBW4FXlhVL5xkv1OBP+l6/Vrg5HExvhTYBXgWsAewZ5J9gQuB5zfd9gK2SrIp\n8DzguxMdrKoWVtVeVbXXQW98xQxOS5IkSW0xoZ+ZFwGnVtUvAarqDmAf4AvN9hPoJMjT+UFVLauq\n1cBVwI4zOXhV/QK4KcmzkzwCeDJw0bhuL22WK4ErgKfQSfAvp5Pcbw2sAC6hk9g/n0kSekmSpGGX\nOfivLd7lZmYC1DR91mxfSfNBKZ3JVPO7+qzoWl/Fur3/JwOvAa4HTq+q8fEE+D9V9W/jd0xyC/Bm\n4GJgCfBCYGfgh+twfEmSJPUhK/Qzcy7wmqY6TpKH00mOD2m2vx74XrN+C7Bns34gsOkMxr8H2Hqa\nPl8GXgm8jnHTbRpnA4cn2aqJ8XFJHt1suxA4qvn5XeCtwFUTfCiQJEkaCWMZm/WlLVboZ6CqliZ5\nP3BBklV0prUcCRyX5O+BX9CpgAN8BvhKkh/Q+SBw7wwOsRD4RpLbJptHX1V3JrkO2LWqfjDB9m8l\neSpwSXOV9XLgDcDtdJL4dwOXVNW9Se7H6TaSJGmEDdOTYk3oZ6iqjgeOH9f8ogn6/Rx4dlfTu5r2\n84Hzu/od0bX+ceDjM4jh5RO0bdW1/jHgYxP0OZeubwqq6knTHUuSJEmDwYRekiRJI6fNi1hnmwl9\nn0nyfWCzcc2HVtU1bcQjSZKk/mZC32eqau+2Y5AkSRp2bT4IarZ5lxtJkiRpgFmhlyRJ0shxDr0k\nSZI0wJxyI0mSJKkvWKHXlDabt3nbIay16dhMHrq78fTTV3WFD/2dyqpa1XYIa/Xbn+M2n2w4kdf8\nze88bqM1Xzrma22H0OPDL3hH2yH08O+dweGD4SeWPvv7b0MMz5lIkiRJI8gKvSRJkkZOP33TvqFM\n6CVJkjRyvChWkiRJUl+wQi9JkqSREyv0kiRJkvqBFXpJkiSNnDEvipUkSZIGl1NuJEmSJPUFK/SS\nJEkaOT4pVpIkSVJfsEIvSZKkkTNMF8VaoZ8FSd6eZMtp+pyVZNtp+hyW5LGzG93vHGPHJH86l8eQ\nJEnSxmNCPzveDkyZ0FfVH1bVXdOMcxiwTgl9knX9lmVHwIRekiSNtCSzvrRlZBL6JG9MsiTJ1UlO\nSPKEJOc2becm2aHptyjJwV37LW9+7pfk/CSnJrk+yYnpOJJOEn5ekvOmOP4tSR7ZVMh/mOQzSZYm\n+VaSLZpj7gWcmOSqpm3PJBckuTzJ2Um2a8Y6P8k/J7kA+Oskj0pyWpLLmuW5Tb8XNGNdleTKJFsD\nHwCe37T9zSSxLkiyOMniU4//8qy8/5IkSf0kc/BfW0ZiDn2SpwHvBp5bVb9M8nDgeOBzVXV8ksOB\nY4FXTjPUM4CnAbcCFzXjHZvkb4EXVtUvZxjSLsDrqurPknwJeFVVfT7JEcBRVbU4yabAx4EDq+oX\nSV4LvB84vBlj26p6QXN+XwCOqarvNR9MzgaeChwF/FVVXZRkK+B+4J3NMV4+WXBVtRBYCLDkjsU1\nw3OSJElSC0YioQdeBJy6JuGuqjuS7AP8SbP9BOBDMxjnB1W1DCDJVXSmr3xvPeK5uaquatYvb8YZ\n78nAbsC3m69w5gG3dW0/uWt9f2DXrq96HtpU4y8CPpLkRODLVbVsmB6iIEmStL6GKScalYQ+wHSV\n5jXbV9JMRUrnNz2/q8+KrvVVrP/7N36cLSboE2BpVe0zyRj3dq2PAftU1W/G9flAkq8DfwhcmmT/\n9YxXkiRJfWpU5tCfC7wmySMAmik3FwOHNNtfz28r7bcAezbrBwKbzmD8e4CtZyHO7nFuAB7VfJNA\nkk2bqUMT+RZwxJoXSfZofu5cVddU1QeBxcBTZjFWSZKkgTVGZn1p71xGQFUtpTP//IIkVwMfAY4E\n3pxkCXAo8NdN988AL0jyA2Bveivhk1kIfGOqi2JnaBHw6WY6zzzgYOCDTcxXAc+ZZL8jgb2aC3yv\nA97atL89ybXN/r8BvgEsAVY2FwdPeFGsJEnSsEvGZn1p7VyqvOZRk+uni2Lnj82fvtNGNC/z2g5h\nrZW1su0Qesyft1nbIfTop7/nNp+3edsh9Hhg9QNth9DjXd89pu0Q1vrSMV9rO4QePznjwrZD6LGq\nVrUdQo/VtbrtENZq824nE1mx6v62Q+jxlG2f3hdv0Mn/ccKs/+Pw2p0PbeXcRmUOvSRJkrRWv33w\n2hAm9LMsyfeB8eXJQ6vqmjbikSRJ0nAzoZ9lVbV32zFIkiRpat62UpIkSRpgwzTlZiTuciNJkiQN\nKyv0kiRJGjnDNOXGCr0kSZI0wLwPvaZ0/V1L+uYPyM/uXdZ2CD123maXtkNYy/+Pp7bJWP98Gdlv\n9333+Q6T67f7rO/wyn3bDqHHbWde0nYIPZY/uLztENZafPsP2g6hx7P/22TPpWzHjlvt0hel8dNv\nPmnW//E8aKdDpj23JAcAH6PzENF/r6oPjNv+t8B/B1YCvwAOr6r/nGpMK/SSJEnSRpBkHvBJ4GXA\nrsDrkuw6rtuVwF5V9XTgVOBD041rQi9JkqSRk2TWlxl4FnBjVd1UVQ8AJwEHdneoqvOq6r7m5aXA\n46cbtH++h5YkSZI2krRT134c8NOu18uAqZ5h9BbgG9MNakIvSZIkzYIkC4AFXU0Lq2phd5cJdptw\nLn+SNwB7AS+Y7rgm9JIkSRo5c3HbyiZ5XzhFl2XA9l2vHw/cOr5Tkv2BdwMvqKoV0x3XOfSSJEnS\nxnEZsEuSnZLMBw4BzuzukOQZwL8Br6iq22cyqBV6SZIkjZxMOPtlblXVyiRHAGfTuW3lcVW1NMl7\ngcVVdSbwL8BWwCnNtwg/qapXTDWuCb0kSZJGzlhLT4qtqrOAs8a1vadrff91HdMpN5IkSdIAs0Iv\nSZKkkdPGlJu5YoVekiRJGmBW6AdYkqOB5VX14Um2vxL4UVVdt1EDkyRJ6nNzcdvKtlihH26vBHZt\nOwhJkqR+E8ZmfWmLCf2ASfLuJDckOQd4ctP2Z0kuS3J1ktOSbJnkOcArgH9JclWSnZvlm0kuT/Ld\nJE9p9WQkSZK0wUzoB0iSPek8gOAZwJ8Af9Bs+nJV/UFV/T7wQ+AtVXUxnQcV/H1V7VFV/0HnyWVv\nq6o9gaOAT01ynAVJFidZ/KVFp87xWUmSJG18SWZ9aYtz6AfL84HTq+o+gCRrniy2W5L/DWxL50EE\nZ4/fMclWwHP47UMKADab6CDdjy2+/q4lNZsnIEmSpNllQj94JkqwFwGvrKqrkxwG7DdBnzHgrqra\nY+5CkyRJGgxj3rZSLbkQOCjJFkm2Bv64ad8auC3JpsDru/rf02yjqn4N3Jzk1QDp+P2NF7okSZLm\nggn9AKmqK4CTgauA04DvNpv+Efg+8G3g+q5dTgL+PsmVSXamk+y/JcnVwFLgwI0VuyRJUj9xDr1a\nU1XvB94/waZ/naDvRfzubSsPmIu4JEmSBolPipUkSZLUF6zQS5IkaeT4pFhJkiRJfcEKvSRJkkZO\nhqiubUIvSZKkkTPmlBtJkiRJ/cAKvSRJkkaOt62UJEmS1Bes0GtKW27ykLZDWGv3R+zRdgg9lj94\nT9sh9K1+q3qsXL2y7RDWGuuzOko/vTcAq7O67RDWKqrtEHrcduYlbYfQY7tX7NN2CD366f3Z97H7\ntR1Cj/tX3d92CH1pmG5baUIvSZKkkdNvxacN0V+lIkmSJEnrxAq9JEmSRs4wTbmxQi9JkiQNMCv0\nkiRJGjn9dpOCDTE8ZyJJkiSNICv0kiRJGjnDNIfehF6SJEkjx9tWSpIkSeoLVuglSZI0coZpyo0V\nekmSJGmAmdAPkCQ7Jrm27TgkSZIGXebgv7Y45WbEJdmkqla2HYckSdLG5EWxatO8JJ9JsjTJt5Js\nkWSPJJcmWZLk9CQPA0hyfpK9mvVHJrmlWT8sySlJvgp8q71TkSRJ0oYyoR88uwCfrKqnAXcBrwI+\nB/yPqno6cA3wTzMYZx/gTVX1ovEbkixIsjjJ4i8cd9Ishi5JktQnktlfWuKUm8Fzc1Vd1axfDuwM\nbFtVFzRtxwOnzGCcb1fVHRNtqKqFwEKAnyz/j9rAeCVJkjSHTOgHz4qu9VXAtlP0Xclvv4XZfNy2\ne2czKEmSpEHiHHr1k7uBO5M8v3l9KLCmWn8LsGezfvBGjkuSJKlvJZn1pS1W6IfDm4BPJ9kSuAl4\nc9P+YeBLSQ4FvtNWcJIkSZo7JvQDpKpuAXbrev3hrs3PnqD/9cDTu5r+oWlfBCyaixglSZIGgVNu\nJEmSJPUFK/SSJEkaOcNUoTehlyRJ0shp8yLW2eaUG0mSJGmAWaGXJEnSyBmmKTdW6CVJkqQBZoVe\nkiRJI8cKvSRJkqS+YIVeU1pZK9sOYa17H1zedgiaoaLaDkEDalWtajuEvrW8z/4OvO3MS9oOocd2\nr9in7RDWuvHL57QdQo9hqkTPpmG6y40JvSRJkkbOMH3QccqNJEmSNMCs0EuSJGnkDNOUGyv0kiRJ\n0gCzQi9JkqSRM0xz6E3oJUmSNHKGKaF3yo0kSZI0wKzQS5IkaeR4UawkSZKkvmCFXpIkSSNnmObQ\nm9BLkiRp5AxTQu+Umz6W5BVJ3rkB+789yZazGZMkSZL6ixX6PlZVZwJnbsAQbwc+D9w3OxFJkiQN\nBy+K1QZLsmOS65P8e5Jrk5yYZP8kFyX5cZJnJTksySea/ouSHJvk4iQ3JTm4ad8vyde6xv1Es9+R\nwGOB85Kc12x7aZJLklyR5JQkW7Vx7pIkSZo9JvTteiLwMeDpwFOAPwWeBxwF/M8J+m/XbH858IGp\nBq6qY4FbgRdW1QuTPBL4B2D/qnomsBj424n2TbIgyeIki7/42ZPX68QkSZL6W+ZgaYdTbtp1c1Vd\nA5BkKXBuVVWSa4AdJ+h/RlWtBq5L8ph1PNazgV2Bi5qvmOYDl0zUsaoWAgsBbrrnhlrH40iSJGkS\nSQ6gU9CdB/x7VX1g3PbNgM8BewK/Al5bVbdMNaYJfbtWdK2v7nq9mol/N93913wMXEnvNy2bT3Ks\nAN+uqtetR5ySJElDpY059EnmAZ8EXgIsAy5LcmZVXdfV7S3AnVX1xCSHAB8EXjvVuE65GXz/Ceya\nZLMk2wAv7tp2D7B1s34p8NwkTwRIsmWSJ23cUCVJkvpD5uC/GXgWcGNV3VRVDwAnAQeO63MgcHyz\nfirw4kzz6cOEfsBV1U+BLwFLgBOBK7s2LwS+keS8qvoFcBjwxSRL6CT4T9nI4UqSJA2t7usQm2XB\nuC6PA37a9XpZ0zZhn6paCdwNPGKq4zrlpiXNXKjdul4fNsm2ReO3N6+36lp/B/COCY7xceDjXa+/\nA/zBBgcvSZI04ObiwVLd1yFOetgJdluPPj2s0EuSJEkbxzJg+67Xj6dzV8IJ+yTZBNgGuGOqQU3o\nJUmSNHKSzPoyA5cBuyTZKcl84BB+9yGiZwJvatYPBr5TVVNW6J1yI0mSpJEzF1NuplNVK5McAZxN\n57aVx1XV0iTvBRZX1ZnA/wNOSHIjncr8IdONa0IvSZIkbSRVdRZw1ri293St3w+8el3GNKGXJEnS\nyGmjQj9XnEMvSZIkDTAr9JIkSRo5bTwpdq6Y0EuSJGnkDNOUGxN6TemuB+5sO4S+te38h7Udwlo1\n9fMmRt68zGs7hLX6KRaAVbWq7RB6THNntpG2+PYftB1Cj30fu1/bIfS48cvntB3CWk/8k/3bDqFH\nP703mhsm9JIkSRo5wzTlxotiJUmSpAFmhV6SJEkjZ5jm0FuhlyRJkgaYFXpJkiSNoOGp0JvQS5Ik\naeQMTzrvlBtJkiRpoFmhlyRJ0sjxtpWSJEmS+oIVekmSJI2g4anQm9BLkiRp5Pz/7d13nGRVnf7x\nzzMMcWAQZFwFRBAJIjKAgCQJiqI/xQCoSBCzKMvgb1d/q7goi6suZgQVdRFwMJAkGAFRQGAABxyy\nGAiCIkFJkgee3x/n1kx102FGpu+5Pf28edWru25Vdz3TXXSdOvd7vmfxGc6n5GZMSbqoAxn+0Xzc\nWNIsSddIulLSm2tni4iIiIinLjP0Y8j21mPxfSVNtj13Ib/sQeCttn8vaVXgMkln2r5nDCJGRERE\ndNziM0efAf0YkvQP28tLehZwAjCV8jN/n+1fDfc1wNeBHYG7gT1s3ynpXOAiYBvgDEknA98CpgF3\nAm+3/SdJawHfbR7nZ73va/t3fZ//RdIdzddmQB8RERExjqXkph17Amfa3hiYDswZ4b5TgMttbwqc\nB3y877an2d7e9ueBI4Fv294I+A7w5eY+hwNfs7058NehHkDSFsBSwB+Huf09kmZLmv2D405b4H9k\nRERExHghaZFfaskMfTt+DXxL0pLAabZHGtA/QZnNBzge+EHfbSf0fb4VsGvz+UzgM83n2wC79R0/\nrP+bN2cLZgL72n5iqAC2vwF8A+Dyv13sEbJGRERERGWZoW+B7fOB7YA/AzMlvXVhvrzv8wcW8H5D\nDsIlTQV+DPyn7YsXIkNEREREdFQG9C2Q9BzgDtvfBI4GNh3h7pOA3ZvP9wQuGOZ+FwF7NJ/v1Xe/\nCwcd72VYCjiVUqZz0sL+GyIiIiIWJxqD/2pJyU07dgA+JOkx4B/ASDP0DwAvkHQZcC8wXHvJGZQy\nng/RLIptjh8IfFfSgcApffd/E+UswdMlva059rZRyn8iIiIiouMyoB9DtpdvPh4HHLcQX3cwcPCg\nYzsMun4T8NIhvvZGSn19z/80x4+n1ORHRERETHg1Z9QXtZTcRERERESMY5mhr0TSJcDSgw7v05vV\nj4iIiIhYEBnQV2L7xbUzRERERExUNfvGL2opuYmIiIiIGMcyoI+IiIiIGMcyoI+IiIiIGMdSQx8R\nERERE87i1LYyA/qIiIiImIAWnwG9bNfOEB123T1XdOYJ8rgfrx1hgB2/dEDtCPPcdc1ttSMMsN3u\nW41+pxZ97mXvqx1hnmUnL1c7wgBTJnerU+6jTzxSO8I8XXt9XGbysrUjDPCEn6gdobNMt547z9t1\np9oRBvDZt3ZiJP33R+5c5L+olZeeVuXflhn6iIiIiJhwOvGuYhHJotiIiIiIiHEsM/QRERERMeEs\nThtLZUAfERERERPQ4jOgT8lNRERERMQ4lhn6iIiIiJhwFp/5+czQR0RERESMa5mhj4iIiIgJaPGZ\no88MfURERETEOJYZ+oiIiIiYcBantpWZoY+IiIiIGMcWiwG9pLdJOrLr3/OfyLCmpKufwtcftKi+\nV0RERER002IxoF9cSFpiEX/Lg0a/S0RERMTEozH4r5bqA3pJp0m6TNI1kt7THHunpN9JOlfSN3sz\n5ZKmSTpF0q+byzZDfL8n3UfSJEk3SXpa3/3+IOlfFuR7DpP7WEm7913/R/PxWZLOlzRH0tWSXtIc\nf4WkWZIul3SSpOWb4zdJ+pikC4A3SnqRpCskzQL27/v+S0j6bJPxSknvbY7v0DzeqZKulXRU8+/9\nH2DZJsd3mm+zRPPzvEbSWZKWXZjfVURERER0T/UBPfAO2y8CNgNmSFoNOBjYEng5sH7ffQ8Hvmh7\nc2A34H+H+H5Puo/tJ4DTgTcASHoxcJPt2xfwey6MPYEzbW8MTAfmSFoF+E9gJ9ubArOBf+v7modt\nb2v7+8AxwAzbWw36vu8E7m1ybg68W9JazW1bAP8OvBBYG9jV9oeBh2xvbHuv5n7rAF+x/QLgnubf\n+ySS3iNptqTZJx578lP5WURERER0lMbgUkcXutzMkPSG5vNnA/sA59n+O4Ckk4B1m9t3AjboW5U8\nVdIKg77fcPc5AfgYZcC8R3N9Qb/nwvg18C1JSwKn2Z4jaXtgA+DC5nGWAmb1fc0Jzb91ReBpts9r\njs8EXtV8/gpgo76zAitSBuiPApfavqH5Ht8DtgWGGonfaHtO8/llwJpD/QNsfwP4BsB191zhBf+n\nR0RERIwPi0+Pm8oDekk7UAbUW9l+UNK5wPXA84f5kknNfR8a9H0W5D6zgOdJmga8HvjvhfieQ5nb\nfC0qd14KwPb5krYDXg3MlPRZ4G7gbNtvGeZ7PdB7WGC4AbSAA2yfOSjnDkN8zXDf45G+zx8HUnIT\nERERMc7VLrlZEbi7GcyvTymzWQ7YXtJKkiYzsCzkLOBfe1ckbTzE9xzyPrYNnAp8AbjO9t8W4nsO\n5SbgRc3nrwOWbL7+OcAdtr8JHA1sClwMbCPpec19lpO07uBvaPse4F5J2zaH9uq7+Uzgfc3MP5LW\nlTSluW0LSWtJmgS8GbigOf5Y7/4RERERMZ+kRX6ppfaA/mfAZElXAp+gDHz/DHwKuAT4OXAtcG9z\n/xnAZs2i0GuB/Yb4niPd5wRgb+aX2yzo9xzKNylvPC4FXsz8WfYdKHXzv6G8GTnc9p3A24DvNf/W\nixm4NqDf24GvNGcU+s8a/C/lZ3G5SvvJrzP/DMss4H+Aq4EbKW9coJTNXNm3KDYiIiIiFjMqE9fd\nIml52/9oZuhPBb5l+9TRvm4iakpuPmj7NWPx/btUQ/+4H68dYYAdv3RA7Qjz3HXNbbUjDLDd7oPX\ndNf1uZe9r3aEeZadvFztCANMmbx87QgDPPrEI6PfqSVde31cZnK3qiSf8BO1I3SWh618reN5u+5U\nO8IAPvvWTpSvPzD3/qSC8QYAACAASURBVEX+i5oyeYUq/7baM/TDOUTSHObPOJ9WOU9ERERELEYW\nnx433ehy8yS2P1g7Q4+kjwJvHHT4JNufrJFnMNvnAudWjhERERERlXRyQN8lzcC9E4P3iIiIiFhU\nOlH5s0h0teQmIiIiIiIWQGboIyIiImLCqdlmclHLDH1ERERExDiWAX1ERERExDiWkpuIiIiImHC0\nGC2K7eTGUrF4kfQe29+onaMneUbWpTxdygLJM5IuZYHkGU2X8nQpCyTPSLqUJQZKyU204T21AwyS\nPCPrUp4uZYHkGUmXskDyjKZLebqUBZJnJF3KEn0yoI+IiIiIGMcyoI+IiIiIGMcyoI82dK3eLnlG\n1qU8XcoCyTOSLmWB5BlNl/J0KQskz0i6lCX6ZFFsRERERMQ4lhn6iIiIiIhxLAP6iIiIiIhxLAP6\niIiIiIhxLAP6iIiIiIhxbHLtALF4kfRvI91u+wttZekiSbuOdLvtH7SVJRaMpJnAv9q+t7n+HOBb\ntl9WN1k3SFrT9k2Djm1u+9cVsswEzgd+Zfu3bT/+EHkOtH34aMcmGklLAGfa3ql2lh5JywH/Dqxh\n+92S1gHWs/2jClmmAtNs/3HQ8Y1sX1khz9LAbsCa9I0bbR/adpYYXgb0sait0HxcD9gcOKO5vgvl\nhbYzJF1l+4UtP+wuzcdnAFsDv2iu7wicC7Q6oJd0BDBsqyvbM1qMM0/zxucwys9JzcW2p1aIcwFw\nSfNmdTXgQ5QX/k6Q9LHKL6w/kLSL7T83ebYHjgTa/n8L4BhgW+AISc8F5gDnVxxA7wsMfuy3DXFs\nzElaF/ga8C+2N5S0EfBa2//ddhbbj0t6UNKKvTfKHXAMcBmwVXP9VuAkoNUBvaQ3AV8C7pC0JPC2\nvjfHxwKbtpmncTpwL+Xn80iFx48FkLaVMSYknQXsZvv+5voKwEm2X9lyjuFmxAUcZXtam3nmPbj0\nI+Ddtm9rrj8L+IrtEWfwxyDHviPdbvu4trL0k/QHYBfb19V4/MEkbQv8ErgL2MT2XytHmkfSn2yv\nUfHxNwe+SnmzuinwKcrv7pZKeZagTCbsCOwHPGR7/ZYzvAXYk/Lm4ld9N00F5taYmZZ0HuXN6Ndt\nb9Icu9r2hm1naR77RGBL4Gzggd7xipMIs21vJuk3fT+fK2xPbznHHOBVtm+TtAXwbeAg2z/oz9Zy\npmrPk1hwmaGPsbIG8Gjf9Ucpp+vadgLwHYaehV6m5Sz91uwN5hu3A+u2HWLwgF3SFNsPDHf/Ft3e\nocH8PsDBwFuBjYCfSHq77StazHDfcDcBy7aVYyi2fy1pBnAW8DDwctt31sgi6RxgCjCLMpDe3PYd\nFaJcBNwGrAJ8vu/4/UDrJRON5WxfKqn/2NxKWQB+3Fy64lFJy9K8Vkhamzqz0Uv0Xhua39eOwI8k\nrc4IZ1PH2EWSXmj7qkqPHwsgA/oYKzOBSyWdSvkj9AbKTEPbrgQ+Z/vqwTdIqlm/ea6kM4HvUX4+\ne1BmgKuQtBVwNLA8sIak6cB7bb+/5Ry9MxSzJZ0AnEbfi2qlNQa7Ads2A8PvNc/p44CNW8xwD2Vw\nevvgGyTVmgn/IQMHGMtRTssfLQnbr60Q60rgRcCGTZZ7JM2y/VCbIWzfDNzc/I15yPYTTcnL+kCt\nQdFdzSC1N2DdnfKmo4paZ/9GcAjwM+DZkr4DbAO8vUKO+yWt3aufb2bqd6D8LXxBm0EkXUV5vkwG\n3i7pBsrf414J5EZt5omRpeQmxoykTYGXNFfPt/2bChleAtxs+09D3LaZ7dltZ+p7/F0Z+PM5tWKW\nS4DdgTNqno6XdMwIN9v2O1oLMwJJS9l+dPR7LrLH+2/K7+bSIW47zPZ/tJWl73G3H+l22+e1lWUw\nSctTBmMfBJ5pe+lKOS6j/D++EnAxMBt40PZeFbI8F/gGZe3O3cCNwF7Nm4/WSbqRIWacbT+3QhwA\nJD2dUgYk4GLbd1XIMJ3yHPn9oONLAm+y/Z0WszxnpNtrPXdiaBnQxyIlaart+yStPNTttv/edqYF\nIekjtj9dO0ctki6x/eLa9aN9ebaxfeFox1rK0pnFhKOR9ALb17T8mGsBt9l+uLm+LOVndVObOZrH\n/lfKAPpFwM3M73jzixG/cOzyXG57U0kHAMva/kzFOui1bN8oaQowyfb9vWNtZ2nyPL3v6jLAG4GV\nbX+sUp5zBneuGurYRCRppu19RjsWdaUPfSxq320+XkaZjepdete76o1tPIikC5qP90u6r+9y/wh1\n0m24RdLWgCUtJemDQM0a9iMW8Fgbvgl8BHgMoGkbt0elLKOZWeExTwKe6Lv+eHOshmWBLwDr236Z\n7f+qNZhvqCln24v59eK1Sl1PAbD9QK9ZAXBypSzY/lvf5c+2vwS8tO0ckpZpJqBWkbSSpJWby5rA\nqhXyDH5tGHBpO09jQKlPs/D8RZWyxDBSQx+LlO3XNB/XGul+NWYSR6HR7/LU2d62+bjCSPeTtJLt\nu9vI1NiP0kpvNUq7trOA/Vt8fGBeLf/WwDQN3NNgKrBE23kaXVtMOJJWnseDTO4vP7L9qKSlKuTA\n9mebjkT7AMdImgYsX2sWGvgA5c3gqbavacpeWl0rI2l9yoBsRQ3s+jWVio0BmpLMnknAZsxve9ym\n91J+T6sCl/cdvw/4Sttheq8Nkg4F/kp5ky7Km8JWfz6SPgIcBCzbvJno/X15lFK+FR2SAX3UMpM6\n/XSH07Xas3No9+ejGnW9Q1iKsjB3MgNfvO6j1PjX0KnFhKOo8Ty+U9JrbZ8BIOl1lPaerZP0ccrA\ncD1KX/ElgeMpCxxb16wjOK8pc8H2DUDbbRnXA14DPI35+2BA6bjz7paz9Ovv/jMXuAl4U9shXPYo\nOFzSAbZrnQUcys62X9x3/WvNWqfPtBWgKUP9tKRP2/5IW48b/5wM6KOWGjOJI5noeS5qFqmdAJxi\n+56WHx8YMAA61vbNKjsmuq9EoIb9KbNR60v6M2Ux4d4V83TNfsB3JB1Jed7eQmnxWcMbgE1oZlpt\n/0VlD4wqutA9yvbpwOmStrI9q63HHY3tHWtnGOReSU963tqu0Z0N4HFJewHfp7xRfwulnK2Gg5qz\nO9s2WX5l+7RKWWIYGdBHLa3OJEpaeZQFubVqfofT6s/H9joqm5jsAXxU0rXA920f32aOPtNUNt/q\nnX6+F3iH7cvaDtLMqu7Uv5iw7QxQirGB1T3yhk2tdd7padrrbdl0llHlN1+P2rak3tmUKRWzQNnx\nc2eaHbNtXyFpu0pZfiNpf0r5zbxSm1qdoyStCHwc6P08zgMOdb2dYzfv+3wZ4GWUN4a1BvR7Usog\ne7sKX9Acq+ErwPMobZYB9pP0ctutl2XG8NLlJqrodX9o8fF+T9kG/hjgp+74E7/tn8+gx16FsrBw\nL9tV6tYlXQnsb/tXzfVtga+22fd4UA3/k9j+QltZeiRdZrsTi9Ek7W37+OF+TpV+Ph8E1gFeDnwa\neAfw3VqlFF3qHiXpJOC3lEHhoZSa7OtsH9h2libPKcDVlD0doKx7mO6Wd8seTvOGY6br7KfQKZKu\nATbsvW5KmgRcZbvVvvgxsnS5iVranklcl1I2sQ/wB0mfatoRdlWrJTeSpkraV9JPmb/L5RZtZhjk\n/t5gHsD2BZSa3zat0Fw2A95HWTC8GqXEZIOWs/RcLGnz0e/Wit7s9wrDXFpn+3OUzi2nUGrHP1a5\nLrpL3aOeZ/tg4AGXTZ1eDbywUhaAtW1/3PYNzeW/gGo96IfwIOXNYRWSVpd0qqQ7JN0u6RSV3WJr\nuJ6y+3vPs6m343EMIzP0MWYkrQY8h77SLtvn10tUqGylfTxlQHIF8OEataVNl4deTeKFti/vu220\nEqFFneVGyk6EJ3ahzlbSFyk7j/Z20n0zZTOcXuu9y4f/6kWe5Sxgt14pSVOTfZLtV7aVoS/LtZQ3\npzcDD5AdGzutOdt1OLAT5Xd1FnCg7b9VyHKp7S0knQ+8n9JB5VJX2shJ0izgQ82bdSRtQ9nVe6tK\nefp3Pp5EedN+ou0PV8pzNqUNdK8V7d6Us6Yvr5DlPEpJUm9ju82BWZQ3PbV2hY5BMqCPMSHpMMog\n7FrmL+Rxrf/xVTYx2ZsyQ387ZaHaGcDGlMHZiG02xyDPxyi973/QHHp9k6PKZkWS1NQeT7H9QI0M\ng/KM1NrPtlvrVy3pt5RSgEea60sDV9hev60MfVmG3LnRFXZslPTlkW633Vo3F0kX2N5W0v0MXH/S\ne8Mzta0sfZmWAGbY/mLbjz0USe+ivCF+IXAsZaHuwba/XinPxpRymxWbQ3cD+7rs81AjT//Ox3Mp\nO4zfWiNLk2eO7Y1HO9ZSls7uCh3zZUAfY0LS9cBGvUFQbZJ+R5npOGbwH2lJ/2H7sJbzXAds4oG7\na15u+/lt5ujLM68bh+0q3Ti6StJHKe30TqUMFt9Ambn7VMVMz2DgwsY/Vciw70i3N2UdE5qkc23v\nUDnDUGsceiV9rrHWAea9Md4dWJvSUvPeJs+hNfJ0jaSfU9549RaivgV4uyvtXNtMJqxj++fN69Xk\nygvgY5B0uYmxcgOlB3T1AX0zU/Yj258Y6va2B/ONmygDsoeb60sDf6yQo6dL3TiQ9C/Ap4BVbb9K\n0gbAVraPbjuL7U9K+hmlPArKi+pv2s4BIOm1lP7dqwJ3UErarmPQTo5t6OKAXdLhlO5M1cvGGhc2\n7TxPoJRIAe2WjDF/PcN6lFKJM5rruwA1SyBPB+6hdJL5c8UcADRtGQ8DnkF5w1Pt7E7jHcCRwBcp\nEwkXNcdaJ+ndwHuAlSlvwFYHjqJ0AoqOyIA+xsqDwBxJ59A3qG/zNHzfYz7ezDh3ySPANU2dpCld\nOS7olTFU+jndooG7odbqeQxlZuoY4KPN9d9RBkWtD+gbcygLhScDSFqjxqw48AlgS+Dntjdp1oO8\npUKOeZrn8Bvd7F0gaSXKoHrnCnEuBw5uFryfCpxge3aFHD1bNx/7Z50NtFYy1iw27a0F2bRvLcgh\n1G3Xu3qNdSgj+Aywi+1ai5bnaSahdutQbfr+lCYJlwDY/n1zljA6JAP6GCtnMH8mqAvmSDqD8gLW\nP1P2g+G/ZEyd2lx6zq2Uo2dANw7KbpY1X9hWsX2iytbj2J4rqcobDEkHUPpl3055kyPKoKzGQtTH\nbP9N0iRJk2z/slmvUtM0921EZvvuWi/2zVmD4yStDOwGHNa8+arSrcTd2jxpDQZ2F3sUWLNOFKBs\nZvdC21dVzNDv9i4M5mHeJNTrKLPzXfCI7Ud7Ez6SJtO93dUnvAzoY0zYPq4ZGPZaQ15v+7GKkVYG\n/sbAmTEzf1FqqzpYrrAfpRvHasCtlG4cNTcNeaBZyNzre7wlpca2hgOB9Wp0JhnCPSobOP2Ksjvr\nHZQFfDU93n/Goqm1rf1i/zxgfcqA9dpaIbpUOkZZQ3SppP61IK3/HZJ0VfP4k4G3S7qBcsaydsem\n2ZJOoHT76j+rXGvSpwvlWj3nSToIWFbSyyldkn5YIUeMIItiY0xI2oHyYnET5Q/1sykdDKq3reyC\npk3kk/7nq9VCrmualp5HABtSNp+ZBuxeowNG03Hn5bZrD5x73ZG+RSn/2ZvSIeQ7Nd9sSHolZY+H\nXqeL7YD32D6zQpbDgF0p61FOAE7tP3tQIc9PaUrHbE9vZjZ/Y7tK//fm/6uXNFfPr7EWZLhOTT01\nOjYBSDpmiMN2vZ10h+r01WqHr74sk4B3Aq+gvJ6fCfyvM4DslAzoY0xIugzY0/b1zfV1ge+50i6X\nKhtyHAFsQxlIX0DpB12lLVkz+9yzDKWF5cq2P9ZyjiMYYTa1Ri1/TzP4WY/yAlLtDI+ko5scP2bg\nzF2NnVA/Tum483fg+8DJtm9vO8dgTb/1LSm/q1m27+q77QW2r2kpx36Un8ldw9zeWpbm8X5te3MN\n3Cm2SuvBiFi8peQmxsqSvcE8gO3fSVqyYp5jKJt0vLG5vndzrPVNOgCGmFH9kqQLgFYH9EDNBYNP\n0nSaGMq6kmqd/v5Tc1mquVTTLHD8L0kbUfZ5OE/SrbZ3qpzrLuBHw9w8E9i0pRxHjXKX1rI0ulQ6\nFkOQ9P9sf2a4yY1akxrN8+bjzN988ALg0DbPxvWVRw2pYnlUDCED+hgrs5uZzd4ud3sBl1XMM812\n/ynVYyV9oFaY5tR3zyRgM+a3l2vNgtbySzrC9gFjnYfSSg9K67itgV8013ekLBxufUDf6xLSMXdQ\ndvr8G+Vn1WUa/S6taTvLv1GaA6wt6UJK6dgbR/6SaFlvIexs6q/96Pd9SlvR3Zrre1HKyNp88/6a\n5mNvPVX/6/mDLeaIBZCSmxgTzaYh+1NmF0T5w/TVWhtNdXCTjv76yLmUtQaf6z+r0SWSLrfd2sym\npB8B77Z9W3P9WcBXbA83gz8WGb5k+wMauCX8PDVaykl6H2VmfhpwMqUtY7VFnwui7efOSCo8j5em\ndEaaVzoGTKr1dzCGJ2lz4CDKQureZGe1RbqSLhtcoipptu3NKmS50PY2ox2LujJDH2OiecH6QnPp\ngs5s0gGda2fXRWv2BvON25nfMaktvdmoz7X8uCN5DvAB23NqB4kFMqt5AzGvbl/S5bRb9hML5njg\nQ8BVwBOVswD8UtIewInN9d0p63hqmCJpW9sXADQtjqdUyhLDyAx9LFKSTrT9puFq71JzV0hakVIf\n2duN9TxKfWQn62srzGweCaxDOaNiYA/gDy2V/SwUSafY3m30e05Mki62vWXtHNBeFknPpLSAPR7Y\nk/mlPlOBo2yvP9YZYuFIusD2tqPfsx2S7qcMmnv7byzB/PaVdos72DYlosdQumqZsg7kHZVaaMYw\nMqCPRUrSs2zfNlxrsootyb48xOF7gdm2T6+Q5xRKO8ZeDfs+wPQ2S0oWRn+XjhYf8w3Mf8Nzvu1T\nR7p/LTV+Nl0iaRtgju0HJO1NmX0+vMb/65LOGVxGN9SxFnLsC7yNsjamf+H5/cCxFXubxzAkvYxS\nijl4d/NO/q7a6tjUtKzc3WWjv6mUcWMnJ54mugzoY0xImgI8ZPuJpmXl+sBPK7Ye/EaTobfV+W6U\n0+DPBm6w3eoC2aFa13WhnZ2kKbYfGOL422wfWyHSkCTNsr1V7RzQrRrxGiRdCUyn7Jw7Ezga2NX2\n9i1mWAZYDvglsAMDZ8R/avv5bWUZlGs326fUeOxYOJKOp7xGXMP8kptqfehH0+bfHUnn295u9HtG\nTamhj7FyPvASSStRZjxmUxbz7VUpz/OAl/Y2B5L0NcpuqC+n1Ey27aFBNYnbAA9VyEHz+FsD/wss\nD6whaTrwXtvvB+jSYL6xTO0AMc9c21bZqv5w20c3M9Rtei/wAWBVSjet3oD+PuArLWfp9yNJezJw\noSW2D62WKIYzvdaGX/+kNjs2nS3pgzx519q/t5ghRpEBfYwV2X5Q0juBI5o+v63vSthnNUo9Yu9U\n4RTKduyPS6rRcWI/4NtNLT3A3UDbg6B+XwR2prTYw/YVkro8I9OlU4tdastYw/2SPkLZ22E7SUsA\nre45Yftw4HBJB9g+os3HHsXplL85l9FXxhGddLGkDbreNapPm38De2cp9u87ZiA7m3dIBvQxViRp\nK8qM/DubYzWfb58B5kg6lzIA2w74VFMa9PM2gzQ1ieu5bAU/FcD2fW1mGIrtW6QBY9PHh7vvRCLp\nwGbAONyx/6gQq0veTFn4+U7bf5W0BvDZGkFsH9GcbVqTgTPi366RB1jd9isrPXYsnG2BfSXdSHnz\nJSq2rewS22vVzhCjSw19jAlJ2wP/Dlxo+zBJz6W026uy616T6VnAFpQ/1Jfa/kvFLJ2qSZR0MqXF\n6JHAlsAMYDPbe1QNNow2F6IOVas60RfCdpWkmcDawBzmvyF1xd0+v0E5Q1mjrC8WQtcaOYymze5R\nkpajbJK2hu33SFqHMik13O7QUUEG9DHmmhnp5WvOQqtMPe8FPNf2oc0s4jNtX1opz8GUmvlO1CRK\nWgU4nLILoSjrCw50i9uM92VZAjjT9rA7Ikra0PbVY5zjLZSZ522BX/XdtALw+Ej5JoJem7+mvV7/\nC0lvZrO1tnp9ma4DNnBHXtgkXUtZv5NZ31goHesedQKlbOyttjeUtCxlj4WqTRxioAzoY0xI+i6l\nTvxxyh+CFYEv2K5yKr5ZBPsEZWHs85vFumfZ3rxSnhuHOGzbqUkEJJ0B7FOzPVozY7cW8Gngw303\n3Q9c2VtgHd0h6SRgxqBNyaoZb7O+0R1d6B7Vl2W27c36z0xKusL29LazxPBSQx9jZQPb90naC/gJ\npc74MirV1gIvtr1pb2Gu7bslLVUpS+dqErvWpx94GLhK0tkMPIPRWulEM+i6GehEe8xYIKsA10q6\nlIG9xF/bZghJU5szkve3+bixWOlC96ieR5tZeQNIWpss8u6cDOhjrCwpaUng9cCRth+TVPN00GNN\nKUfvD9I0Km7vLWmoDaTuBa6yfUfbeShtIIfq0/9OSTu23aefssV5rW3OB2h+V4cBz6CUTFQrKYlR\nHVI7QOO7wGsokxhmYCekdAeJBVG9e1SfjwM/A54t6TvANpSN06JDUnITY0LSDMqs/BXAq4E1gONt\nv6RSnr0o3Tg2pezOujtwsO0TK+X5MWXm95fNoR2Ai4F1gUNtz2w5zy+AV/T16Z9MX59+2xu0mafJ\nsCxlEdb1bT/2oBx/AHaxfV3NHLH4aWu3zxh/JD2Tsobn17Z/1az72qFWxyZJT6c0TBBwse27auSI\n4WVAH62RNLlm3bGk9YGXUf4gnVNzgCbph8C7bN/eXP8X4GvAu4DzbW/Ycp7rgS16NetNf/xLbK9f\no6OLpF2AzwFL2V5L0saUNzqtlk40WS60vU3bjxsLb9AC3aUoM5oPdPVsykTfZTjGhy4t0I3hpeQm\nxkQzQP0UZfOmV0nagDIjfXSlPDNt7wP8dohjNazZG8w37gDWtf13SY9VyNOZPv2NQygtRs8FsD1H\nUq11B7ObLg+nMbAu+weV8sQwbK/Qf13S6ynPo66a6JuSxSBd7B5FmWya3uwg/iHgW8C3gdYX6Mbw\nMqCPsXIscAzw0eb67ygtGqsM6IEX9F9p6hFfVCkLwK8k/YiBNevnNwPoe9oO0yy4+inQe9NzFnCr\n7Qcof8DbNtf2vYM2uqp1OnEq8CDwikFZMqDvONunSfrw6PesJqfIYwDb2zYfVxjtvi3qX6D75coL\ndGMYGdDHWFnF9onNoh5sz5XU+s6jzeMfBCwr6T7mz4g9Cnyj7Tx99gd2pfQ4F2W245Smf/aObYeR\n9C7gQGB1yqY8WwKzgJe2naVxtaQ9gSWaTUxmABfVCGL77TUeNxbeoMXmk4DNyKA54qnq0gLdGMak\n2gFisfVAs4im11VmS0oXl1bZ/nQz0/FZ21Ntr9Bcnm77I23n6ctl26fY/r+2P2D75P7NcCTNajnS\ngcDmwM22dwQ2Ae5sOUO/AyhnVR4BvgfcB7TdaQcASetKOkfS1c31jST9Z40sMapd+i47U9pGvq5q\nopE9WjtAxAJ4M+Vv8Ttt/xVYjXotqGMYWRQbY0LSpsARwIbA1cA0YHfbV1bMtBKwDqVFIwC2z6+V\nZyRtL0SV9Gvbm0uaQ+nZ/4ikOV3YCbCZDZpSa6dhSedRyo6+3repytVtL1yO8UfSObZfNtqxiIin\nKjP0MVbWBl4FbA2cCfyeiiVeTUnJ+U2W/2o+HlIrzwJo+532rZKeRln4ebak04G/tJxhHknflTS1\nWVNwDXC9pBq1/ADL2b500LHsEttBklaXdKqkOyTdLukUSatXyLGMpJWBVSStJGnl5rImsGrbeSKe\nCkn3S7qvuTws6XFJ1XbxjqFlQB9j5eBmRnUlYCdKvfrXKubpWklJp9h+g+17bB8CHExZvPz6ipE2\naJ4/r6fsNLwGZcFuDXc1OyP2ysd2B26rlCVGdgxwBmXQvBrww+ZY295L2VRq/eZj73I68JUKeSL+\naU2Z6tTmsgyliUOexx2TAX2Mld4C2FcDR9k+ndIXupaHbT8MIGlp278F1quYZzTV2tnZPs/2GbZr\n1vf27zR8uu0arTx79ge+Dqwv6c+UWv73VcwTw5tm+xjbc5vLsZRyv1bZPtz2WsAHbT/X9lrNZbrt\nI9vOE7Eo2T6Neg0TYhjpchNj5c+Svk6ZnT9M0tLUfQM5uKTkbiqVlDQ14Wfa3mmEu9Waje6Ko4Ab\ngSsp7TyfQ4VF1QC2bwB2asp/Jtm+v0aOWCB3NRvffK+5/hbgb7XC2D5C0tbAmvS93tba7TPin5Hu\nUeNDFsXGmJC0HPBK4Crbv5f0LOCFts+qHA1J2wMrAj+rNQst6Qxgn97OrDGQpI/3XTXlRWQJ2wdX\nyPI04K08eVA2o+0sMTJJawBHUjaxM6XV6Qzbf6qUZyZlPdEc5p+1dJ47MZ5I6i9bmwvcBHzT9h11\nEsVQMkMfY8L2g/RtvGP7NirWHTdtM6+xfb/t8yStQKmjv6RSpIeBqySdDTzQO5gX+nn+0ff5MpQF\n1tdVyvIT4GLgKuCJShliwXwC2Nf23QDNwtTPAe+olGczynqQzJzFuDXaXhySPmL7023liaFlhj4m\nBEm/ATbtvbBKmgTMtr1ppTxD7rJn+7i2s4wHTcnWGbZ3rvDYl9d6nsTCGarda9stYAc99kmUMwRZ\nRB2LrfyN7IbM0MdEof5ZMttPSKr2/Ld9nKRlgTVsX18rxziyHPDcSo89U9K7gR9RNlcBwPbfK+WJ\n4U2StNKgGfqar3OrANdKupSBz53X1osUschVa+IQ82VAHxPFDZJmML915vuBG2qFkbQLpRRgKWAt\nSRsDh+aFvpB0FfMXXS1B6VRyaKU4j1J2RfxoXyZT7w1GDO/zwEWSTqb8jt4EfLJinkMqPnZEW1Lq\n0QEpuYkJQdIzgC9TWm0ZOAf4QK1FPZIua7Kc27f76FW2X1gjT9c0XW165gK3266ymZOkP1J2z72r\nxuPHwpG0AeX/GLrl4gAACxdJREFULQHn2L62cqSIxVrNsraYLzP0MSE0A/c9hru9wqKeubbvlQac\nqcy764btm2tn6HMN8GDtELFgmgF8Jwbxku5n/v/XSwFLAg/YnlovVcQid1LtAJEBfUTPG4E2B/RX\nS9oTWELSOsAMSou96J7HgTmSfsnAOuh0JIoR2V6h/7qk1wNbVIoTsVAkHcEIE029v4G2P9VaqBhW\nBvQRRduLeg6g1GQ/QtkE50xKy73ontOaS8RTYvs0SR+unSNiAc2uHSAWXGroI6jbdqvZOXaK7ftq\nPH6MTtJSwPqU2arra21IFuPLMDtsbm97q0qRIv5pkqbYfmD0e0YNk2oHiOiIVmfoJX1X0lRJUyg1\n2tdL+lCbGWLBSPo/wB8pi6qPBP4g6VV1U8U4sUvfZWfgfuB1VRNFLCRJW0m6lmZzP0nTJX21cqwY\nJDP0MSFI2sb2hcMdk3RQm3WAkubY3ljSXsCLgP8ALrO9UVsZYsFI+i3wGtt/aK6vDfzY9vp1k0VE\njD1JlwC7Uzb363Vlu9r2hnWTRb/M0MdEccRIxyos6llS0pLA64HTbT/W8uPHgrujN5hv3ABUaXca\n44uk1SWdKukOSbdLOkXS6rVzRSws27cMOvR4lSAxrCyKjcWapK2ArYFpkv6t76aplA2LajkKuBG4\nEji/6bt+b8U8MbxrJP0EOJFSQ/9G4Ne9+mjbP6gZLjrtGOC7lOcMwN7NsZdXSxSx8G6RtDXgZj3R\nDJrym+iODOhjcbcUsDzlud7fQu4+yinEWlYGvtl8fjDlbNm51dLESJYBbge2b67fSfn97UIZ4GdA\nH8OZZvuYvuvHSvpAtTQR/5z9gMOB1YBbgbOA/asmiifJgD4Wa7bPA86TdGzHNiv6R9/nywCvIjMe\nnWT77bUzxLh1l6S9Ka1pAd4C/K1inoh/hmzvVTtEjCyLYmNCkHQ28Ebb9zTXVwK+b3vnuskKSUtT\nFhx1Ik/M19Q8HwFsQ5mRvwA40PatVYNF50lag9IZaSvKc+ciYIbtP1UNFrEQJP2eUiJ6AnBK73U0\nuiWLYmOiWKX/j5Dtu4FnVMwz2HLAc2uHiCEdA5wBrEo55fzD5ljEaD4B7Gt7mu1nAO8ADqkbKWLh\n2F4H+E/gBcDlkn7UnHmKDsmAPiaKJ5rZMgCaRajVTk9JukrSlc3lGuB6So1idM8028fYnttcjgWm\n1Q4V48JGzeQBALb/DmxSMU/EP8X2pbb/DdgC+DtwXOVIMUhq6GOi+ChwgaTzmuvbAe+pmOc1fZ/P\nBW63PbdWmBhR6qDjnzVJ0kq9Qb2klcnrbowzkqYCbwD2ANYGTqUM7KNDUkMfE4akVYAtKbvCzrJ9\nV+VIMQ4MUwd9YMcWWUcHSXor8BHgZMpz503AJ23PrBosYiFIuhE4DTjR9qzaeWJoGdDHhCBpu6GO\n2z6/7SwRMXFI2gB4KWUi4Rzb11aOFLFQJMm2JU2x/UDtPDG0DOhjQpD0w76ry1BOF15m+6WVIkXH\nSTqCEdZZ2J7RYpyIiCqaDRqPBpa3vYak6cB7bb+/crTok1q+mBBs79J/XdKzgc9UihPjw+zaASIi\nOuBLwM6Ubl/YvmK4s95RTwb0MVHdCmxYO0R0l+0BXRxyujkiJirbt0jqP/R4rSwxtAzoY0IYVD4x\nCdgYuKJeohgv+k83AzndHBETzS2StgYsaSlgBtnZvHNSQx8TgqR9+67OBW6yfWGtPDF+SLoE2J2y\nk+8mzbGrbecMT0Qs9poOcYcDO1EWd59F6fSV9r0dkhn6mBAGl09ELIycbo6Iiapp8bxX7Rwxsgzo\nY7Em6SpG7lSyUYtxYnzK6eaImHDS6Wt8yYA+Fne9HVn3bz72NnTZC3iw/TgxDu1HOd28GmUx9VnM\nfz5FRCyu0ulrHEkNfUwIki60vc1oxyIGkzTN9p21c0RE1JROX902qXaAiJZMkbRt70pTQjGlYp4Y\nPy6SdJakd0p6Wu0wERFtkrSVpGtpSg0lTZf01cqxYpDM0MeEIOlFwLeAFZtD9wDvsH15vVQxXkja\nAtgDeD1wLfB928fXTRURMfbS6Wt8yIA+JhRJUynP+3trZ4nxp2nf9gVgL9tL1M4TETHWJF1i+8WS\nftM3oL/C9vTa2WK+lNzEhCBpRUlfAH4BnCPp85JWHO3rIiRNlbSvpJ8CFwG3AVtUjhUR0ZYBnb4k\nfZB0+uqczNDHhCDpFOBqoNePfh9guu1d66WK8UDSjcBpwIm2Z9XOExHRpmwsNT5kQB8TgqQ5tjce\n7VjEYJJk2+nwEBERXZU+9DFRPCRpW9sXAEjaBniocqYYH7aUdDSwPLCGpOnAe22/v3KuiIgxJ+nL\nQxy+F5ht+/S288TQUkMfE8V+wFck3STpJuBI4L11I8U48SVgZ+BvALavALarmigioj3LABsDv28u\nGwErA++U9KWawWK+zNDHYk/SJGA929ObLjfYvq9yrBhHbN8iqf/Q47WyRES07HnAS23PBZD0NUod\n/cuBq2oGi/kyQx+LPdtPAP/afH5fBvOxkNLhISImstUYuBHjFGBV248Dj9SJFINlhj4mirObgdgJ\nwLyFjbb/Xi9SjBP7UTo8rAbcSpmZ2r9qooiI9nwGmCPpXEqXm+2AT0maAvy8ZrCYL11uYkJoWg8+\n6clu+7kV4kRERIwbklaltHv+LWWG/lbb59dNFf0yoI8JQdKywPuBbSkD+18BR9lOp5sYkqQjGOJN\nYI/tGS3GiYioQtK7gAOB1YE5wJbALNsvrRosBkgNfUwUxwHPB74MHNF8ftyIXxET3WzgMkqHh02Z\n3+FhY7IoNiImjgOBzYGbbe8IbALcWTdSDJYa+pgo1rM9ve/6LyVdUS1NdJ7t4wAkvQ3Y0fZjzfWj\nKHX0ERETwcO2H5aEpKVt/1bSerVDxUCZoY+J4jeStuxdkfRi4MKKeWL8WBVYoe/68s2xiIiJ4FZJ\nTwNOozSYOB34S+VMMUhq6GNCkHQdsB7wp+bQGpTWg08Atr1RrWzRbZLeDhwC/LI5tD1wSG8GPyJi\nopC0PbAi8DPbj9bOE/NlQB8TgqTnjHS77ZvbyhLjj6RnAi9url5i+699t73A9jV1kkVERGRAHxHx\nlEi63PamtXNERMTElRr6iIinRrUDRETExJYBfUTEU5PTnBERUVUG9BERERER41gG9BERT006PURE\nRFUZ0EdEjEDSKZJeLWnIv5e2txzqeERERFsyoI+IGNnXgD2B30v6H0nr1w4UERHRL20rIyIWgKQV\ngbcAHwVuAb4JHG/7sarBIiJiwssMfUTEKCQ9HXgb8C7gN8DhwKbA2RVjRUREAJmhj4gYkaQfAOsD\nM4Fjbd/Wd9ts25tVCxcREUEG9BERI5L0f2z/ZNCxpW0/UitTREREv5TcRESM7L+HODar9RQRERHD\nmFw7QEREF0l6JrAasKykTQA1N00FlqsWLCIiYpAM6CMihrYzZSHs6sAX+o7fDxxUI1BERMRQUkMf\nETECSbvZPqV2joiIiOFkQB8RMQRJe9s+XtK/A0/6Q2n7C0N8WUREROtSchMRMbQpzcflh7gtMyER\nEdEZGdBHRAzB9tebT58LHGj7HgBJKwGfrxYsIiJikLStjIgY2Ua9wTyA7buBTSrmiYiIGCAD+oiI\nkU1qZuUBkLQyObsZEREdkheliIiRfR64SNLJlNr5NwGfrBspIiJivnS5iYgYhaQNgJdSNpc6x/a1\nlSNFRETMkwF9RERERMQ4lhr6iIiIiIhxLAP6iIiIiIhxLAP6iIiIiIhxLAP6iIiIiIhxLAP6iIiI\niIhx7P8DloJxdClzzLoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(df.corr(),cmap='Greens')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not a single good correlated value!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 146,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# differentiating categorical and other variables\n",
    "cat_cols = ['user_id','product','campaign_id','webpage_id','gender']\n",
    "int_cols = [col for col in train.columns if col not in cat_cols]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 147,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['session_id', 'user_id', 'product', 'campaign_id', 'webpage_id',\n",
       "       'product_category_1', 'user_group_id', 'gender', 'age_level',\n",
       "       'user_depth', 'city_development_index', 'var_1', 'is_click',\n",
       "       'count_view', 'count_interest', 'date', 'hour', 'minute', 'is_prod2',\n",
       "       'agelevel_userdepth'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 147,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 148,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Missing data in train: Index([], dtype='object')\n",
      "Missing data in test: Index([], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# missing data if any\n",
    "print('Missing data in train:',df.columns[df.isnull().any(axis=0)])\n",
    "print('Missing data in test:',dftest.columns[dftest.isnull().any(axis=0)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction\n",
    "\n",
    "### Feature Engineering Tried\n",
    "**Try 1**\n",
    "* Converted webpage_id, campaign_id as categorical variables\n",
    "* One hot encode all the categorical variables so there was no chance of having NaN values\n",
    "\n",
    "**Try 2**\n",
    "* Converted user_id also as the categorical value.\n",
    "* Gnerated few features such as is_prod2 from product_category_2\n",
    "* Imputed most of the variables\n",
    "\n",
    "**Try 3**\n",
    "* Converted most of the variables to categorical for Catboost to get its efficiency\n",
    "* Imputed the variables\n",
    "* Made categorical indices for Catboost to learn and it handles it very well\n",
    "\n",
    "### Models Tried\n",
    "\n",
    "* #### Random ForestClassifier\n",
    "    * **Model Score** :- Gave about 54.XX of public LB accuracy but was not enough\n",
    "    \n",
    "* #### DecisionTreeClassifier\n",
    "    * **Model Score** :- Gave about 51.XX accuracy, a simple DC was giving either 0.5 or 1\n",
    "\n",
    "* #### Logistic Regression\n",
    "    * **Model Score** :- Not able to go up than 54.XX , tried to tune it but was no avail\n",
    "\n",
    "* #### XGBClassifier\n",
    "    * **Model Score** :- Helped me a lot in training, spiked up the score to 58.XX, as did learn from Black Friday hackathon. Using user_id also a s field gave me a boost, tried to label encode it, but was hampering the score.\n",
    "\n",
    "* #### Catboost\n",
    "    * **Model Score** :- Best performance given with Try 3 which increased the score to 62.XX, tried playing with the params and got the score.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 236,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from catboost import CatBoostClassifier,Pool, cv\n",
    "from sklearn.model_selection import StratifiedKFold,train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import accuracy_score,confusion_matrix,roc_auc_score\n",
    "import warnings\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import itertools\n",
    "from scipy.stats import mode\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Test Split\n",
    "\n",
    "* Tried splitting them and taking out only session_id and is_click variable\n",
    "* Tried splitting them and taking out session_id, is_click user_depth and is_prod2 variable as it has less feature importances as compared to others"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X_train,X_val,y_train,y_val = train_test_split(df.drop(['session_id','is_click'],axis=1),df['is_click'],\n",
    "#                                                  test_size=0.25,random_state = 1994)\n",
    "#try 2 dropping 'is_prod2''user_depth'\n",
    "X_train,X_val,y_train,y_val = train_test_split(df.drop(['session_id','is_click','user_depth','is_prod2'],axis=1),df['is_click'],\n",
    "                                                 test_size=0.25,random_state = 1994)\n",
    "# X_train,X_val,y_train,y_val = train_test_split(df.drop(['session_id','is_click','user_depth','is_prod2','gender'],axis=1),df['is_click'],\n",
    "#                                                  test_size=0.25,random_state = 1994)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 237,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# categorical indices\n",
    "categorical_features_indices = np.where(X_train.dtypes =='object')[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Catboost Model Final"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 238,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\ttest: 0.5846687\tbest: 0.5846687 (0)\ttotal: 447ms\tremaining: 7m 26s\n",
      "1:\ttest: 0.5854164\tbest: 0.5854164 (1)\ttotal: 883ms\tremaining: 7m 20s\n",
      "2:\ttest: 0.5861178\tbest: 0.5861178 (2)\ttotal: 1.16s\tremaining: 6m 24s\n",
      "3:\ttest: 0.5853224\tbest: 0.5861178 (2)\ttotal: 1.54s\tremaining: 6m 24s\n",
      "4:\ttest: 0.5844146\tbest: 0.5861178 (2)\ttotal: 1.96s\tremaining: 6m 30s\n",
      "5:\ttest: 0.5866180\tbest: 0.5866180 (5)\ttotal: 2.33s\tremaining: 6m 26s\n",
      "6:\ttest: 0.5864579\tbest: 0.5866180 (5)\ttotal: 2.68s\tremaining: 6m 19s\n",
      "7:\ttest: 0.5871545\tbest: 0.5871545 (7)\ttotal: 3.02s\tremaining: 6m 14s\n",
      "8:\ttest: 0.5875622\tbest: 0.5875622 (8)\ttotal: 3.48s\tremaining: 6m 23s\n",
      "9:\ttest: 0.5868675\tbest: 0.5875622 (8)\ttotal: 3.88s\tremaining: 6m 24s\n",
      "10:\ttest: 0.5869501\tbest: 0.5875622 (8)\ttotal: 4.27s\tremaining: 6m 24s\n",
      "11:\ttest: 0.5877651\tbest: 0.5877651 (11)\ttotal: 4.71s\tremaining: 6m 28s\n",
      "12:\ttest: 0.5874768\tbest: 0.5877651 (11)\ttotal: 5.15s\tremaining: 6m 31s\n",
      "13:\ttest: 0.5874807\tbest: 0.5877651 (11)\ttotal: 5.57s\tremaining: 6m 31s\n",
      "14:\ttest: 0.5877171\tbest: 0.5877651 (11)\ttotal: 6s\tremaining: 6m 34s\n",
      "15:\ttest: 0.5880868\tbest: 0.5880868 (15)\ttotal: 6.46s\tremaining: 6m 37s\n",
      "16:\ttest: 0.5876550\tbest: 0.5880868 (15)\ttotal: 6.88s\tremaining: 6m 37s\n",
      "17:\ttest: 0.5887231\tbest: 0.5887231 (17)\ttotal: 7.35s\tremaining: 6m 40s\n",
      "18:\ttest: 0.5886390\tbest: 0.5887231 (17)\ttotal: 7.77s\tremaining: 6m 41s\n",
      "19:\ttest: 0.5889290\tbest: 0.5889290 (19)\ttotal: 8.21s\tremaining: 6m 42s\n",
      "20:\ttest: 0.5888108\tbest: 0.5889290 (19)\ttotal: 8.64s\tremaining: 6m 42s\n",
      "21:\ttest: 0.5884067\tbest: 0.5889290 (19)\ttotal: 9.06s\tremaining: 6m 42s\n",
      "22:\ttest: 0.5885743\tbest: 0.5889290 (19)\ttotal: 9.5s\tremaining: 6m 43s\n",
      "23:\ttest: 0.5886997\tbest: 0.5889290 (19)\ttotal: 9.94s\tremaining: 6m 44s\n",
      "24:\ttest: 0.5887475\tbest: 0.5889290 (19)\ttotal: 10.4s\tremaining: 6m 43s\n",
      "25:\ttest: 0.5887234\tbest: 0.5889290 (19)\ttotal: 10.7s\tremaining: 6m 41s\n",
      "26:\ttest: 0.5893691\tbest: 0.5893691 (26)\ttotal: 11.2s\tremaining: 6m 42s\n",
      "27:\ttest: 0.5893560\tbest: 0.5893691 (26)\ttotal: 11.6s\tremaining: 6m 43s\n",
      "28:\ttest: 0.5897670\tbest: 0.5897670 (28)\ttotal: 12.1s\tremaining: 6m 44s\n",
      "29:\ttest: 0.5899513\tbest: 0.5899513 (29)\ttotal: 12.5s\tremaining: 6m 44s\n",
      "30:\ttest: 0.5901766\tbest: 0.5901766 (30)\ttotal: 13s\tremaining: 6m 45s\n",
      "31:\ttest: 0.5903260\tbest: 0.5903260 (31)\ttotal: 13.4s\tremaining: 6m 44s\n",
      "32:\ttest: 0.5902520\tbest: 0.5903260 (31)\ttotal: 13.8s\tremaining: 6m 45s\n",
      "33:\ttest: 0.5901530\tbest: 0.5903260 (31)\ttotal: 14.3s\tremaining: 6m 45s\n",
      "34:\ttest: 0.5901502\tbest: 0.5903260 (31)\ttotal: 14.7s\tremaining: 6m 45s\n",
      "35:\ttest: 0.5901088\tbest: 0.5903260 (31)\ttotal: 15.2s\tremaining: 6m 46s\n",
      "36:\ttest: 0.5901858\tbest: 0.5903260 (31)\ttotal: 15.6s\tremaining: 6m 46s\n",
      "37:\ttest: 0.5901699\tbest: 0.5903260 (31)\ttotal: 16.1s\tremaining: 6m 47s\n",
      "38:\ttest: 0.5901473\tbest: 0.5903260 (31)\ttotal: 16.6s\tremaining: 6m 48s\n",
      "39:\ttest: 0.5901328\tbest: 0.5903260 (31)\ttotal: 17s\tremaining: 6m 49s\n",
      "40:\ttest: 0.5901039\tbest: 0.5903260 (31)\ttotal: 17.5s\tremaining: 6m 49s\n",
      "41:\ttest: 0.5903534\tbest: 0.5903534 (41)\ttotal: 18s\tremaining: 6m 50s\n",
      "42:\ttest: 0.5904029\tbest: 0.5904029 (42)\ttotal: 18.5s\tremaining: 6m 51s\n",
      "43:\ttest: 0.5904869\tbest: 0.5904869 (43)\ttotal: 18.9s\tremaining: 6m 51s\n",
      "44:\ttest: 0.5967381\tbest: 0.5967381 (44)\ttotal: 19.3s\tremaining: 6m 50s\n",
      "45:\ttest: 0.6021565\tbest: 0.6021565 (45)\ttotal: 19.7s\tremaining: 6m 48s\n",
      "46:\ttest: 0.6067161\tbest: 0.6067161 (46)\ttotal: 20s\tremaining: 6m 45s\n",
      "47:\ttest: 0.6098547\tbest: 0.6098547 (47)\ttotal: 20.3s\tremaining: 6m 43s\n",
      "48:\ttest: 0.6128784\tbest: 0.6128784 (48)\ttotal: 20.7s\tremaining: 6m 42s\n",
      "49:\ttest: 0.6148887\tbest: 0.6148887 (49)\ttotal: 21s\tremaining: 6m 39s\n",
      "50:\ttest: 0.6164926\tbest: 0.6164926 (50)\ttotal: 21.3s\tremaining: 6m 37s\n",
      "51:\ttest: 0.6184384\tbest: 0.6184384 (51)\ttotal: 21.6s\tremaining: 6m 34s\n",
      "52:\ttest: 0.6197457\tbest: 0.6197457 (52)\ttotal: 21.9s\tremaining: 6m 30s\n",
      "53:\ttest: 0.6208746\tbest: 0.6208746 (53)\ttotal: 22.2s\tremaining: 6m 29s\n",
      "54:\ttest: 0.6217538\tbest: 0.6217538 (54)\ttotal: 22.7s\tremaining: 6m 29s\n",
      "55:\ttest: 0.6224554\tbest: 0.6224554 (55)\ttotal: 23s\tremaining: 6m 28s\n",
      "56:\ttest: 0.6235697\tbest: 0.6235697 (56)\ttotal: 23.4s\tremaining: 6m 27s\n",
      "57:\ttest: 0.6242746\tbest: 0.6242746 (57)\ttotal: 23.8s\tremaining: 6m 25s\n",
      "58:\ttest: 0.6250101\tbest: 0.6250101 (58)\ttotal: 24.1s\tremaining: 6m 24s\n",
      "59:\ttest: 0.6252406\tbest: 0.6252406 (59)\ttotal: 24.5s\tremaining: 6m 23s\n",
      "60:\ttest: 0.6255699\tbest: 0.6255699 (60)\ttotal: 24.8s\tremaining: 6m 22s\n",
      "61:\ttest: 0.6257738\tbest: 0.6257738 (61)\ttotal: 25.2s\tremaining: 6m 21s\n",
      "62:\ttest: 0.6259495\tbest: 0.6259495 (62)\ttotal: 25.6s\tremaining: 6m 20s\n",
      "63:\ttest: 0.6261467\tbest: 0.6261467 (63)\ttotal: 26s\tremaining: 6m 19s\n",
      "64:\ttest: 0.6264110\tbest: 0.6264110 (64)\ttotal: 26.4s\tremaining: 6m 19s\n",
      "65:\ttest: 0.6267007\tbest: 0.6267007 (65)\ttotal: 26.8s\tremaining: 6m 19s\n",
      "66:\ttest: 0.6269619\tbest: 0.6269619 (66)\ttotal: 27.2s\tremaining: 6m 19s\n",
      "67:\ttest: 0.6270555\tbest: 0.6270555 (67)\ttotal: 27.7s\tremaining: 6m 19s\n",
      "68:\ttest: 0.6274416\tbest: 0.6274416 (68)\ttotal: 28.1s\tremaining: 6m 19s\n",
      "69:\ttest: 0.6275813\tbest: 0.6275813 (69)\ttotal: 28.5s\tremaining: 6m 18s\n",
      "70:\ttest: 0.6276665\tbest: 0.6276665 (70)\ttotal: 29s\tremaining: 6m 19s\n",
      "71:\ttest: 0.6279223\tbest: 0.6279223 (71)\ttotal: 29.3s\tremaining: 6m 18s\n",
      "72:\ttest: 0.6280910\tbest: 0.6280910 (72)\ttotal: 29.7s\tremaining: 6m 17s\n",
      "73:\ttest: 0.6282352\tbest: 0.6282352 (73)\ttotal: 30.1s\tremaining: 6m 16s\n",
      "74:\ttest: 0.6285254\tbest: 0.6285254 (74)\ttotal: 30.5s\tremaining: 6m 16s\n",
      "75:\ttest: 0.6286133\tbest: 0.6286133 (75)\ttotal: 30.9s\tremaining: 6m 16s\n",
      "76:\ttest: 0.6288759\tbest: 0.6288759 (76)\ttotal: 31.3s\tremaining: 6m 15s\n",
      "77:\ttest: 0.6290054\tbest: 0.6290054 (77)\ttotal: 31.8s\tremaining: 6m 15s\n",
      "78:\ttest: 0.6291702\tbest: 0.6291702 (78)\ttotal: 32.1s\tremaining: 6m 14s\n",
      "79:\ttest: 0.6293328\tbest: 0.6293328 (79)\ttotal: 32.5s\tremaining: 6m 13s\n",
      "80:\ttest: 0.6292891\tbest: 0.6293328 (79)\ttotal: 32.9s\tremaining: 6m 12s\n",
      "81:\ttest: 0.6293946\tbest: 0.6293946 (81)\ttotal: 33.3s\tremaining: 6m 12s\n",
      "82:\ttest: 0.6294732\tbest: 0.6294732 (82)\ttotal: 33.7s\tremaining: 6m 12s\n",
      "83:\ttest: 0.6295261\tbest: 0.6295261 (83)\ttotal: 34.1s\tremaining: 6m 11s\n",
      "84:\ttest: 0.6296690\tbest: 0.6296690 (84)\ttotal: 34.4s\tremaining: 6m 10s\n",
      "85:\ttest: 0.6297378\tbest: 0.6297378 (85)\ttotal: 34.8s\tremaining: 6m 9s\n",
      "86:\ttest: 0.6297373\tbest: 0.6297378 (85)\ttotal: 35.2s\tremaining: 6m 8s\n",
      "87:\ttest: 0.6298434\tbest: 0.6298434 (87)\ttotal: 35.6s\tremaining: 6m 8s\n",
      "88:\ttest: 0.6298940\tbest: 0.6298940 (88)\ttotal: 36s\tremaining: 6m 8s\n",
      "89:\ttest: 0.6300006\tbest: 0.6300006 (89)\ttotal: 36.5s\tremaining: 6m 8s\n",
      "90:\ttest: 0.6299593\tbest: 0.6300006 (89)\ttotal: 37s\tremaining: 6m 9s\n",
      "91:\ttest: 0.6299728\tbest: 0.6300006 (89)\ttotal: 37.4s\tremaining: 6m 8s\n",
      "92:\ttest: 0.6300947\tbest: 0.6300947 (92)\ttotal: 37.7s\tremaining: 6m 8s\n",
      "93:\ttest: 0.6300772\tbest: 0.6300947 (92)\ttotal: 38.2s\tremaining: 6m 8s\n",
      "94:\ttest: 0.6300747\tbest: 0.6300947 (92)\ttotal: 38.7s\tremaining: 6m 8s\n",
      "95:\ttest: 0.6301202\tbest: 0.6301202 (95)\ttotal: 39.2s\tremaining: 6m 8s\n",
      "96:\ttest: 0.6301580\tbest: 0.6301580 (96)\ttotal: 39.6s\tremaining: 6m 8s\n",
      "97:\ttest: 0.6301984\tbest: 0.6301984 (97)\ttotal: 40.1s\tremaining: 6m 9s\n",
      "98:\ttest: 0.6304513\tbest: 0.6304513 (98)\ttotal: 40.6s\tremaining: 6m 9s\n",
      "99:\ttest: 0.6305038\tbest: 0.6305038 (99)\ttotal: 41s\tremaining: 6m 9s\n",
      "100:\ttest: 0.6306385\tbest: 0.6306385 (100)\ttotal: 41.5s\tremaining: 6m 9s\n",
      "101:\ttest: 0.6308030\tbest: 0.6308030 (101)\ttotal: 41.9s\tremaining: 6m 8s\n",
      "102:\ttest: 0.6308522\tbest: 0.6308522 (102)\ttotal: 42.4s\tremaining: 6m 9s\n",
      "103:\ttest: 0.6308976\tbest: 0.6308976 (103)\ttotal: 42.8s\tremaining: 6m 9s\n",
      "104:\ttest: 0.6310754\tbest: 0.6310754 (104)\ttotal: 43.2s\tremaining: 6m 7s\n",
      "105:\ttest: 0.6310900\tbest: 0.6310900 (105)\ttotal: 43.6s\tremaining: 6m 8s\n",
      "106:\ttest: 0.6311886\tbest: 0.6311886 (106)\ttotal: 44.1s\tremaining: 6m 8s\n",
      "107:\ttest: 0.6312503\tbest: 0.6312503 (107)\ttotal: 44.5s\tremaining: 6m 7s\n",
      "108:\ttest: 0.6312195\tbest: 0.6312503 (107)\ttotal: 44.9s\tremaining: 6m 7s\n",
      "109:\ttest: 0.6312798\tbest: 0.6312798 (109)\ttotal: 45.4s\tremaining: 6m 7s\n",
      "110:\ttest: 0.6313307\tbest: 0.6313307 (110)\ttotal: 45.9s\tremaining: 6m 7s\n",
      "111:\ttest: 0.6312946\tbest: 0.6313307 (110)\ttotal: 46.4s\tremaining: 6m 7s\n",
      "112:\ttest: 0.6313637\tbest: 0.6313637 (112)\ttotal: 46.7s\tremaining: 6m 6s\n",
      "113:\ttest: 0.6314321\tbest: 0.6314321 (113)\ttotal: 47.1s\tremaining: 6m 6s\n",
      "114:\ttest: 0.6314366\tbest: 0.6314366 (114)\ttotal: 47.6s\tremaining: 6m 6s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "115:\ttest: 0.6314544\tbest: 0.6314544 (115)\ttotal: 48s\tremaining: 6m 5s\n",
      "116:\ttest: 0.6315099\tbest: 0.6315099 (116)\ttotal: 48.5s\tremaining: 6m 5s\n",
      "117:\ttest: 0.6316390\tbest: 0.6316390 (117)\ttotal: 49s\tremaining: 6m 5s\n",
      "118:\ttest: 0.6318739\tbest: 0.6318739 (118)\ttotal: 49.3s\tremaining: 6m 5s\n",
      "119:\ttest: 0.6319733\tbest: 0.6319733 (119)\ttotal: 49.8s\tremaining: 6m 5s\n",
      "120:\ttest: 0.6319791\tbest: 0.6319791 (120)\ttotal: 50.2s\tremaining: 6m 4s\n",
      "121:\ttest: 0.6319985\tbest: 0.6319985 (121)\ttotal: 50.6s\tremaining: 6m 3s\n",
      "122:\ttest: 0.6320428\tbest: 0.6320428 (122)\ttotal: 51s\tremaining: 6m 3s\n",
      "123:\ttest: 0.6321320\tbest: 0.6321320 (123)\ttotal: 51.5s\tremaining: 6m 3s\n",
      "124:\ttest: 0.6321719\tbest: 0.6321719 (124)\ttotal: 51.9s\tremaining: 6m 3s\n",
      "125:\ttest: 0.6322023\tbest: 0.6322023 (125)\ttotal: 52.3s\tremaining: 6m 2s\n",
      "126:\ttest: 0.6323169\tbest: 0.6323169 (126)\ttotal: 52.8s\tremaining: 6m 2s\n",
      "127:\ttest: 0.6323806\tbest: 0.6323806 (127)\ttotal: 53.1s\tremaining: 6m 1s\n",
      "128:\ttest: 0.6324997\tbest: 0.6324997 (128)\ttotal: 53.5s\tremaining: 6m 1s\n",
      "129:\ttest: 0.6325120\tbest: 0.6325120 (129)\ttotal: 54.1s\tremaining: 6m 2s\n",
      "130:\ttest: 0.6326201\tbest: 0.6326201 (130)\ttotal: 54.4s\tremaining: 6m\n",
      "131:\ttest: 0.6326216\tbest: 0.6326216 (131)\ttotal: 54.9s\tremaining: 6m 1s\n",
      "132:\ttest: 0.6326595\tbest: 0.6326595 (132)\ttotal: 55.3s\tremaining: 6m\n",
      "133:\ttest: 0.6326586\tbest: 0.6326595 (132)\ttotal: 55.8s\tremaining: 6m\n",
      "134:\ttest: 0.6326576\tbest: 0.6326595 (132)\ttotal: 56.3s\tremaining: 6m\n",
      "135:\ttest: 0.6326931\tbest: 0.6326931 (135)\ttotal: 56.6s\tremaining: 5m 59s\n",
      "136:\ttest: 0.6327093\tbest: 0.6327093 (136)\ttotal: 57s\tremaining: 5m 59s\n",
      "137:\ttest: 0.6327423\tbest: 0.6327423 (137)\ttotal: 57.5s\tremaining: 5m 59s\n",
      "138:\ttest: 0.6327179\tbest: 0.6327423 (137)\ttotal: 58s\tremaining: 5m 59s\n",
      "139:\ttest: 0.6327942\tbest: 0.6327942 (139)\ttotal: 58.4s\tremaining: 5m 58s\n",
      "140:\ttest: 0.6328230\tbest: 0.6328230 (140)\ttotal: 58.9s\tremaining: 5m 58s\n",
      "141:\ttest: 0.6328945\tbest: 0.6328945 (141)\ttotal: 59.3s\tremaining: 5m 58s\n",
      "142:\ttest: 0.6329293\tbest: 0.6329293 (142)\ttotal: 59.9s\tremaining: 5m 59s\n",
      "143:\ttest: 0.6329271\tbest: 0.6329293 (142)\ttotal: 1m\tremaining: 5m 59s\n",
      "144:\ttest: 0.6329706\tbest: 0.6329706 (144)\ttotal: 1m\tremaining: 5m 59s\n",
      "145:\ttest: 0.6330506\tbest: 0.6330506 (145)\ttotal: 1m 1s\tremaining: 5m 58s\n",
      "146:\ttest: 0.6330846\tbest: 0.6330846 (146)\ttotal: 1m 1s\tremaining: 5m 58s\n",
      "147:\ttest: 0.6331544\tbest: 0.6331544 (147)\ttotal: 1m 2s\tremaining: 5m 58s\n",
      "148:\ttest: 0.6332062\tbest: 0.6332062 (148)\ttotal: 1m 2s\tremaining: 5m 58s\n",
      "149:\ttest: 0.6332349\tbest: 0.6332349 (149)\ttotal: 1m 3s\tremaining: 5m 58s\n",
      "150:\ttest: 0.6332194\tbest: 0.6332349 (149)\ttotal: 1m 3s\tremaining: 5m 58s\n",
      "151:\ttest: 0.6332793\tbest: 0.6332793 (151)\ttotal: 1m 4s\tremaining: 5m 57s\n",
      "152:\ttest: 0.6333316\tbest: 0.6333316 (152)\ttotal: 1m 4s\tremaining: 5m 57s\n",
      "153:\ttest: 0.6333326\tbest: 0.6333326 (153)\ttotal: 1m 5s\tremaining: 5m 57s\n",
      "154:\ttest: 0.6333213\tbest: 0.6333326 (153)\ttotal: 1m 5s\tremaining: 5m 56s\n",
      "155:\ttest: 0.6333359\tbest: 0.6333359 (155)\ttotal: 1m 5s\tremaining: 5m 56s\n",
      "156:\ttest: 0.6333611\tbest: 0.6333611 (156)\ttotal: 1m 6s\tremaining: 5m 55s\n",
      "157:\ttest: 0.6333596\tbest: 0.6333611 (156)\ttotal: 1m 6s\tremaining: 5m 55s\n",
      "158:\ttest: 0.6334334\tbest: 0.6334334 (158)\ttotal: 1m 7s\tremaining: 5m 56s\n",
      "159:\ttest: 0.6334219\tbest: 0.6334334 (158)\ttotal: 1m 7s\tremaining: 5m 56s\n",
      "160:\ttest: 0.6334024\tbest: 0.6334334 (158)\ttotal: 1m 8s\tremaining: 5m 55s\n",
      "161:\ttest: 0.6333975\tbest: 0.6334334 (158)\ttotal: 1m 8s\tremaining: 5m 55s\n",
      "162:\ttest: 0.6334316\tbest: 0.6334334 (158)\ttotal: 1m 9s\tremaining: 5m 55s\n",
      "163:\ttest: 0.6334043\tbest: 0.6334334 (158)\ttotal: 1m 9s\tremaining: 5m 55s\n",
      "164:\ttest: 0.6334553\tbest: 0.6334553 (164)\ttotal: 1m 10s\tremaining: 5m 55s\n",
      "165:\ttest: 0.6334496\tbest: 0.6334553 (164)\ttotal: 1m 10s\tremaining: 5m 55s\n",
      "166:\ttest: 0.6334490\tbest: 0.6334553 (164)\ttotal: 1m 11s\tremaining: 5m 55s\n",
      "167:\ttest: 0.6334225\tbest: 0.6334553 (164)\ttotal: 1m 11s\tremaining: 5m 54s\n",
      "168:\ttest: 0.6334931\tbest: 0.6334931 (168)\ttotal: 1m 12s\tremaining: 5m 54s\n",
      "169:\ttest: 0.6335221\tbest: 0.6335221 (169)\ttotal: 1m 12s\tremaining: 5m 54s\n",
      "170:\ttest: 0.6335277\tbest: 0.6335277 (170)\ttotal: 1m 13s\tremaining: 5m 54s\n",
      "171:\ttest: 0.6335321\tbest: 0.6335321 (171)\ttotal: 1m 13s\tremaining: 5m 54s\n",
      "172:\ttest: 0.6335472\tbest: 0.6335472 (172)\ttotal: 1m 14s\tremaining: 5m 53s\n",
      "173:\ttest: 0.6335670\tbest: 0.6335670 (173)\ttotal: 1m 14s\tremaining: 5m 53s\n",
      "174:\ttest: 0.6335825\tbest: 0.6335825 (174)\ttotal: 1m 15s\tremaining: 5m 53s\n",
      "175:\ttest: 0.6335736\tbest: 0.6335825 (174)\ttotal: 1m 15s\tremaining: 5m 54s\n",
      "176:\ttest: 0.6336318\tbest: 0.6336318 (176)\ttotal: 1m 16s\tremaining: 5m 53s\n",
      "177:\ttest: 0.6337026\tbest: 0.6337026 (177)\ttotal: 1m 16s\tremaining: 5m 53s\n",
      "178:\ttest: 0.6337280\tbest: 0.6337280 (178)\ttotal: 1m 16s\tremaining: 5m 52s\n",
      "179:\ttest: 0.6337476\tbest: 0.6337476 (179)\ttotal: 1m 17s\tremaining: 5m 52s\n",
      "180:\ttest: 0.6337859\tbest: 0.6337859 (180)\ttotal: 1m 17s\tremaining: 5m 52s\n",
      "181:\ttest: 0.6338442\tbest: 0.6338442 (181)\ttotal: 1m 18s\tremaining: 5m 52s\n",
      "182:\ttest: 0.6338855\tbest: 0.6338855 (182)\ttotal: 1m 18s\tremaining: 5m 52s\n",
      "183:\ttest: 0.6339265\tbest: 0.6339265 (183)\ttotal: 1m 19s\tremaining: 5m 53s\n",
      "184:\ttest: 0.6339486\tbest: 0.6339486 (184)\ttotal: 1m 20s\tremaining: 5m 53s\n",
      "185:\ttest: 0.6339501\tbest: 0.6339501 (185)\ttotal: 1m 20s\tremaining: 5m 53s\n",
      "186:\ttest: 0.6339737\tbest: 0.6339737 (186)\ttotal: 1m 21s\tremaining: 5m 53s\n",
      "187:\ttest: 0.6339806\tbest: 0.6339806 (187)\ttotal: 1m 21s\tremaining: 5m 53s\n",
      "188:\ttest: 0.6339973\tbest: 0.6339973 (188)\ttotal: 1m 22s\tremaining: 5m 53s\n",
      "189:\ttest: 0.6339762\tbest: 0.6339973 (188)\ttotal: 1m 22s\tremaining: 5m 53s\n",
      "190:\ttest: 0.6340310\tbest: 0.6340310 (190)\ttotal: 1m 23s\tremaining: 5m 53s\n",
      "191:\ttest: 0.6340759\tbest: 0.6340759 (191)\ttotal: 1m 23s\tremaining: 5m 53s\n",
      "192:\ttest: 0.6340973\tbest: 0.6340973 (192)\ttotal: 1m 24s\tremaining: 5m 52s\n",
      "193:\ttest: 0.6341349\tbest: 0.6341349 (193)\ttotal: 1m 24s\tremaining: 5m 52s\n",
      "194:\ttest: 0.6341534\tbest: 0.6341534 (194)\ttotal: 1m 25s\tremaining: 5m 52s\n",
      "195:\ttest: 0.6341867\tbest: 0.6341867 (195)\ttotal: 1m 25s\tremaining: 5m 52s\n",
      "196:\ttest: 0.6341878\tbest: 0.6341878 (196)\ttotal: 1m 26s\tremaining: 5m 52s\n",
      "197:\ttest: 0.6341818\tbest: 0.6341878 (196)\ttotal: 1m 26s\tremaining: 5m 51s\n",
      "198:\ttest: 0.6341046\tbest: 0.6341878 (196)\ttotal: 1m 27s\tremaining: 5m 52s\n",
      "199:\ttest: 0.6341710\tbest: 0.6341878 (196)\ttotal: 1m 28s\tremaining: 5m 52s\n",
      "200:\ttest: 0.6341582\tbest: 0.6341878 (196)\ttotal: 1m 28s\tremaining: 5m 52s\n",
      "201:\ttest: 0.6341538\tbest: 0.6341878 (196)\ttotal: 1m 29s\tremaining: 5m 51s\n",
      "202:\ttest: 0.6342333\tbest: 0.6342333 (202)\ttotal: 1m 29s\tremaining: 5m 51s\n",
      "203:\ttest: 0.6342317\tbest: 0.6342333 (202)\ttotal: 1m 29s\tremaining: 5m 50s\n",
      "204:\ttest: 0.6342354\tbest: 0.6342354 (204)\ttotal: 1m 30s\tremaining: 5m 50s\n",
      "205:\ttest: 0.6342390\tbest: 0.6342390 (205)\ttotal: 1m 30s\tremaining: 5m 49s\n",
      "206:\ttest: 0.6342523\tbest: 0.6342523 (206)\ttotal: 1m 31s\tremaining: 5m 49s\n",
      "207:\ttest: 0.6342765\tbest: 0.6342765 (207)\ttotal: 1m 31s\tremaining: 5m 49s\n",
      "208:\ttest: 0.6343028\tbest: 0.6343028 (208)\ttotal: 1m 32s\tremaining: 5m 48s\n",
      "209:\ttest: 0.6342988\tbest: 0.6343028 (208)\ttotal: 1m 32s\tremaining: 5m 47s\n",
      "210:\ttest: 0.6343014\tbest: 0.6343028 (208)\ttotal: 1m 33s\tremaining: 5m 47s\n",
      "211:\ttest: 0.6342997\tbest: 0.6343028 (208)\ttotal: 1m 33s\tremaining: 5m 48s\n",
      "212:\ttest: 0.6343529\tbest: 0.6343529 (212)\ttotal: 1m 34s\tremaining: 5m 47s\n",
      "213:\ttest: 0.6344183\tbest: 0.6344183 (213)\ttotal: 1m 34s\tremaining: 5m 47s\n",
      "214:\ttest: 0.6344430\tbest: 0.6344430 (214)\ttotal: 1m 35s\tremaining: 5m 46s\n",
      "215:\ttest: 0.6344426\tbest: 0.6344430 (214)\ttotal: 1m 35s\tremaining: 5m 46s\n",
      "216:\ttest: 0.6344696\tbest: 0.6344696 (216)\ttotal: 1m 35s\tremaining: 5m 46s\n",
      "217:\ttest: 0.6344615\tbest: 0.6344696 (216)\ttotal: 1m 36s\tremaining: 5m 45s\n",
      "218:\ttest: 0.6344403\tbest: 0.6344696 (216)\ttotal: 1m 36s\tremaining: 5m 45s\n",
      "219:\ttest: 0.6344965\tbest: 0.6344965 (219)\ttotal: 1m 37s\tremaining: 5m 45s\n",
      "220:\ttest: 0.6345168\tbest: 0.6345168 (220)\ttotal: 1m 37s\tremaining: 5m 44s\n",
      "221:\ttest: 0.6345216\tbest: 0.6345216 (221)\ttotal: 1m 38s\tremaining: 5m 44s\n",
      "222:\ttest: 0.6345099\tbest: 0.6345216 (221)\ttotal: 1m 38s\tremaining: 5m 43s\n",
      "223:\ttest: 0.6345313\tbest: 0.6345313 (223)\ttotal: 1m 39s\tremaining: 5m 43s\n",
      "224:\ttest: 0.6345054\tbest: 0.6345313 (223)\ttotal: 1m 39s\tremaining: 5m 42s\n",
      "225:\ttest: 0.6345004\tbest: 0.6345313 (223)\ttotal: 1m 40s\tremaining: 5m 42s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "226:\ttest: 0.6345220\tbest: 0.6345313 (223)\ttotal: 1m 40s\tremaining: 5m 42s\n",
      "227:\ttest: 0.6344753\tbest: 0.6345313 (223)\ttotal: 1m 41s\tremaining: 5m 42s\n",
      "228:\ttest: 0.6344638\tbest: 0.6345313 (223)\ttotal: 1m 41s\tremaining: 5m 41s\n",
      "229:\ttest: 0.6344617\tbest: 0.6345313 (223)\ttotal: 1m 41s\tremaining: 5m 41s\n",
      "230:\ttest: 0.6344728\tbest: 0.6345313 (223)\ttotal: 1m 42s\tremaining: 5m 41s\n",
      "231:\ttest: 0.6344679\tbest: 0.6345313 (223)\ttotal: 1m 43s\tremaining: 5m 41s\n",
      "232:\ttest: 0.6344832\tbest: 0.6345313 (223)\ttotal: 1m 43s\tremaining: 5m 40s\n",
      "233:\ttest: 0.6344854\tbest: 0.6345313 (223)\ttotal: 1m 44s\tremaining: 5m 40s\n",
      "234:\ttest: 0.6345062\tbest: 0.6345313 (223)\ttotal: 1m 44s\tremaining: 5m 40s\n",
      "235:\ttest: 0.6344726\tbest: 0.6345313 (223)\ttotal: 1m 45s\tremaining: 5m 40s\n",
      "236:\ttest: 0.6344953\tbest: 0.6345313 (223)\ttotal: 1m 45s\tremaining: 5m 40s\n",
      "237:\ttest: 0.6345532\tbest: 0.6345532 (237)\ttotal: 1m 46s\tremaining: 5m 39s\n",
      "238:\ttest: 0.6345719\tbest: 0.6345719 (238)\ttotal: 1m 46s\tremaining: 5m 39s\n",
      "239:\ttest: 0.6345845\tbest: 0.6345845 (239)\ttotal: 1m 47s\tremaining: 5m 39s\n",
      "240:\ttest: 0.6345787\tbest: 0.6345845 (239)\ttotal: 1m 47s\tremaining: 5m 38s\n",
      "241:\ttest: 0.6345771\tbest: 0.6345845 (239)\ttotal: 1m 48s\tremaining: 5m 38s\n",
      "242:\ttest: 0.6345991\tbest: 0.6345991 (242)\ttotal: 1m 48s\tremaining: 5m 38s\n",
      "243:\ttest: 0.6346859\tbest: 0.6346859 (243)\ttotal: 1m 49s\tremaining: 5m 37s\n",
      "244:\ttest: 0.6347123\tbest: 0.6347123 (244)\ttotal: 1m 49s\tremaining: 5m 37s\n",
      "245:\ttest: 0.6347266\tbest: 0.6347266 (245)\ttotal: 1m 50s\tremaining: 5m 37s\n",
      "246:\ttest: 0.6347529\tbest: 0.6347529 (246)\ttotal: 1m 50s\tremaining: 5m 36s\n",
      "247:\ttest: 0.6347354\tbest: 0.6347529 (246)\ttotal: 1m 51s\tremaining: 5m 36s\n",
      "248:\ttest: 0.6347621\tbest: 0.6347621 (248)\ttotal: 1m 51s\tremaining: 5m 36s\n",
      "249:\ttest: 0.6348003\tbest: 0.6348003 (249)\ttotal: 1m 51s\tremaining: 5m 35s\n",
      "250:\ttest: 0.6347866\tbest: 0.6348003 (249)\ttotal: 1m 52s\tremaining: 5m 35s\n",
      "251:\ttest: 0.6348231\tbest: 0.6348231 (251)\ttotal: 1m 52s\tremaining: 5m 34s\n",
      "252:\ttest: 0.6348090\tbest: 0.6348231 (251)\ttotal: 1m 53s\tremaining: 5m 34s\n",
      "253:\ttest: 0.6347418\tbest: 0.6348231 (251)\ttotal: 1m 53s\tremaining: 5m 33s\n",
      "254:\ttest: 0.6347409\tbest: 0.6348231 (251)\ttotal: 1m 54s\tremaining: 5m 33s\n",
      "255:\ttest: 0.6347034\tbest: 0.6348231 (251)\ttotal: 1m 54s\tremaining: 5m 33s\n",
      "256:\ttest: 0.6346889\tbest: 0.6348231 (251)\ttotal: 1m 55s\tremaining: 5m 32s\n",
      "257:\ttest: 0.6347114\tbest: 0.6348231 (251)\ttotal: 1m 55s\tremaining: 5m 32s\n",
      "258:\ttest: 0.6346868\tbest: 0.6348231 (251)\ttotal: 1m 56s\tremaining: 5m 32s\n",
      "259:\ttest: 0.6346752\tbest: 0.6348231 (251)\ttotal: 1m 56s\tremaining: 5m 31s\n",
      "260:\ttest: 0.6346238\tbest: 0.6348231 (251)\ttotal: 1m 56s\tremaining: 5m 31s\n",
      "261:\ttest: 0.6346787\tbest: 0.6348231 (251)\ttotal: 1m 57s\tremaining: 5m 30s\n",
      "262:\ttest: 0.6346338\tbest: 0.6348231 (251)\ttotal: 1m 57s\tremaining: 5m 30s\n",
      "263:\ttest: 0.6346891\tbest: 0.6348231 (251)\ttotal: 1m 58s\tremaining: 5m 30s\n",
      "264:\ttest: 0.6346762\tbest: 0.6348231 (251)\ttotal: 1m 58s\tremaining: 5m 29s\n",
      "265:\ttest: 0.6346393\tbest: 0.6348231 (251)\ttotal: 1m 59s\tremaining: 5m 29s\n",
      "266:\ttest: 0.6346205\tbest: 0.6348231 (251)\ttotal: 1m 59s\tremaining: 5m 29s\n",
      "267:\ttest: 0.6346263\tbest: 0.6348231 (251)\ttotal: 2m\tremaining: 5m 28s\n",
      "268:\ttest: 0.6345798\tbest: 0.6348231 (251)\ttotal: 2m\tremaining: 5m 28s\n",
      "269:\ttest: 0.6346419\tbest: 0.6348231 (251)\ttotal: 2m 1s\tremaining: 5m 28s\n",
      "270:\ttest: 0.6346780\tbest: 0.6348231 (251)\ttotal: 2m 1s\tremaining: 5m 28s\n",
      "271:\ttest: 0.6346829\tbest: 0.6348231 (251)\ttotal: 2m 2s\tremaining: 5m 27s\n",
      "272:\ttest: 0.6346868\tbest: 0.6348231 (251)\ttotal: 2m 2s\tremaining: 5m 27s\n",
      "273:\ttest: 0.6346611\tbest: 0.6348231 (251)\ttotal: 2m 3s\tremaining: 5m 27s\n",
      "274:\ttest: 0.6346782\tbest: 0.6348231 (251)\ttotal: 2m 3s\tremaining: 5m 26s\n",
      "275:\ttest: 0.6346734\tbest: 0.6348231 (251)\ttotal: 2m 4s\tremaining: 5m 26s\n",
      "276:\ttest: 0.6346938\tbest: 0.6348231 (251)\ttotal: 2m 4s\tremaining: 5m 25s\n",
      "277:\ttest: 0.6347193\tbest: 0.6348231 (251)\ttotal: 2m 5s\tremaining: 5m 25s\n",
      "278:\ttest: 0.6347089\tbest: 0.6348231 (251)\ttotal: 2m 5s\tremaining: 5m 25s\n",
      "279:\ttest: 0.6347060\tbest: 0.6348231 (251)\ttotal: 2m 6s\tremaining: 5m 24s\n",
      "280:\ttest: 0.6347092\tbest: 0.6348231 (251)\ttotal: 2m 6s\tremaining: 5m 24s\n",
      "281:\ttest: 0.6347341\tbest: 0.6348231 (251)\ttotal: 2m 7s\tremaining: 5m 23s\n",
      "282:\ttest: 0.6347368\tbest: 0.6348231 (251)\ttotal: 2m 7s\tremaining: 5m 23s\n",
      "283:\ttest: 0.6347974\tbest: 0.6348231 (251)\ttotal: 2m 8s\tremaining: 5m 23s\n",
      "284:\ttest: 0.6347871\tbest: 0.6348231 (251)\ttotal: 2m 8s\tremaining: 5m 22s\n",
      "285:\ttest: 0.6347429\tbest: 0.6348231 (251)\ttotal: 2m 8s\tremaining: 5m 21s\n",
      "286:\ttest: 0.6347255\tbest: 0.6348231 (251)\ttotal: 2m 9s\tremaining: 5m 21s\n",
      "287:\ttest: 0.6347150\tbest: 0.6348231 (251)\ttotal: 2m 9s\tremaining: 5m 21s\n",
      "288:\ttest: 0.6347553\tbest: 0.6348231 (251)\ttotal: 2m 10s\tremaining: 5m 20s\n",
      "289:\ttest: 0.6347228\tbest: 0.6348231 (251)\ttotal: 2m 10s\tremaining: 5m 20s\n",
      "290:\ttest: 0.6347325\tbest: 0.6348231 (251)\ttotal: 2m 11s\tremaining: 5m 19s\n",
      "291:\ttest: 0.6347002\tbest: 0.6348231 (251)\ttotal: 2m 11s\tremaining: 5m 19s\n",
      "292:\ttest: 0.6347135\tbest: 0.6348231 (251)\ttotal: 2m 12s\tremaining: 5m 19s\n",
      "293:\ttest: 0.6347468\tbest: 0.6348231 (251)\ttotal: 2m 12s\tremaining: 5m 18s\n",
      "294:\ttest: 0.6347432\tbest: 0.6348231 (251)\ttotal: 2m 13s\tremaining: 5m 18s\n",
      "295:\ttest: 0.6347430\tbest: 0.6348231 (251)\ttotal: 2m 13s\tremaining: 5m 17s\n",
      "296:\ttest: 0.6347776\tbest: 0.6348231 (251)\ttotal: 2m 14s\tremaining: 5m 17s\n",
      "297:\ttest: 0.6347963\tbest: 0.6348231 (251)\ttotal: 2m 14s\tremaining: 5m 16s\n",
      "298:\ttest: 0.6348755\tbest: 0.6348755 (298)\ttotal: 2m 15s\tremaining: 5m 16s\n",
      "299:\ttest: 0.6348906\tbest: 0.6348906 (299)\ttotal: 2m 15s\tremaining: 5m 16s\n",
      "300:\ttest: 0.6349041\tbest: 0.6349041 (300)\ttotal: 2m 16s\tremaining: 5m 16s\n",
      "301:\ttest: 0.6349120\tbest: 0.6349120 (301)\ttotal: 2m 16s\tremaining: 5m 15s\n",
      "302:\ttest: 0.6348895\tbest: 0.6349120 (301)\ttotal: 2m 17s\tremaining: 5m 15s\n",
      "303:\ttest: 0.6348887\tbest: 0.6349120 (301)\ttotal: 2m 17s\tremaining: 5m 15s\n",
      "304:\ttest: 0.6349130\tbest: 0.6349130 (304)\ttotal: 2m 18s\tremaining: 5m 14s\n",
      "305:\ttest: 0.6349288\tbest: 0.6349288 (305)\ttotal: 2m 18s\tremaining: 5m 14s\n",
      "306:\ttest: 0.6349589\tbest: 0.6349589 (306)\ttotal: 2m 18s\tremaining: 5m 13s\n",
      "307:\ttest: 0.6349483\tbest: 0.6349589 (306)\ttotal: 2m 19s\tremaining: 5m 13s\n",
      "308:\ttest: 0.6349197\tbest: 0.6349589 (306)\ttotal: 2m 19s\tremaining: 5m 12s\n",
      "309:\ttest: 0.6348875\tbest: 0.6349589 (306)\ttotal: 2m 20s\tremaining: 5m 12s\n",
      "310:\ttest: 0.6349169\tbest: 0.6349589 (306)\ttotal: 2m 20s\tremaining: 5m 11s\n",
      "311:\ttest: 0.6349468\tbest: 0.6349589 (306)\ttotal: 2m 21s\tremaining: 5m 11s\n",
      "312:\ttest: 0.6349350\tbest: 0.6349589 (306)\ttotal: 2m 21s\tremaining: 5m 10s\n",
      "313:\ttest: 0.6349447\tbest: 0.6349589 (306)\ttotal: 2m 22s\tremaining: 5m 10s\n",
      "314:\ttest: 0.6349586\tbest: 0.6349589 (306)\ttotal: 2m 22s\tremaining: 5m 10s\n",
      "315:\ttest: 0.6349602\tbest: 0.6349602 (315)\ttotal: 2m 23s\tremaining: 5m 10s\n",
      "316:\ttest: 0.6349361\tbest: 0.6349602 (315)\ttotal: 2m 23s\tremaining: 5m 9s\n",
      "317:\ttest: 0.6349266\tbest: 0.6349602 (315)\ttotal: 2m 24s\tremaining: 5m 9s\n",
      "318:\ttest: 0.6349407\tbest: 0.6349602 (315)\ttotal: 2m 24s\tremaining: 5m 9s\n",
      "319:\ttest: 0.6349255\tbest: 0.6349602 (315)\ttotal: 2m 25s\tremaining: 5m 8s\n",
      "320:\ttest: 0.6349378\tbest: 0.6349602 (315)\ttotal: 2m 25s\tremaining: 5m 8s\n",
      "321:\ttest: 0.6348922\tbest: 0.6349602 (315)\ttotal: 2m 26s\tremaining: 5m 7s\n",
      "322:\ttest: 0.6348774\tbest: 0.6349602 (315)\ttotal: 2m 26s\tremaining: 5m 7s\n",
      "323:\ttest: 0.6348687\tbest: 0.6349602 (315)\ttotal: 2m 27s\tremaining: 5m 7s\n",
      "324:\ttest: 0.6348659\tbest: 0.6349602 (315)\ttotal: 2m 27s\tremaining: 5m 6s\n",
      "325:\ttest: 0.6349070\tbest: 0.6349602 (315)\ttotal: 2m 28s\tremaining: 5m 6s\n",
      "326:\ttest: 0.6349071\tbest: 0.6349602 (315)\ttotal: 2m 28s\tremaining: 5m 5s\n",
      "327:\ttest: 0.6349102\tbest: 0.6349602 (315)\ttotal: 2m 29s\tremaining: 5m 5s\n",
      "328:\ttest: 0.6349144\tbest: 0.6349602 (315)\ttotal: 2m 29s\tremaining: 5m 5s\n",
      "329:\ttest: 0.6349191\tbest: 0.6349602 (315)\ttotal: 2m 30s\tremaining: 5m 4s\n",
      "330:\ttest: 0.6349338\tbest: 0.6349602 (315)\ttotal: 2m 30s\tremaining: 5m 4s\n",
      "331:\ttest: 0.6349481\tbest: 0.6349602 (315)\ttotal: 2m 30s\tremaining: 5m 3s\n",
      "332:\ttest: 0.6349040\tbest: 0.6349602 (315)\ttotal: 2m 31s\tremaining: 5m 3s\n",
      "333:\ttest: 0.6348864\tbest: 0.6349602 (315)\ttotal: 2m 31s\tremaining: 5m 2s\n",
      "334:\ttest: 0.6348465\tbest: 0.6349602 (315)\ttotal: 2m 32s\tremaining: 5m 2s\n",
      "335:\ttest: 0.6347864\tbest: 0.6349602 (315)\ttotal: 2m 32s\tremaining: 5m 2s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "336:\ttest: 0.6347835\tbest: 0.6349602 (315)\ttotal: 2m 33s\tremaining: 5m 1s\n",
      "337:\ttest: 0.6347309\tbest: 0.6349602 (315)\ttotal: 2m 33s\tremaining: 5m 1s\n",
      "338:\ttest: 0.6347230\tbest: 0.6349602 (315)\ttotal: 2m 34s\tremaining: 5m\n",
      "339:\ttest: 0.6347118\tbest: 0.6349602 (315)\ttotal: 2m 34s\tremaining: 5m\n",
      "340:\ttest: 0.6347381\tbest: 0.6349602 (315)\ttotal: 2m 35s\tremaining: 5m\n",
      "341:\ttest: 0.6347130\tbest: 0.6349602 (315)\ttotal: 2m 35s\tremaining: 4m 59s\n",
      "342:\ttest: 0.6347167\tbest: 0.6349602 (315)\ttotal: 2m 36s\tremaining: 4m 59s\n",
      "343:\ttest: 0.6346888\tbest: 0.6349602 (315)\ttotal: 2m 36s\tremaining: 4m 58s\n",
      "344:\ttest: 0.6346836\tbest: 0.6349602 (315)\ttotal: 2m 37s\tremaining: 4m 58s\n",
      "345:\ttest: 0.6347058\tbest: 0.6349602 (315)\ttotal: 2m 37s\tremaining: 4m 58s\n",
      "346:\ttest: 0.6347103\tbest: 0.6349602 (315)\ttotal: 2m 38s\tremaining: 4m 57s\n",
      "347:\ttest: 0.6347412\tbest: 0.6349602 (315)\ttotal: 2m 38s\tremaining: 4m 57s\n",
      "348:\ttest: 0.6347341\tbest: 0.6349602 (315)\ttotal: 2m 39s\tremaining: 4m 57s\n",
      "349:\ttest: 0.6347710\tbest: 0.6349602 (315)\ttotal: 2m 39s\tremaining: 4m 56s\n",
      "350:\ttest: 0.6347575\tbest: 0.6349602 (315)\ttotal: 2m 40s\tremaining: 4m 56s\n",
      "351:\ttest: 0.6347534\tbest: 0.6349602 (315)\ttotal: 2m 40s\tremaining: 4m 55s\n",
      "352:\ttest: 0.6347254\tbest: 0.6349602 (315)\ttotal: 2m 40s\tremaining: 4m 54s\n",
      "353:\ttest: 0.6347441\tbest: 0.6349602 (315)\ttotal: 2m 41s\tremaining: 4m 54s\n",
      "354:\ttest: 0.6347586\tbest: 0.6349602 (315)\ttotal: 2m 41s\tremaining: 4m 53s\n",
      "355:\ttest: 0.6347535\tbest: 0.6349602 (315)\ttotal: 2m 42s\tremaining: 4m 53s\n",
      "356:\ttest: 0.6347995\tbest: 0.6349602 (315)\ttotal: 2m 42s\tremaining: 4m 52s\n",
      "357:\ttest: 0.6348170\tbest: 0.6349602 (315)\ttotal: 2m 43s\tremaining: 4m 52s\n",
      "358:\ttest: 0.6348163\tbest: 0.6349602 (315)\ttotal: 2m 43s\tremaining: 4m 52s\n",
      "359:\ttest: 0.6348010\tbest: 0.6349602 (315)\ttotal: 2m 44s\tremaining: 4m 51s\n",
      "360:\ttest: 0.6348241\tbest: 0.6349602 (315)\ttotal: 2m 44s\tremaining: 4m 51s\n",
      "361:\ttest: 0.6348677\tbest: 0.6349602 (315)\ttotal: 2m 45s\tremaining: 4m 50s\n",
      "362:\ttest: 0.6348819\tbest: 0.6349602 (315)\ttotal: 2m 45s\tremaining: 4m 50s\n",
      "363:\ttest: 0.6349088\tbest: 0.6349602 (315)\ttotal: 2m 45s\tremaining: 4m 49s\n",
      "364:\ttest: 0.6348696\tbest: 0.6349602 (315)\ttotal: 2m 46s\tremaining: 4m 49s\n",
      "365:\ttest: 0.6348957\tbest: 0.6349602 (315)\ttotal: 2m 47s\tremaining: 4m 49s\n",
      "366:\ttest: 0.6348677\tbest: 0.6349602 (315)\ttotal: 2m 47s\tremaining: 4m 49s\n",
      "367:\ttest: 0.6348612\tbest: 0.6349602 (315)\ttotal: 2m 48s\tremaining: 4m 48s\n",
      "368:\ttest: 0.6348712\tbest: 0.6349602 (315)\ttotal: 2m 48s\tremaining: 4m 48s\n",
      "369:\ttest: 0.6348283\tbest: 0.6349602 (315)\ttotal: 2m 49s\tremaining: 4m 47s\n",
      "370:\ttest: 0.6348250\tbest: 0.6349602 (315)\ttotal: 2m 49s\tremaining: 4m 47s\n",
      "371:\ttest: 0.6348294\tbest: 0.6349602 (315)\ttotal: 2m 50s\tremaining: 4m 47s\n",
      "372:\ttest: 0.6348215\tbest: 0.6349602 (315)\ttotal: 2m 50s\tremaining: 4m 46s\n",
      "373:\ttest: 0.6348245\tbest: 0.6349602 (315)\ttotal: 2m 51s\tremaining: 4m 46s\n",
      "374:\ttest: 0.6348810\tbest: 0.6349602 (315)\ttotal: 2m 51s\tremaining: 4m 46s\n",
      "375:\ttest: 0.6348635\tbest: 0.6349602 (315)\ttotal: 2m 52s\tremaining: 4m 45s\n",
      "376:\ttest: 0.6348930\tbest: 0.6349602 (315)\ttotal: 2m 52s\tremaining: 4m 45s\n",
      "377:\ttest: 0.6348752\tbest: 0.6349602 (315)\ttotal: 2m 53s\tremaining: 4m 44s\n",
      "378:\ttest: 0.6348685\tbest: 0.6349602 (315)\ttotal: 2m 53s\tremaining: 4m 43s\n",
      "379:\ttest: 0.6348695\tbest: 0.6349602 (315)\ttotal: 2m 53s\tremaining: 4m 43s\n",
      "380:\ttest: 0.6348318\tbest: 0.6349602 (315)\ttotal: 2m 54s\tremaining: 4m 43s\n",
      "381:\ttest: 0.6348311\tbest: 0.6349602 (315)\ttotal: 2m 54s\tremaining: 4m 42s\n",
      "382:\ttest: 0.6348590\tbest: 0.6349602 (315)\ttotal: 2m 55s\tremaining: 4m 42s\n",
      "383:\ttest: 0.6348622\tbest: 0.6349602 (315)\ttotal: 2m 55s\tremaining: 4m 41s\n",
      "384:\ttest: 0.6348161\tbest: 0.6349602 (315)\ttotal: 2m 55s\tremaining: 4m 41s\n",
      "385:\ttest: 0.6348431\tbest: 0.6349602 (315)\ttotal: 2m 56s\tremaining: 4m 40s\n",
      "386:\ttest: 0.6348333\tbest: 0.6349602 (315)\ttotal: 2m 56s\tremaining: 4m 40s\n",
      "387:\ttest: 0.6348527\tbest: 0.6349602 (315)\ttotal: 2m 57s\tremaining: 4m 39s\n",
      "388:\ttest: 0.6348375\tbest: 0.6349602 (315)\ttotal: 2m 57s\tremaining: 4m 39s\n",
      "389:\ttest: 0.6348318\tbest: 0.6349602 (315)\ttotal: 2m 58s\tremaining: 4m 39s\n",
      "390:\ttest: 0.6348646\tbest: 0.6349602 (315)\ttotal: 2m 59s\tremaining: 4m 38s\n",
      "391:\ttest: 0.6348406\tbest: 0.6349602 (315)\ttotal: 2m 59s\tremaining: 4m 38s\n",
      "392:\ttest: 0.6348494\tbest: 0.6349602 (315)\ttotal: 2m 59s\tremaining: 4m 37s\n",
      "393:\ttest: 0.6348464\tbest: 0.6349602 (315)\ttotal: 3m\tremaining: 4m 37s\n",
      "394:\ttest: 0.6347873\tbest: 0.6349602 (315)\ttotal: 3m\tremaining: 4m 37s\n",
      "395:\ttest: 0.6347196\tbest: 0.6349602 (315)\ttotal: 3m 1s\tremaining: 4m 36s\n",
      "396:\ttest: 0.6347262\tbest: 0.6349602 (315)\ttotal: 3m 1s\tremaining: 4m 36s\n",
      "397:\ttest: 0.6347279\tbest: 0.6349602 (315)\ttotal: 3m 2s\tremaining: 4m 35s\n",
      "398:\ttest: 0.6347105\tbest: 0.6349602 (315)\ttotal: 3m 2s\tremaining: 4m 35s\n",
      "399:\ttest: 0.6347035\tbest: 0.6349602 (315)\ttotal: 3m 3s\tremaining: 4m 34s\n",
      "400:\ttest: 0.6347413\tbest: 0.6349602 (315)\ttotal: 3m 3s\tremaining: 4m 34s\n",
      "401:\ttest: 0.6347470\tbest: 0.6349602 (315)\ttotal: 3m 4s\tremaining: 4m 33s\n",
      "402:\ttest: 0.6347461\tbest: 0.6349602 (315)\ttotal: 3m 4s\tremaining: 4m 33s\n",
      "403:\ttest: 0.6347326\tbest: 0.6349602 (315)\ttotal: 3m 5s\tremaining: 4m 32s\n",
      "404:\ttest: 0.6347167\tbest: 0.6349602 (315)\ttotal: 3m 5s\tremaining: 4m 32s\n",
      "405:\ttest: 0.6346931\tbest: 0.6349602 (315)\ttotal: 3m 6s\tremaining: 4m 32s\n",
      "406:\ttest: 0.6347001\tbest: 0.6349602 (315)\ttotal: 3m 6s\tremaining: 4m 31s\n",
      "407:\ttest: 0.6347107\tbest: 0.6349602 (315)\ttotal: 3m 7s\tremaining: 4m 31s\n",
      "408:\ttest: 0.6346764\tbest: 0.6349602 (315)\ttotal: 3m 7s\tremaining: 4m 31s\n",
      "409:\ttest: 0.6346198\tbest: 0.6349602 (315)\ttotal: 3m 8s\tremaining: 4m 30s\n",
      "410:\ttest: 0.6345943\tbest: 0.6349602 (315)\ttotal: 3m 8s\tremaining: 4m 30s\n",
      "411:\ttest: 0.6346117\tbest: 0.6349602 (315)\ttotal: 3m 9s\tremaining: 4m 29s\n",
      "412:\ttest: 0.6346673\tbest: 0.6349602 (315)\ttotal: 3m 9s\tremaining: 4m 29s\n",
      "413:\ttest: 0.6346699\tbest: 0.6349602 (315)\ttotal: 3m 10s\tremaining: 4m 29s\n",
      "414:\ttest: 0.6346815\tbest: 0.6349602 (315)\ttotal: 3m 10s\tremaining: 4m 28s\n",
      "415:\ttest: 0.6346899\tbest: 0.6349602 (315)\ttotal: 3m 10s\tremaining: 4m 28s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.634960244\n",
      "bestIteration = 315\n",
      "\n",
      "Shrink model to first 316 iterations.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x2ee82831908>"
      ]
     },
     "execution_count": 238,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_model = CatBoostClassifier(n_estimators=1000, # use large n_estimators deliberately to make use of the early stopping\n",
    "#                          reg_lambda=1.0,\n",
    "#                          l2_leaf_reg=4.0,\n",
    "                         eval_metric='AUC',\n",
    "                         random_seed=1994,\n",
    "                         learning_rate = 0.05,\n",
    "                         depth = 8,\n",
    "                               \n",
    "#                                boosting_type = 'Ordered',\n",
    "#                          subsample = 0.8\n",
    "                         #rsm = 0.7,\n",
    "                         #silent=True,\n",
    "                         #max_ctr_complexity = 5,  # no of categorical cols combined\n",
    "#                          boosting_type = 'Ordered',\n",
    "#                          od_type = 'IncToDec',  #overfitting params\n",
    "#                          od_wait = 20)\n",
    "                         #bagging_temperature = 1.0)\n",
    "                              )\n",
    "# lr=0.05, no od type of vars -- highest\n",
    "    \n",
    "cat_model.fit(X_train.values,y_train.values,cat_features=categorical_features_indices,eval_set=(X_val, y_val),\n",
    "        plot=False,early_stopping_rounds=100,use_best_model=True) # early stopping set to 100 to prevent overfitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Feature Importances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 239,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(19.926034611403992, 'user_id'),\n",
       " (9.755135140614396, 'count_view'),\n",
       " (9.03019067461406, 'date'),\n",
       " (8.808414260522698, 'webpage_id'),\n",
       " (8.523766746737548, 'product'),\n",
       " (7.144021224638525, 'minute'),\n",
       " (7.139652646729797, 'hour'),\n",
       " (6.623665064731843, 'campaign_id'),\n",
       " (5.509184757880177, 'product_category_1'),\n",
       " (3.4860206749139264, 'city_development_index'),\n",
       " (3.254264701296693, 'agelevel_userdepth'),\n",
       " (2.9125578829313605, 'user_group_id'),\n",
       " (2.910917398933745, 'var_1'),\n",
       " (2.790281194788812, 'age_level'),\n",
       " (2.1858930192624055, 'count_interest')]"
      ]
     },
     "execution_count": 239,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# feature importance - duration month is most important variable followed by poi\n",
    "sorted(zip(cat_model.feature_importances_,X_train),reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 240,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'depth': 8,\n",
       " 'eval_metric': 'AUC',\n",
       " 'learning_rate': 0.05,\n",
       " 'loss_function': 'Logloss',\n",
       " 'n_estimators': 1000,\n",
       " 'random_seed': 1994}"
      ]
     },
     "execution_count": 240,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_model.best_score_\n",
    "cat_model.get_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Predictions on Validation set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 242,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6349602439611522\n"
     ]
    }
   ],
   "source": [
    "preds = cat_model.predict_proba(X_val)[:,1]\n",
    "print(roc_auc_score(y_val,preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Taking the whole dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 244,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(463291, 15) (128858, 15)\n"
     ]
    }
   ],
   "source": [
    "X,y=df.drop(['session_id','is_click','user_depth','is_prod2'],axis=1),df['is_click']\n",
    "Xtest=dftest.drop(['session_id','user_depth','is_prod2'],axis=1)\n",
    "\n",
    "print(X.shape,Xtest.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 245,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\ttest: 0.5885829\tbest: 0.5885829 (0)\ttotal: 742ms\tremaining: 12m 21s\n",
      "1:\ttest: 0.5906916\tbest: 0.5906916 (1)\ttotal: 1.48s\tremaining: 12m 17s\n",
      "2:\ttest: 0.5938645\tbest: 0.5938645 (2)\ttotal: 2.05s\tremaining: 11m 21s\n",
      "3:\ttest: 0.5941771\tbest: 0.5941771 (3)\ttotal: 2.65s\tremaining: 11m 1s\n",
      "4:\ttest: 0.5942992\tbest: 0.5942992 (4)\ttotal: 3.4s\tremaining: 11m 17s\n",
      "5:\ttest: 0.5959203\tbest: 0.5959203 (5)\ttotal: 4.11s\tremaining: 11m 21s\n",
      "6:\ttest: 0.5967858\tbest: 0.5967858 (6)\ttotal: 4.81s\tremaining: 11m 21s\n",
      "7:\ttest: 0.5976066\tbest: 0.5976066 (7)\ttotal: 5.56s\tremaining: 11m 28s\n",
      "8:\ttest: 0.5991746\tbest: 0.5991746 (8)\ttotal: 6.33s\tremaining: 11m 36s\n",
      "9:\ttest: 0.6002171\tbest: 0.6002171 (9)\ttotal: 6.94s\tremaining: 11m 26s\n",
      "10:\ttest: 0.6020737\tbest: 0.6020737 (10)\ttotal: 7.66s\tremaining: 11m 28s\n",
      "11:\ttest: 0.6024914\tbest: 0.6024914 (11)\ttotal: 8.43s\tremaining: 11m 33s\n",
      "12:\ttest: 0.6023937\tbest: 0.6024914 (11)\ttotal: 9.07s\tremaining: 11m 28s\n",
      "13:\ttest: 0.6029978\tbest: 0.6029978 (13)\ttotal: 9.7s\tremaining: 11m 23s\n",
      "14:\ttest: 0.6040512\tbest: 0.6040512 (14)\ttotal: 10.4s\tremaining: 11m 22s\n",
      "15:\ttest: 0.6044287\tbest: 0.6044287 (15)\ttotal: 11.1s\tremaining: 11m 21s\n",
      "16:\ttest: 0.6051034\tbest: 0.6051034 (16)\ttotal: 11.8s\tremaining: 11m 24s\n",
      "17:\ttest: 0.6056339\tbest: 0.6056339 (17)\ttotal: 12.5s\tremaining: 11m 22s\n",
      "18:\ttest: 0.6062426\tbest: 0.6062426 (18)\ttotal: 13.3s\tremaining: 11m 25s\n",
      "19:\ttest: 0.6067989\tbest: 0.6067989 (19)\ttotal: 14s\tremaining: 11m 25s\n",
      "20:\ttest: 0.6078641\tbest: 0.6078641 (20)\ttotal: 14.8s\tremaining: 11m 29s\n",
      "21:\ttest: 0.6077879\tbest: 0.6078641 (20)\ttotal: 15.5s\tremaining: 11m 31s\n",
      "22:\ttest: 0.6075971\tbest: 0.6078641 (20)\ttotal: 16.3s\tremaining: 11m 31s\n",
      "23:\ttest: 0.6078856\tbest: 0.6078856 (23)\ttotal: 17s\tremaining: 11m 32s\n",
      "24:\ttest: 0.6075497\tbest: 0.6078856 (23)\ttotal: 17.7s\tremaining: 11m 30s\n",
      "25:\ttest: 0.6082524\tbest: 0.6082524 (25)\ttotal: 18.5s\tremaining: 11m 31s\n",
      "26:\ttest: 0.6086745\tbest: 0.6086745 (26)\ttotal: 19.2s\tremaining: 11m 32s\n",
      "27:\ttest: 0.6087114\tbest: 0.6087114 (27)\ttotal: 19.9s\tremaining: 11m 29s\n",
      "28:\ttest: 0.6086063\tbest: 0.6087114 (27)\ttotal: 20.6s\tremaining: 11m 30s\n",
      "29:\ttest: 0.6083252\tbest: 0.6087114 (27)\ttotal: 21.4s\tremaining: 11m 33s\n",
      "30:\ttest: 0.6086287\tbest: 0.6087114 (27)\ttotal: 22.2s\tremaining: 11m 32s\n",
      "31:\ttest: 0.6088326\tbest: 0.6088326 (31)\ttotal: 22.9s\tremaining: 11m 33s\n",
      "32:\ttest: 0.6088407\tbest: 0.6088407 (32)\ttotal: 23.7s\tremaining: 11m 34s\n",
      "33:\ttest: 0.6091595\tbest: 0.6091595 (33)\ttotal: 24.4s\tremaining: 11m 32s\n",
      "34:\ttest: 0.6096059\tbest: 0.6096059 (34)\ttotal: 25.1s\tremaining: 11m 32s\n",
      "35:\ttest: 0.6095014\tbest: 0.6096059 (34)\ttotal: 25.9s\tremaining: 11m 32s\n",
      "36:\ttest: 0.6094718\tbest: 0.6096059 (34)\ttotal: 26.6s\tremaining: 11m 33s\n",
      "37:\ttest: 0.6098507\tbest: 0.6098507 (37)\ttotal: 27.4s\tremaining: 11m 33s\n",
      "38:\ttest: 0.6098789\tbest: 0.6098789 (38)\ttotal: 28.2s\tremaining: 11m 35s\n",
      "39:\ttest: 0.6098564\tbest: 0.6098789 (38)\ttotal: 29.2s\tremaining: 11m 40s\n",
      "40:\ttest: 0.6099663\tbest: 0.6099663 (40)\ttotal: 30.1s\tremaining: 11m 44s\n",
      "41:\ttest: 0.6098309\tbest: 0.6099663 (40)\ttotal: 31s\tremaining: 11m 47s\n",
      "42:\ttest: 0.6099772\tbest: 0.6099772 (42)\ttotal: 31.8s\tremaining: 11m 47s\n",
      "43:\ttest: 0.6100672\tbest: 0.6100672 (43)\ttotal: 32.7s\tremaining: 11m 51s\n",
      "44:\ttest: 0.6099702\tbest: 0.6100672 (43)\ttotal: 33.6s\tremaining: 11m 53s\n",
      "45:\ttest: 0.6102001\tbest: 0.6102001 (45)\ttotal: 34.5s\tremaining: 11m 54s\n",
      "46:\ttest: 0.6421091\tbest: 0.6421091 (46)\ttotal: 35.2s\tremaining: 11m 53s\n",
      "47:\ttest: 0.6686328\tbest: 0.6686328 (47)\ttotal: 35.6s\tremaining: 11m 46s\n",
      "48:\ttest: 0.6895512\tbest: 0.6895512 (48)\ttotal: 36.3s\tremaining: 11m 45s\n",
      "49:\ttest: 0.7085375\tbest: 0.7085375 (49)\ttotal: 37s\tremaining: 11m 43s\n",
      "50:\ttest: 0.7245564\tbest: 0.7245564 (50)\ttotal: 37.7s\tremaining: 11m 41s\n",
      "51:\ttest: 0.7385175\tbest: 0.7385175 (51)\ttotal: 38.3s\tremaining: 11m 38s\n",
      "52:\ttest: 0.7516664\tbest: 0.7516664 (52)\ttotal: 38.9s\tremaining: 11m 34s\n",
      "53:\ttest: 0.7601274\tbest: 0.7601274 (53)\ttotal: 39.6s\tremaining: 11m 33s\n",
      "54:\ttest: 0.7694848\tbest: 0.7694848 (54)\ttotal: 40.1s\tremaining: 11m 29s\n",
      "55:\ttest: 0.7781411\tbest: 0.7781411 (55)\ttotal: 40.6s\tremaining: 11m 23s\n",
      "56:\ttest: 0.7838728\tbest: 0.7838728 (56)\ttotal: 41.2s\tremaining: 11m 22s\n",
      "57:\ttest: 0.7885192\tbest: 0.7885192 (57)\ttotal: 42s\tremaining: 11m 21s\n",
      "58:\ttest: 0.7936232\tbest: 0.7936232 (58)\ttotal: 42.5s\tremaining: 11m 18s\n",
      "59:\ttest: 0.7984883\tbest: 0.7984883 (59)\ttotal: 43.2s\tremaining: 11m 16s\n",
      "60:\ttest: 0.8018220\tbest: 0.8018220 (60)\ttotal: 43.8s\tremaining: 11m 14s\n",
      "61:\ttest: 0.8052112\tbest: 0.8052112 (61)\ttotal: 44.5s\tremaining: 11m 13s\n",
      "62:\ttest: 0.8096513\tbest: 0.8096513 (62)\ttotal: 45.2s\tremaining: 11m 11s\n",
      "63:\ttest: 0.8129837\tbest: 0.8129837 (63)\ttotal: 45.8s\tremaining: 11m 10s\n",
      "64:\ttest: 0.8156318\tbest: 0.8156318 (64)\ttotal: 46.6s\tremaining: 11m 9s\n",
      "65:\ttest: 0.8181915\tbest: 0.8181915 (65)\ttotal: 47.3s\tremaining: 11m 9s\n",
      "66:\ttest: 0.8203846\tbest: 0.8203846 (66)\ttotal: 48.1s\tremaining: 11m 9s\n",
      "67:\ttest: 0.8212102\tbest: 0.8212102 (67)\ttotal: 48.9s\tremaining: 11m 9s\n",
      "68:\ttest: 0.8230875\tbest: 0.8230875 (68)\ttotal: 49.6s\tremaining: 11m 8s\n",
      "69:\ttest: 0.8248972\tbest: 0.8248972 (69)\ttotal: 50.2s\tremaining: 11m 6s\n",
      "70:\ttest: 0.8260631\tbest: 0.8260631 (70)\ttotal: 50.9s\tremaining: 11m 5s\n",
      "71:\ttest: 0.8271331\tbest: 0.8271331 (71)\ttotal: 51.7s\tremaining: 11m 6s\n",
      "72:\ttest: 0.8277005\tbest: 0.8277005 (72)\ttotal: 52.5s\tremaining: 11m 6s\n",
      "73:\ttest: 0.8290879\tbest: 0.8290879 (73)\ttotal: 53.2s\tremaining: 11m 6s\n",
      "74:\ttest: 0.8300509\tbest: 0.8300509 (74)\ttotal: 54s\tremaining: 11m 5s\n",
      "75:\ttest: 0.8308323\tbest: 0.8308323 (75)\ttotal: 54.6s\tremaining: 11m 3s\n",
      "76:\ttest: 0.8316181\tbest: 0.8316181 (76)\ttotal: 55.1s\tremaining: 11m\n",
      "77:\ttest: 0.8319414\tbest: 0.8319414 (77)\ttotal: 55.8s\tremaining: 10m 59s\n",
      "78:\ttest: 0.8328177\tbest: 0.8328177 (78)\ttotal: 56.5s\tremaining: 10m 58s\n",
      "79:\ttest: 0.8338199\tbest: 0.8338199 (79)\ttotal: 57.1s\tremaining: 10m 56s\n",
      "80:\ttest: 0.8347477\tbest: 0.8347477 (80)\ttotal: 57.7s\tremaining: 10m 54s\n",
      "81:\ttest: 0.8351696\tbest: 0.8351696 (81)\ttotal: 58.4s\tremaining: 10m 53s\n",
      "82:\ttest: 0.8359059\tbest: 0.8359059 (82)\ttotal: 59s\tremaining: 10m 51s\n",
      "83:\ttest: 0.8359871\tbest: 0.8359871 (83)\ttotal: 59.7s\tremaining: 10m 50s\n",
      "84:\ttest: 0.8362843\tbest: 0.8362843 (84)\ttotal: 1m\tremaining: 10m 49s\n",
      "85:\ttest: 0.8371007\tbest: 0.8371007 (85)\ttotal: 1m 1s\tremaining: 10m 48s\n",
      "86:\ttest: 0.8394618\tbest: 0.8394618 (86)\ttotal: 1m 1s\tremaining: 10m 47s\n",
      "87:\ttest: 0.8414690\tbest: 0.8414690 (87)\ttotal: 1m 2s\tremaining: 10m 46s\n",
      "88:\ttest: 0.8439037\tbest: 0.8439037 (88)\ttotal: 1m 3s\tremaining: 10m 44s\n",
      "89:\ttest: 0.8447968\tbest: 0.8447968 (89)\ttotal: 1m 3s\tremaining: 10m 43s\n",
      "90:\ttest: 0.8461224\tbest: 0.8461224 (90)\ttotal: 1m 4s\tremaining: 10m 42s\n",
      "91:\ttest: 0.8475944\tbest: 0.8475944 (91)\ttotal: 1m 4s\tremaining: 10m 40s\n",
      "92:\ttest: 0.8487065\tbest: 0.8487065 (92)\ttotal: 1m 5s\tremaining: 10m 37s\n",
      "93:\ttest: 0.8500332\tbest: 0.8500332 (93)\ttotal: 1m 5s\tremaining: 10m 34s\n",
      "94:\ttest: 0.8494880\tbest: 0.8500332 (93)\ttotal: 1m 6s\tremaining: 10m 35s\n",
      "95:\ttest: 0.8504202\tbest: 0.8504202 (95)\ttotal: 1m 7s\tremaining: 10m 33s\n",
      "96:\ttest: 0.8499178\tbest: 0.8504202 (95)\ttotal: 1m 7s\tremaining: 10m 31s\n",
      "97:\ttest: 0.8502939\tbest: 0.8504202 (95)\ttotal: 1m 8s\tremaining: 10m 31s\n",
      "98:\ttest: 0.8506204\tbest: 0.8506204 (98)\ttotal: 1m 9s\tremaining: 10m 29s\n",
      "99:\ttest: 0.8517305\tbest: 0.8517305 (99)\ttotal: 1m 9s\tremaining: 10m 29s\n",
      "100:\ttest: 0.8523616\tbest: 0.8523616 (100)\ttotal: 1m 10s\tremaining: 10m 30s\n",
      "101:\ttest: 0.8530125\tbest: 0.8530125 (101)\ttotal: 1m 11s\tremaining: 10m 29s\n",
      "102:\ttest: 0.8535430\tbest: 0.8535430 (102)\ttotal: 1m 12s\tremaining: 10m 28s\n",
      "103:\ttest: 0.8534755\tbest: 0.8535430 (102)\ttotal: 1m 12s\tremaining: 10m 27s\n",
      "104:\ttest: 0.8532318\tbest: 0.8535430 (102)\ttotal: 1m 13s\tremaining: 10m 27s\n",
      "105:\ttest: 0.8544123\tbest: 0.8544123 (105)\ttotal: 1m 14s\tremaining: 10m 26s\n",
      "106:\ttest: 0.8550720\tbest: 0.8550720 (106)\ttotal: 1m 15s\tremaining: 10m 27s\n",
      "107:\ttest: 0.8553196\tbest: 0.8553196 (107)\ttotal: 1m 15s\tremaining: 10m 25s\n",
      "108:\ttest: 0.8561216\tbest: 0.8561216 (108)\ttotal: 1m 16s\tremaining: 10m 24s\n",
      "109:\ttest: 0.8566637\tbest: 0.8566637 (109)\ttotal: 1m 17s\tremaining: 10m 24s\n",
      "110:\ttest: 0.8573291\tbest: 0.8573291 (110)\ttotal: 1m 18s\tremaining: 10m 24s\n",
      "111:\ttest: 0.8573649\tbest: 0.8573649 (111)\ttotal: 1m 18s\tremaining: 10m 25s\n",
      "112:\ttest: 0.8580085\tbest: 0.8580085 (112)\ttotal: 1m 19s\tremaining: 10m 26s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "113:\ttest: 0.8583492\tbest: 0.8583492 (113)\ttotal: 1m 20s\tremaining: 10m 25s\n",
      "114:\ttest: 0.8584387\tbest: 0.8584387 (114)\ttotal: 1m 21s\tremaining: 10m 25s\n",
      "115:\ttest: 0.8584629\tbest: 0.8584629 (115)\ttotal: 1m 22s\tremaining: 10m 24s\n",
      "116:\ttest: 0.8590576\tbest: 0.8590576 (116)\ttotal: 1m 22s\tremaining: 10m 25s\n",
      "117:\ttest: 0.8596070\tbest: 0.8596070 (117)\ttotal: 1m 23s\tremaining: 10m 24s\n",
      "118:\ttest: 0.8600132\tbest: 0.8600132 (118)\ttotal: 1m 24s\tremaining: 10m 23s\n",
      "119:\ttest: 0.8599427\tbest: 0.8600132 (118)\ttotal: 1m 24s\tremaining: 10m 22s\n",
      "120:\ttest: 0.8598606\tbest: 0.8600132 (118)\ttotal: 1m 25s\tremaining: 10m 21s\n",
      "121:\ttest: 0.8602385\tbest: 0.8602385 (121)\ttotal: 1m 26s\tremaining: 10m 21s\n",
      "122:\ttest: 0.8604256\tbest: 0.8604256 (122)\ttotal: 1m 27s\tremaining: 10m 20s\n",
      "123:\ttest: 0.8604879\tbest: 0.8604879 (123)\ttotal: 1m 27s\tremaining: 10m 21s\n",
      "124:\ttest: 0.8605805\tbest: 0.8605805 (124)\ttotal: 1m 28s\tremaining: 10m 21s\n",
      "125:\ttest: 0.8606343\tbest: 0.8606343 (125)\ttotal: 1m 29s\tremaining: 10m 19s\n",
      "126:\ttest: 0.8604887\tbest: 0.8606343 (125)\ttotal: 1m 30s\tremaining: 10m 18s\n",
      "127:\ttest: 0.8604798\tbest: 0.8606343 (125)\ttotal: 1m 30s\tremaining: 10m 18s\n",
      "128:\ttest: 0.8605905\tbest: 0.8606343 (125)\ttotal: 1m 31s\tremaining: 10m 17s\n",
      "129:\ttest: 0.8608890\tbest: 0.8608890 (129)\ttotal: 1m 32s\tremaining: 10m 16s\n",
      "130:\ttest: 0.8608775\tbest: 0.8608890 (129)\ttotal: 1m 32s\tremaining: 10m 16s\n",
      "131:\ttest: 0.8607156\tbest: 0.8608890 (129)\ttotal: 1m 33s\tremaining: 10m 15s\n",
      "132:\ttest: 0.8610455\tbest: 0.8610455 (132)\ttotal: 1m 34s\tremaining: 10m 15s\n",
      "133:\ttest: 0.8609654\tbest: 0.8610455 (132)\ttotal: 1m 35s\tremaining: 10m 15s\n",
      "134:\ttest: 0.8609499\tbest: 0.8610455 (132)\ttotal: 1m 36s\tremaining: 10m 15s\n",
      "135:\ttest: 0.8608769\tbest: 0.8610455 (132)\ttotal: 1m 36s\tremaining: 10m 14s\n",
      "136:\ttest: 0.8608355\tbest: 0.8610455 (132)\ttotal: 1m 37s\tremaining: 10m 13s\n",
      "137:\ttest: 0.8610244\tbest: 0.8610455 (132)\ttotal: 1m 38s\tremaining: 10m 13s\n",
      "138:\ttest: 0.8610987\tbest: 0.8610987 (138)\ttotal: 1m 39s\tremaining: 10m 13s\n",
      "139:\ttest: 0.8610400\tbest: 0.8610987 (138)\ttotal: 1m 39s\tremaining: 10m 12s\n",
      "140:\ttest: 0.8609864\tbest: 0.8610987 (138)\ttotal: 1m 40s\tremaining: 10m 12s\n",
      "141:\ttest: 0.8610377\tbest: 0.8610987 (138)\ttotal: 1m 41s\tremaining: 10m 11s\n",
      "142:\ttest: 0.8610853\tbest: 0.8610987 (138)\ttotal: 1m 41s\tremaining: 10m 10s\n",
      "143:\ttest: 0.8608710\tbest: 0.8610987 (138)\ttotal: 1m 42s\tremaining: 10m 9s\n",
      "144:\ttest: 0.8608384\tbest: 0.8610987 (138)\ttotal: 1m 43s\tremaining: 10m 9s\n",
      "145:\ttest: 0.8613412\tbest: 0.8613412 (145)\ttotal: 1m 44s\tremaining: 10m 9s\n",
      "146:\ttest: 0.8612772\tbest: 0.8613412 (145)\ttotal: 1m 44s\tremaining: 10m 8s\n",
      "147:\ttest: 0.8613591\tbest: 0.8613591 (147)\ttotal: 1m 45s\tremaining: 10m 8s\n",
      "148:\ttest: 0.8612910\tbest: 0.8613591 (147)\ttotal: 1m 46s\tremaining: 10m 7s\n",
      "149:\ttest: 0.8612257\tbest: 0.8613591 (147)\ttotal: 1m 47s\tremaining: 10m 7s\n",
      "150:\ttest: 0.8611423\tbest: 0.8613591 (147)\ttotal: 1m 48s\tremaining: 10m 7s\n",
      "151:\ttest: 0.8611078\tbest: 0.8613591 (147)\ttotal: 1m 48s\tremaining: 10m 7s\n",
      "152:\ttest: 0.8611531\tbest: 0.8613591 (147)\ttotal: 1m 49s\tremaining: 10m 6s\n",
      "153:\ttest: 0.8612228\tbest: 0.8613591 (147)\ttotal: 1m 50s\tremaining: 10m 6s\n",
      "154:\ttest: 0.8611991\tbest: 0.8613591 (147)\ttotal: 1m 51s\tremaining: 10m 6s\n",
      "155:\ttest: 0.8614637\tbest: 0.8614637 (155)\ttotal: 1m 51s\tremaining: 10m 5s\n",
      "156:\ttest: 0.8612586\tbest: 0.8614637 (155)\ttotal: 1m 52s\tremaining: 10m 4s\n",
      "157:\ttest: 0.8614374\tbest: 0.8614637 (155)\ttotal: 1m 53s\tremaining: 10m 4s\n",
      "158:\ttest: 0.8614495\tbest: 0.8614637 (155)\ttotal: 1m 54s\tremaining: 10m 4s\n",
      "159:\ttest: 0.8617786\tbest: 0.8617786 (159)\ttotal: 1m 54s\tremaining: 10m 3s\n",
      "160:\ttest: 0.8618129\tbest: 0.8618129 (160)\ttotal: 1m 55s\tremaining: 10m 3s\n",
      "161:\ttest: 0.8618136\tbest: 0.8618136 (161)\ttotal: 1m 56s\tremaining: 10m 2s\n",
      "162:\ttest: 0.8614470\tbest: 0.8618136 (161)\ttotal: 1m 57s\tremaining: 10m 1s\n",
      "163:\ttest: 0.8617531\tbest: 0.8618136 (161)\ttotal: 1m 57s\tremaining: 10m 1s\n",
      "164:\ttest: 0.8617311\tbest: 0.8618136 (161)\ttotal: 1m 58s\tremaining: 10m 1s\n",
      "165:\ttest: 0.8618093\tbest: 0.8618136 (161)\ttotal: 1m 59s\tremaining: 10m\n",
      "166:\ttest: 0.8617809\tbest: 0.8618136 (161)\ttotal: 2m\tremaining: 10m\n",
      "167:\ttest: 0.8617288\tbest: 0.8618136 (161)\ttotal: 2m 1s\tremaining: 10m\n",
      "168:\ttest: 0.8616844\tbest: 0.8618136 (161)\ttotal: 2m 1s\tremaining: 9m 59s\n",
      "169:\ttest: 0.8616535\tbest: 0.8618136 (161)\ttotal: 2m 2s\tremaining: 9m 58s\n",
      "170:\ttest: 0.8615943\tbest: 0.8618136 (161)\ttotal: 2m 3s\tremaining: 9m 58s\n",
      "171:\ttest: 0.8615081\tbest: 0.8618136 (161)\ttotal: 2m 4s\tremaining: 9m 57s\n",
      "172:\ttest: 0.8614743\tbest: 0.8618136 (161)\ttotal: 2m 4s\tremaining: 9m 57s\n",
      "173:\ttest: 0.8614534\tbest: 0.8618136 (161)\ttotal: 2m 5s\tremaining: 9m 56s\n",
      "174:\ttest: 0.8612591\tbest: 0.8618136 (161)\ttotal: 2m 6s\tremaining: 9m 56s\n",
      "175:\ttest: 0.8615560\tbest: 0.8618136 (161)\ttotal: 2m 7s\tremaining: 9m 55s\n",
      "176:\ttest: 0.8615614\tbest: 0.8618136 (161)\ttotal: 2m 8s\tremaining: 9m 55s\n",
      "177:\ttest: 0.8615107\tbest: 0.8618136 (161)\ttotal: 2m 8s\tremaining: 9m 54s\n",
      "178:\ttest: 0.8615072\tbest: 0.8618136 (161)\ttotal: 2m 9s\tremaining: 9m 54s\n",
      "179:\ttest: 0.8617664\tbest: 0.8618136 (161)\ttotal: 2m 10s\tremaining: 9m 54s\n",
      "180:\ttest: 0.8618845\tbest: 0.8618845 (180)\ttotal: 2m 11s\tremaining: 9m 53s\n",
      "181:\ttest: 0.8618650\tbest: 0.8618845 (180)\ttotal: 2m 12s\tremaining: 9m 53s\n",
      "182:\ttest: 0.8618643\tbest: 0.8618845 (180)\ttotal: 2m 12s\tremaining: 9m 53s\n",
      "183:\ttest: 0.8619040\tbest: 0.8619040 (183)\ttotal: 2m 13s\tremaining: 9m 52s\n",
      "184:\ttest: 0.8618525\tbest: 0.8619040 (183)\ttotal: 2m 14s\tremaining: 9m 51s\n",
      "185:\ttest: 0.8617862\tbest: 0.8619040 (183)\ttotal: 2m 15s\tremaining: 9m 51s\n",
      "186:\ttest: 0.8618456\tbest: 0.8619040 (183)\ttotal: 2m 15s\tremaining: 9m 51s\n",
      "187:\ttest: 0.8618460\tbest: 0.8619040 (183)\ttotal: 2m 16s\tremaining: 9m 50s\n",
      "188:\ttest: 0.8618067\tbest: 0.8619040 (183)\ttotal: 2m 17s\tremaining: 9m 50s\n",
      "189:\ttest: 0.8617871\tbest: 0.8619040 (183)\ttotal: 2m 18s\tremaining: 9m 50s\n",
      "190:\ttest: 0.8618087\tbest: 0.8619040 (183)\ttotal: 2m 19s\tremaining: 9m 50s\n",
      "191:\ttest: 0.8619087\tbest: 0.8619087 (191)\ttotal: 2m 20s\tremaining: 9m 50s\n",
      "192:\ttest: 0.8620913\tbest: 0.8620913 (192)\ttotal: 2m 20s\tremaining: 9m 48s\n",
      "193:\ttest: 0.8620890\tbest: 0.8620913 (192)\ttotal: 2m 21s\tremaining: 9m 48s\n",
      "194:\ttest: 0.8620829\tbest: 0.8620913 (192)\ttotal: 2m 22s\tremaining: 9m 47s\n",
      "195:\ttest: 0.8618916\tbest: 0.8620913 (192)\ttotal: 2m 23s\tremaining: 9m 47s\n",
      "196:\ttest: 0.8618958\tbest: 0.8620913 (192)\ttotal: 2m 24s\tremaining: 9m 47s\n",
      "197:\ttest: 0.8618721\tbest: 0.8620913 (192)\ttotal: 2m 24s\tremaining: 9m 47s\n",
      "198:\ttest: 0.8618730\tbest: 0.8620913 (192)\ttotal: 2m 25s\tremaining: 9m 46s\n",
      "199:\ttest: 0.8619103\tbest: 0.8620913 (192)\ttotal: 2m 26s\tremaining: 9m 46s\n",
      "200:\ttest: 0.8619345\tbest: 0.8620913 (192)\ttotal: 2m 27s\tremaining: 9m 46s\n",
      "201:\ttest: 0.8620576\tbest: 0.8620913 (192)\ttotal: 2m 28s\tremaining: 9m 45s\n",
      "202:\ttest: 0.8620560\tbest: 0.8620913 (192)\ttotal: 2m 29s\tremaining: 9m 46s\n",
      "203:\ttest: 0.8617984\tbest: 0.8620913 (192)\ttotal: 2m 30s\tremaining: 9m 45s\n",
      "204:\ttest: 0.8617736\tbest: 0.8620913 (192)\ttotal: 2m 30s\tremaining: 9m 45s\n",
      "205:\ttest: 0.8618372\tbest: 0.8620913 (192)\ttotal: 2m 31s\tremaining: 9m 44s\n",
      "206:\ttest: 0.8619658\tbest: 0.8620913 (192)\ttotal: 2m 32s\tremaining: 9m 43s\n",
      "207:\ttest: 0.8619058\tbest: 0.8620913 (192)\ttotal: 2m 32s\tremaining: 9m 42s\n",
      "208:\ttest: 0.8617929\tbest: 0.8620913 (192)\ttotal: 2m 33s\tremaining: 9m 42s\n",
      "209:\ttest: 0.8616592\tbest: 0.8620913 (192)\ttotal: 2m 34s\tremaining: 9m 40s\n",
      "210:\ttest: 0.8616710\tbest: 0.8620913 (192)\ttotal: 2m 35s\tremaining: 9m 40s\n",
      "211:\ttest: 0.8616446\tbest: 0.8620913 (192)\ttotal: 2m 36s\tremaining: 9m 40s\n",
      "212:\ttest: 0.8615587\tbest: 0.8620913 (192)\ttotal: 2m 36s\tremaining: 9m 39s\n",
      "213:\ttest: 0.8614972\tbest: 0.8620913 (192)\ttotal: 2m 37s\tremaining: 9m 38s\n",
      "214:\ttest: 0.8615121\tbest: 0.8620913 (192)\ttotal: 2m 38s\tremaining: 9m 37s\n",
      "215:\ttest: 0.8613745\tbest: 0.8620913 (192)\ttotal: 2m 39s\tremaining: 9m 37s\n",
      "216:\ttest: 0.8615016\tbest: 0.8620913 (192)\ttotal: 2m 39s\tremaining: 9m 36s\n",
      "217:\ttest: 0.8615041\tbest: 0.8620913 (192)\ttotal: 2m 40s\tremaining: 9m 36s\n",
      "218:\ttest: 0.8615087\tbest: 0.8620913 (192)\ttotal: 2m 41s\tremaining: 9m 35s\n",
      "219:\ttest: 0.8616664\tbest: 0.8620913 (192)\ttotal: 2m 42s\tremaining: 9m 35s\n",
      "220:\ttest: 0.8613919\tbest: 0.8620913 (192)\ttotal: 2m 42s\tremaining: 9m 34s\n",
      "221:\ttest: 0.8613959\tbest: 0.8620913 (192)\ttotal: 2m 43s\tremaining: 9m 34s\n",
      "222:\ttest: 0.8613207\tbest: 0.8620913 (192)\ttotal: 2m 44s\tremaining: 9m 33s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "223:\ttest: 0.8612331\tbest: 0.8620913 (192)\ttotal: 2m 45s\tremaining: 9m 32s\n",
      "224:\ttest: 0.8613080\tbest: 0.8620913 (192)\ttotal: 2m 45s\tremaining: 9m 31s\n",
      "225:\ttest: 0.8607290\tbest: 0.8620913 (192)\ttotal: 2m 46s\tremaining: 9m 30s\n",
      "226:\ttest: 0.8607355\tbest: 0.8620913 (192)\ttotal: 2m 47s\tremaining: 9m 30s\n",
      "227:\ttest: 0.8603109\tbest: 0.8620913 (192)\ttotal: 2m 48s\tremaining: 9m 29s\n",
      "228:\ttest: 0.8604214\tbest: 0.8620913 (192)\ttotal: 2m 48s\tremaining: 9m 28s\n",
      "229:\ttest: 0.8604348\tbest: 0.8620913 (192)\ttotal: 2m 49s\tremaining: 9m 28s\n",
      "230:\ttest: 0.8604046\tbest: 0.8620913 (192)\ttotal: 2m 50s\tremaining: 9m 28s\n",
      "231:\ttest: 0.8603768\tbest: 0.8620913 (192)\ttotal: 2m 51s\tremaining: 9m 28s\n",
      "232:\ttest: 0.8603490\tbest: 0.8620913 (192)\ttotal: 2m 52s\tremaining: 9m 27s\n",
      "233:\ttest: 0.8601311\tbest: 0.8620913 (192)\ttotal: 2m 53s\tremaining: 9m 27s\n",
      "234:\ttest: 0.8600892\tbest: 0.8620913 (192)\ttotal: 2m 54s\tremaining: 9m 26s\n",
      "235:\ttest: 0.8598009\tbest: 0.8620913 (192)\ttotal: 2m 54s\tremaining: 9m 25s\n",
      "236:\ttest: 0.8598234\tbest: 0.8620913 (192)\ttotal: 2m 55s\tremaining: 9m 25s\n",
      "237:\ttest: 0.8598036\tbest: 0.8620913 (192)\ttotal: 2m 56s\tremaining: 9m 24s\n",
      "238:\ttest: 0.8596521\tbest: 0.8620913 (192)\ttotal: 2m 57s\tremaining: 9m 24s\n",
      "239:\ttest: 0.8592416\tbest: 0.8620913 (192)\ttotal: 2m 58s\tremaining: 9m 23s\n",
      "240:\ttest: 0.8593316\tbest: 0.8620913 (192)\ttotal: 2m 58s\tremaining: 9m 22s\n",
      "241:\ttest: 0.8593542\tbest: 0.8620913 (192)\ttotal: 2m 59s\tremaining: 9m 22s\n",
      "242:\ttest: 0.8593792\tbest: 0.8620913 (192)\ttotal: 3m\tremaining: 9m 21s\n",
      "243:\ttest: 0.8592070\tbest: 0.8620913 (192)\ttotal: 3m\tremaining: 9m 20s\n",
      "244:\ttest: 0.8592231\tbest: 0.8620913 (192)\ttotal: 3m 1s\tremaining: 9m 20s\n",
      "245:\ttest: 0.8592127\tbest: 0.8620913 (192)\ttotal: 3m 2s\tremaining: 9m 19s\n",
      "246:\ttest: 0.8592624\tbest: 0.8620913 (192)\ttotal: 3m 3s\tremaining: 9m 19s\n",
      "247:\ttest: 0.8592664\tbest: 0.8620913 (192)\ttotal: 3m 4s\tremaining: 9m 18s\n",
      "248:\ttest: 0.8590324\tbest: 0.8620913 (192)\ttotal: 3m 5s\tremaining: 9m 18s\n",
      "249:\ttest: 0.8588967\tbest: 0.8620913 (192)\ttotal: 3m 5s\tremaining: 9m 17s\n",
      "250:\ttest: 0.8592536\tbest: 0.8620913 (192)\ttotal: 3m 6s\tremaining: 9m 17s\n",
      "251:\ttest: 0.8592681\tbest: 0.8620913 (192)\ttotal: 3m 7s\tremaining: 9m 16s\n",
      "252:\ttest: 0.8593419\tbest: 0.8620913 (192)\ttotal: 3m 8s\tremaining: 9m 16s\n",
      "253:\ttest: 0.8592727\tbest: 0.8620913 (192)\ttotal: 3m 9s\tremaining: 9m 15s\n",
      "254:\ttest: 0.8592778\tbest: 0.8620913 (192)\ttotal: 3m 10s\tremaining: 9m 15s\n",
      "255:\ttest: 0.8592833\tbest: 0.8620913 (192)\ttotal: 3m 10s\tremaining: 9m 14s\n",
      "256:\ttest: 0.8588077\tbest: 0.8620913 (192)\ttotal: 3m 11s\tremaining: 9m 14s\n",
      "257:\ttest: 0.8588319\tbest: 0.8620913 (192)\ttotal: 3m 12s\tremaining: 9m 14s\n",
      "258:\ttest: 0.8588250\tbest: 0.8620913 (192)\ttotal: 3m 13s\tremaining: 9m 13s\n",
      "259:\ttest: 0.8586083\tbest: 0.8620913 (192)\ttotal: 3m 14s\tremaining: 9m 12s\n",
      "260:\ttest: 0.8586500\tbest: 0.8620913 (192)\ttotal: 3m 15s\tremaining: 9m 12s\n",
      "261:\ttest: 0.8586113\tbest: 0.8620913 (192)\ttotal: 3m 15s\tremaining: 9m 11s\n",
      "262:\ttest: 0.8585598\tbest: 0.8620913 (192)\ttotal: 3m 16s\tremaining: 9m 10s\n",
      "263:\ttest: 0.8585197\tbest: 0.8620913 (192)\ttotal: 3m 17s\tremaining: 9m 10s\n",
      "264:\ttest: 0.8585138\tbest: 0.8620913 (192)\ttotal: 3m 18s\tremaining: 9m 9s\n",
      "265:\ttest: 0.8583467\tbest: 0.8620913 (192)\ttotal: 3m 18s\tremaining: 9m 8s\n",
      "266:\ttest: 0.8583345\tbest: 0.8620913 (192)\ttotal: 3m 19s\tremaining: 9m 7s\n",
      "267:\ttest: 0.8583673\tbest: 0.8620913 (192)\ttotal: 3m 20s\tremaining: 9m 6s\n",
      "268:\ttest: 0.8582830\tbest: 0.8620913 (192)\ttotal: 3m 20s\tremaining: 9m 6s\n",
      "269:\ttest: 0.8583599\tbest: 0.8620913 (192)\ttotal: 3m 21s\tremaining: 9m 5s\n",
      "270:\ttest: 0.8584425\tbest: 0.8620913 (192)\ttotal: 3m 22s\tremaining: 9m 4s\n",
      "271:\ttest: 0.8584286\tbest: 0.8620913 (192)\ttotal: 3m 22s\tremaining: 9m 3s\n",
      "272:\ttest: 0.8583165\tbest: 0.8620913 (192)\ttotal: 3m 23s\tremaining: 9m 2s\n",
      "273:\ttest: 0.8580663\tbest: 0.8620913 (192)\ttotal: 3m 24s\tremaining: 9m 1s\n",
      "274:\ttest: 0.8580759\tbest: 0.8620913 (192)\ttotal: 3m 24s\tremaining: 9m\n",
      "275:\ttest: 0.8580728\tbest: 0.8620913 (192)\ttotal: 3m 25s\tremaining: 8m 59s\n",
      "276:\ttest: 0.8581244\tbest: 0.8620913 (192)\ttotal: 3m 26s\tremaining: 8m 58s\n",
      "277:\ttest: 0.8581587\tbest: 0.8620913 (192)\ttotal: 3m 27s\tremaining: 8m 58s\n",
      "278:\ttest: 0.8582470\tbest: 0.8620913 (192)\ttotal: 3m 28s\tremaining: 8m 57s\n",
      "279:\ttest: 0.8578058\tbest: 0.8620913 (192)\ttotal: 3m 28s\tremaining: 8m 57s\n",
      "280:\ttest: 0.8578918\tbest: 0.8620913 (192)\ttotal: 3m 29s\tremaining: 8m 56s\n",
      "281:\ttest: 0.8577999\tbest: 0.8620913 (192)\ttotal: 3m 30s\tremaining: 8m 55s\n",
      "282:\ttest: 0.8575184\tbest: 0.8620913 (192)\ttotal: 3m 31s\tremaining: 8m 54s\n",
      "283:\ttest: 0.8575355\tbest: 0.8620913 (192)\ttotal: 3m 31s\tremaining: 8m 54s\n",
      "284:\ttest: 0.8574084\tbest: 0.8620913 (192)\ttotal: 3m 32s\tremaining: 8m 52s\n",
      "285:\ttest: 0.8573755\tbest: 0.8620913 (192)\ttotal: 3m 33s\tremaining: 8m 52s\n",
      "286:\ttest: 0.8574510\tbest: 0.8620913 (192)\ttotal: 3m 33s\tremaining: 8m 51s\n",
      "287:\ttest: 0.8574502\tbest: 0.8620913 (192)\ttotal: 3m 34s\tremaining: 8m 50s\n",
      "288:\ttest: 0.8574474\tbest: 0.8620913 (192)\ttotal: 3m 35s\tremaining: 8m 50s\n",
      "289:\ttest: 0.8574159\tbest: 0.8620913 (192)\ttotal: 3m 36s\tremaining: 8m 49s\n",
      "290:\ttest: 0.8574253\tbest: 0.8620913 (192)\ttotal: 3m 37s\tremaining: 8m 48s\n",
      "291:\ttest: 0.8571395\tbest: 0.8620913 (192)\ttotal: 3m 37s\tremaining: 8m 48s\n",
      "292:\ttest: 0.8571835\tbest: 0.8620913 (192)\ttotal: 3m 38s\tremaining: 8m 47s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.8620912906\n",
      "bestIteration = 192\n",
      "\n",
      "Shrink model to first 193 iterations.\n"
     ]
    }
   ],
   "source": [
    "cat_model.fit(X,y,cat_features=categorical_features_indices,eval_set=(X, y),\n",
    "        plot=False,early_stopping_rounds=100,use_best_model=True)\n",
    "y_pred=cat_model.predict_proba(Xtest)[:,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 246,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.08732907, 0.02328576, 0.02194707, ..., 0.0474054 , 0.06777471,\n",
       "       0.05181794])"
      ]
     },
     "execution_count": 246,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 247,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s=pd.DataFrame({'session_id':test['session_id'],'is_click':y_pred})\n",
    "s.head()\n",
    "# s.to_csv('catboost1.csv',index=False) --main 62 max depth-8 lr-0.1\n",
    "s.to_csv('catboost10.csv',index=False) #--nope"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gave a score of 62.XXX on public lb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using StratifiedKFold - 5 splits "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 232,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\ttest: 0.5808461\tbest: 0.5808461 (0)\ttotal: 507ms\tremaining: 8m 26s\n",
      "1:\ttest: 0.5835546\tbest: 0.5835546 (1)\ttotal: 810ms\tremaining: 6m 43s\n",
      "2:\ttest: 0.5860790\tbest: 0.5860790 (2)\ttotal: 1.35s\tremaining: 7m 30s\n",
      "3:\ttest: 0.5855220\tbest: 0.5860790 (2)\ttotal: 1.75s\tremaining: 7m 16s\n",
      "4:\ttest: 0.5854380\tbest: 0.5860790 (2)\ttotal: 2.13s\tremaining: 7m 3s\n",
      "5:\ttest: 0.5853862\tbest: 0.5860790 (2)\ttotal: 2.6s\tremaining: 7m 10s\n",
      "6:\ttest: 0.5860316\tbest: 0.5860790 (2)\ttotal: 3.04s\tremaining: 7m 11s\n",
      "7:\ttest: 0.5852397\tbest: 0.5860790 (2)\ttotal: 3.52s\tremaining: 7m 16s\n",
      "8:\ttest: 0.5853614\tbest: 0.5860790 (2)\ttotal: 3.98s\tremaining: 7m 17s\n",
      "9:\ttest: 0.5857671\tbest: 0.5860790 (2)\ttotal: 4.38s\tremaining: 7m 13s\n",
      "10:\ttest: 0.5856640\tbest: 0.5860790 (2)\ttotal: 4.84s\tremaining: 7m 15s\n",
      "11:\ttest: 0.5858875\tbest: 0.5860790 (2)\ttotal: 5.23s\tremaining: 7m 10s\n",
      "12:\ttest: 0.5864877\tbest: 0.5864877 (12)\ttotal: 5.67s\tremaining: 7m 10s\n",
      "13:\ttest: 0.5870016\tbest: 0.5870016 (13)\ttotal: 6.13s\tremaining: 7m 11s\n",
      "14:\ttest: 0.5867941\tbest: 0.5870016 (13)\ttotal: 6.6s\tremaining: 7m 13s\n",
      "15:\ttest: 0.5869028\tbest: 0.5870016 (13)\ttotal: 7.01s\tremaining: 7m 10s\n",
      "16:\ttest: 0.5867803\tbest: 0.5870016 (13)\ttotal: 7.46s\tremaining: 7m 11s\n",
      "17:\ttest: 0.5875950\tbest: 0.5875950 (17)\ttotal: 7.91s\tremaining: 7m 11s\n",
      "18:\ttest: 0.5872095\tbest: 0.5875950 (17)\ttotal: 8.37s\tremaining: 7m 12s\n",
      "19:\ttest: 0.5873350\tbest: 0.5875950 (17)\ttotal: 8.74s\tremaining: 7m 8s\n",
      "20:\ttest: 0.5874324\tbest: 0.5875950 (17)\ttotal: 9.27s\tremaining: 7m 12s\n",
      "21:\ttest: 0.5872210\tbest: 0.5875950 (17)\ttotal: 9.73s\tremaining: 7m 12s\n",
      "22:\ttest: 0.5873847\tbest: 0.5875950 (17)\ttotal: 10.2s\tremaining: 7m 14s\n",
      "23:\ttest: 0.5881238\tbest: 0.5881238 (23)\ttotal: 10.7s\tremaining: 7m 14s\n",
      "24:\ttest: 0.5879452\tbest: 0.5881238 (23)\ttotal: 11.1s\tremaining: 7m 13s\n",
      "25:\ttest: 0.5880059\tbest: 0.5881238 (23)\ttotal: 11.6s\tremaining: 7m 14s\n",
      "26:\ttest: 0.5880724\tbest: 0.5881238 (23)\ttotal: 12.1s\tremaining: 7m 14s\n",
      "27:\ttest: 0.5883528\tbest: 0.5883528 (27)\ttotal: 12.5s\tremaining: 7m 13s\n",
      "28:\ttest: 0.5883542\tbest: 0.5883542 (28)\ttotal: 13s\tremaining: 7m 13s\n",
      "29:\ttest: 0.5887055\tbest: 0.5887055 (29)\ttotal: 13.4s\tremaining: 7m 14s\n",
      "30:\ttest: 0.5888608\tbest: 0.5888608 (30)\ttotal: 13.9s\tremaining: 7m 13s\n",
      "31:\ttest: 0.5891013\tbest: 0.5891013 (31)\ttotal: 14.3s\tremaining: 7m 14s\n",
      "32:\ttest: 0.5890180\tbest: 0.5891013 (31)\ttotal: 14.8s\tremaining: 7m 12s\n",
      "33:\ttest: 0.5893426\tbest: 0.5893426 (33)\ttotal: 15.3s\tremaining: 7m 13s\n",
      "34:\ttest: 0.5890851\tbest: 0.5893426 (33)\ttotal: 15.7s\tremaining: 7m 13s\n",
      "35:\ttest: 0.5890610\tbest: 0.5893426 (33)\ttotal: 16.2s\tremaining: 7m 12s\n",
      "36:\ttest: 0.5890069\tbest: 0.5893426 (33)\ttotal: 16.7s\tremaining: 7m 14s\n",
      "37:\ttest: 0.5889783\tbest: 0.5893426 (33)\ttotal: 17.2s\tremaining: 7m 14s\n",
      "38:\ttest: 0.5891492\tbest: 0.5893426 (33)\ttotal: 17.7s\tremaining: 7m 16s\n",
      "39:\ttest: 0.5895212\tbest: 0.5895212 (39)\ttotal: 18.2s\tremaining: 7m 16s\n",
      "40:\ttest: 0.5895906\tbest: 0.5895906 (40)\ttotal: 18.6s\tremaining: 7m 16s\n",
      "41:\ttest: 0.5895367\tbest: 0.5895906 (40)\ttotal: 19.1s\tremaining: 7m 16s\n",
      "42:\ttest: 0.5898273\tbest: 0.5898273 (42)\ttotal: 19.6s\tremaining: 7m 16s\n",
      "43:\ttest: 0.5946454\tbest: 0.5946454 (43)\ttotal: 20.1s\tremaining: 7m 16s\n",
      "44:\ttest: 0.5998172\tbest: 0.5998172 (44)\ttotal: 20.3s\tremaining: 7m 11s\n",
      "45:\ttest: 0.6043870\tbest: 0.6043870 (45)\ttotal: 20.6s\tremaining: 7m 7s\n",
      "46:\ttest: 0.6071932\tbest: 0.6071932 (46)\ttotal: 21s\tremaining: 7m 6s\n",
      "47:\ttest: 0.6107913\tbest: 0.6107913 (47)\ttotal: 21.4s\tremaining: 7m 4s\n",
      "48:\ttest: 0.6133576\tbest: 0.6133576 (48)\ttotal: 21.8s\tremaining: 7m 2s\n",
      "49:\ttest: 0.6151436\tbest: 0.6151436 (49)\ttotal: 22.2s\tremaining: 7m\n",
      "50:\ttest: 0.6175359\tbest: 0.6175359 (50)\ttotal: 22.5s\tremaining: 6m 59s\n",
      "51:\ttest: 0.6195109\tbest: 0.6195109 (51)\ttotal: 22.9s\tremaining: 6m 56s\n",
      "52:\ttest: 0.6209060\tbest: 0.6209060 (52)\ttotal: 23.2s\tremaining: 6m 54s\n",
      "53:\ttest: 0.6224685\tbest: 0.6224685 (53)\ttotal: 23.6s\tremaining: 6m 52s\n",
      "54:\ttest: 0.6231306\tbest: 0.6231306 (54)\ttotal: 24s\tremaining: 6m 52s\n",
      "55:\ttest: 0.6244113\tbest: 0.6244113 (55)\ttotal: 24.4s\tremaining: 6m 51s\n",
      "56:\ttest: 0.6248768\tbest: 0.6248768 (56)\ttotal: 24.9s\tremaining: 6m 52s\n",
      "57:\ttest: 0.6255391\tbest: 0.6255391 (57)\ttotal: 25.4s\tremaining: 6m 53s\n",
      "58:\ttest: 0.6263000\tbest: 0.6263000 (58)\ttotal: 25.7s\tremaining: 6m 50s\n",
      "59:\ttest: 0.6270657\tbest: 0.6270657 (59)\ttotal: 26.1s\tremaining: 6m 49s\n",
      "60:\ttest: 0.6274967\tbest: 0.6274967 (60)\ttotal: 26.6s\tremaining: 6m 49s\n",
      "61:\ttest: 0.6278528\tbest: 0.6278528 (61)\ttotal: 27.1s\tremaining: 6m 49s\n",
      "62:\ttest: 0.6281228\tbest: 0.6281228 (62)\ttotal: 27.6s\tremaining: 6m 50s\n",
      "63:\ttest: 0.6284350\tbest: 0.6284350 (63)\ttotal: 28.1s\tremaining: 6m 50s\n",
      "64:\ttest: 0.6290002\tbest: 0.6290002 (64)\ttotal: 28.5s\tremaining: 6m 49s\n",
      "65:\ttest: 0.6291836\tbest: 0.6291836 (65)\ttotal: 28.9s\tremaining: 6m 49s\n",
      "66:\ttest: 0.6294731\tbest: 0.6294731 (66)\ttotal: 29.4s\tremaining: 6m 48s\n",
      "67:\ttest: 0.6297912\tbest: 0.6297912 (67)\ttotal: 29.9s\tremaining: 6m 49s\n",
      "68:\ttest: 0.6298704\tbest: 0.6298704 (68)\ttotal: 30.4s\tremaining: 6m 49s\n",
      "69:\ttest: 0.6302364\tbest: 0.6302364 (69)\ttotal: 30.8s\tremaining: 6m 48s\n",
      "70:\ttest: 0.6302947\tbest: 0.6302947 (70)\ttotal: 31.2s\tremaining: 6m 48s\n",
      "71:\ttest: 0.6307644\tbest: 0.6307644 (71)\ttotal: 31.7s\tremaining: 6m 48s\n",
      "72:\ttest: 0.6310421\tbest: 0.6310421 (72)\ttotal: 32.1s\tremaining: 6m 48s\n",
      "73:\ttest: 0.6313517\tbest: 0.6313517 (73)\ttotal: 32.6s\tremaining: 6m 47s\n",
      "74:\ttest: 0.6316510\tbest: 0.6316510 (74)\ttotal: 32.9s\tremaining: 6m 45s\n",
      "75:\ttest: 0.6319362\tbest: 0.6319362 (75)\ttotal: 33.3s\tremaining: 6m 44s\n",
      "76:\ttest: 0.6321325\tbest: 0.6321325 (76)\ttotal: 33.8s\tremaining: 6m 44s\n",
      "77:\ttest: 0.6321940\tbest: 0.6321940 (77)\ttotal: 34.2s\tremaining: 6m 44s\n",
      "78:\ttest: 0.6323496\tbest: 0.6323496 (78)\ttotal: 34.7s\tremaining: 6m 44s\n",
      "79:\ttest: 0.6325517\tbest: 0.6325517 (79)\ttotal: 35.2s\tremaining: 6m 44s\n",
      "80:\ttest: 0.6326691\tbest: 0.6326691 (80)\ttotal: 35.5s\tremaining: 6m 43s\n",
      "81:\ttest: 0.6327722\tbest: 0.6327722 (81)\ttotal: 36.1s\tremaining: 6m 44s\n",
      "82:\ttest: 0.6329017\tbest: 0.6329017 (82)\ttotal: 36.6s\tremaining: 6m 44s\n",
      "83:\ttest: 0.6329362\tbest: 0.6329362 (83)\ttotal: 37.1s\tremaining: 6m 44s\n",
      "84:\ttest: 0.6330236\tbest: 0.6330236 (84)\ttotal: 37.6s\tremaining: 6m 44s\n",
      "85:\ttest: 0.6330862\tbest: 0.6330862 (85)\ttotal: 38.1s\tremaining: 6m 45s\n",
      "86:\ttest: 0.6331069\tbest: 0.6331069 (86)\ttotal: 38.7s\tremaining: 6m 45s\n",
      "87:\ttest: 0.6332820\tbest: 0.6332820 (87)\ttotal: 39.2s\tremaining: 6m 46s\n",
      "88:\ttest: 0.6332948\tbest: 0.6332948 (88)\ttotal: 39.7s\tremaining: 6m 46s\n",
      "89:\ttest: 0.6335867\tbest: 0.6335867 (89)\ttotal: 40.3s\tremaining: 6m 47s\n",
      "90:\ttest: 0.6337657\tbest: 0.6337657 (90)\ttotal: 40.8s\tremaining: 6m 47s\n",
      "91:\ttest: 0.6338170\tbest: 0.6338170 (91)\ttotal: 41.3s\tremaining: 6m 47s\n",
      "92:\ttest: 0.6339754\tbest: 0.6339754 (92)\ttotal: 41.9s\tremaining: 6m 48s\n",
      "93:\ttest: 0.6341210\tbest: 0.6341210 (93)\ttotal: 42.3s\tremaining: 6m 48s\n",
      "94:\ttest: 0.6343647\tbest: 0.6343647 (94)\ttotal: 42.8s\tremaining: 6m 48s\n",
      "95:\ttest: 0.6345898\tbest: 0.6345898 (95)\ttotal: 43.2s\tremaining: 6m 47s\n",
      "96:\ttest: 0.6346634\tbest: 0.6346634 (96)\ttotal: 43.8s\tremaining: 6m 47s\n",
      "97:\ttest: 0.6348017\tbest: 0.6348017 (97)\ttotal: 44.3s\tremaining: 6m 47s\n",
      "98:\ttest: 0.6348441\tbest: 0.6348441 (98)\ttotal: 44.8s\tremaining: 6m 48s\n",
      "99:\ttest: 0.6349322\tbest: 0.6349322 (99)\ttotal: 45.3s\tremaining: 6m 47s\n",
      "100:\ttest: 0.6349868\tbest: 0.6349868 (100)\ttotal: 45.8s\tremaining: 6m 47s\n",
      "101:\ttest: 0.6351365\tbest: 0.6351365 (101)\ttotal: 46.2s\tremaining: 6m 46s\n",
      "102:\ttest: 0.6351512\tbest: 0.6351512 (102)\ttotal: 46.7s\tremaining: 6m 46s\n",
      "103:\ttest: 0.6351889\tbest: 0.6351889 (103)\ttotal: 47.2s\tremaining: 6m 46s\n",
      "104:\ttest: 0.6352285\tbest: 0.6352285 (104)\ttotal: 47.6s\tremaining: 6m 46s\n",
      "105:\ttest: 0.6352418\tbest: 0.6352418 (105)\ttotal: 48.2s\tremaining: 6m 46s\n",
      "106:\ttest: 0.6354333\tbest: 0.6354333 (106)\ttotal: 48.7s\tremaining: 6m 46s\n",
      "107:\ttest: 0.6355571\tbest: 0.6355571 (107)\ttotal: 49.2s\tremaining: 6m 46s\n",
      "108:\ttest: 0.6356316\tbest: 0.6356316 (108)\ttotal: 49.6s\tremaining: 6m 45s\n",
      "109:\ttest: 0.6356618\tbest: 0.6356618 (109)\ttotal: 50.1s\tremaining: 6m 45s\n",
      "110:\ttest: 0.6357206\tbest: 0.6357206 (110)\ttotal: 50.6s\tremaining: 6m 45s\n",
      "111:\ttest: 0.6358378\tbest: 0.6358378 (111)\ttotal: 51.1s\tremaining: 6m 45s\n",
      "112:\ttest: 0.6358307\tbest: 0.6358378 (111)\ttotal: 51.6s\tremaining: 6m 44s\n",
      "113:\ttest: 0.6358922\tbest: 0.6358922 (113)\ttotal: 52.1s\tremaining: 6m 44s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "114:\ttest: 0.6359293\tbest: 0.6359293 (114)\ttotal: 52.5s\tremaining: 6m 43s\n",
      "115:\ttest: 0.6360562\tbest: 0.6360562 (115)\ttotal: 52.9s\tremaining: 6m 43s\n",
      "116:\ttest: 0.6360841\tbest: 0.6360841 (116)\ttotal: 53.3s\tremaining: 6m 42s\n",
      "117:\ttest: 0.6360291\tbest: 0.6360841 (116)\ttotal: 53.8s\tremaining: 6m 42s\n",
      "118:\ttest: 0.6360588\tbest: 0.6360841 (116)\ttotal: 54.3s\tremaining: 6m 41s\n",
      "119:\ttest: 0.6361529\tbest: 0.6361529 (119)\ttotal: 54.8s\tremaining: 6m 41s\n",
      "120:\ttest: 0.6362225\tbest: 0.6362225 (120)\ttotal: 55.1s\tremaining: 6m 40s\n",
      "121:\ttest: 0.6361905\tbest: 0.6362225 (120)\ttotal: 55.7s\tremaining: 6m 40s\n",
      "122:\ttest: 0.6362111\tbest: 0.6362225 (120)\ttotal: 56.2s\tremaining: 6m 40s\n",
      "123:\ttest: 0.6361962\tbest: 0.6362225 (120)\ttotal: 56.8s\tremaining: 6m 41s\n",
      "124:\ttest: 0.6361911\tbest: 0.6362225 (120)\ttotal: 57.4s\tremaining: 6m 41s\n",
      "125:\ttest: 0.6362070\tbest: 0.6362225 (120)\ttotal: 58s\tremaining: 6m 42s\n",
      "126:\ttest: 0.6362060\tbest: 0.6362225 (120)\ttotal: 58.5s\tremaining: 6m 41s\n",
      "127:\ttest: 0.6361753\tbest: 0.6362225 (120)\ttotal: 58.9s\tremaining: 6m 41s\n",
      "128:\ttest: 0.6361797\tbest: 0.6362225 (120)\ttotal: 59.5s\tremaining: 6m 41s\n",
      "129:\ttest: 0.6361319\tbest: 0.6362225 (120)\ttotal: 60s\tremaining: 6m 41s\n",
      "130:\ttest: 0.6361075\tbest: 0.6362225 (120)\ttotal: 1m\tremaining: 6m 41s\n",
      "131:\ttest: 0.6360783\tbest: 0.6362225 (120)\ttotal: 1m 1s\tremaining: 6m 41s\n",
      "132:\ttest: 0.6360947\tbest: 0.6362225 (120)\ttotal: 1m 1s\tremaining: 6m 42s\n",
      "133:\ttest: 0.6360853\tbest: 0.6362225 (120)\ttotal: 1m 2s\tremaining: 6m 42s\n",
      "134:\ttest: 0.6360773\tbest: 0.6362225 (120)\ttotal: 1m 2s\tremaining: 6m 41s\n",
      "135:\ttest: 0.6360903\tbest: 0.6362225 (120)\ttotal: 1m 3s\tremaining: 6m 42s\n",
      "136:\ttest: 0.6360817\tbest: 0.6362225 (120)\ttotal: 1m 3s\tremaining: 6m 41s\n",
      "137:\ttest: 0.6360704\tbest: 0.6362225 (120)\ttotal: 1m 4s\tremaining: 6m 41s\n",
      "138:\ttest: 0.6360768\tbest: 0.6362225 (120)\ttotal: 1m 4s\tremaining: 6m 40s\n",
      "139:\ttest: 0.6360790\tbest: 0.6362225 (120)\ttotal: 1m 5s\tremaining: 6m 41s\n",
      "140:\ttest: 0.6361173\tbest: 0.6362225 (120)\ttotal: 1m 5s\tremaining: 6m 41s\n",
      "141:\ttest: 0.6361238\tbest: 0.6362225 (120)\ttotal: 1m 6s\tremaining: 6m 40s\n",
      "142:\ttest: 0.6361423\tbest: 0.6362225 (120)\ttotal: 1m 6s\tremaining: 6m 41s\n",
      "143:\ttest: 0.6361559\tbest: 0.6362225 (120)\ttotal: 1m 7s\tremaining: 6m 40s\n",
      "144:\ttest: 0.6362113\tbest: 0.6362225 (120)\ttotal: 1m 8s\tremaining: 6m 41s\n",
      "145:\ttest: 0.6362810\tbest: 0.6362810 (145)\ttotal: 1m 8s\tremaining: 6m 41s\n",
      "146:\ttest: 0.6362926\tbest: 0.6362926 (146)\ttotal: 1m 9s\tremaining: 6m 41s\n",
      "147:\ttest: 0.6362919\tbest: 0.6362926 (146)\ttotal: 1m 9s\tremaining: 6m 40s\n",
      "148:\ttest: 0.6362945\tbest: 0.6362945 (148)\ttotal: 1m 10s\tremaining: 6m 40s\n",
      "149:\ttest: 0.6363255\tbest: 0.6363255 (149)\ttotal: 1m 10s\tremaining: 6m 40s\n",
      "150:\ttest: 0.6363406\tbest: 0.6363406 (150)\ttotal: 1m 11s\tremaining: 6m 40s\n",
      "151:\ttest: 0.6363480\tbest: 0.6363480 (151)\ttotal: 1m 11s\tremaining: 6m 40s\n",
      "152:\ttest: 0.6363972\tbest: 0.6363972 (152)\ttotal: 1m 12s\tremaining: 6m 40s\n",
      "153:\ttest: 0.6363656\tbest: 0.6363972 (152)\ttotal: 1m 12s\tremaining: 6m 40s\n",
      "154:\ttest: 0.6363947\tbest: 0.6363972 (152)\ttotal: 1m 13s\tremaining: 6m 40s\n",
      "155:\ttest: 0.6363764\tbest: 0.6363972 (152)\ttotal: 1m 13s\tremaining: 6m 40s\n",
      "156:\ttest: 0.6365126\tbest: 0.6365126 (156)\ttotal: 1m 14s\tremaining: 6m 39s\n",
      "157:\ttest: 0.6365310\tbest: 0.6365310 (157)\ttotal: 1m 14s\tremaining: 6m 39s\n",
      "158:\ttest: 0.6365128\tbest: 0.6365310 (157)\ttotal: 1m 15s\tremaining: 6m 39s\n",
      "159:\ttest: 0.6364906\tbest: 0.6365310 (157)\ttotal: 1m 16s\tremaining: 6m 39s\n",
      "160:\ttest: 0.6364695\tbest: 0.6365310 (157)\ttotal: 1m 16s\tremaining: 6m 39s\n",
      "161:\ttest: 0.6364807\tbest: 0.6365310 (157)\ttotal: 1m 17s\tremaining: 6m 38s\n",
      "162:\ttest: 0.6364686\tbest: 0.6365310 (157)\ttotal: 1m 17s\tremaining: 6m 38s\n",
      "163:\ttest: 0.6364928\tbest: 0.6365310 (157)\ttotal: 1m 17s\tremaining: 6m 37s\n",
      "164:\ttest: 0.6365146\tbest: 0.6365310 (157)\ttotal: 1m 18s\tremaining: 6m 36s\n",
      "165:\ttest: 0.6365748\tbest: 0.6365748 (165)\ttotal: 1m 18s\tremaining: 6m 36s\n",
      "166:\ttest: 0.6365463\tbest: 0.6365748 (165)\ttotal: 1m 19s\tremaining: 6m 36s\n",
      "167:\ttest: 0.6365963\tbest: 0.6365963 (167)\ttotal: 1m 20s\tremaining: 6m 36s\n",
      "168:\ttest: 0.6366355\tbest: 0.6366355 (168)\ttotal: 1m 20s\tremaining: 6m 35s\n",
      "169:\ttest: 0.6366434\tbest: 0.6366434 (169)\ttotal: 1m 20s\tremaining: 6m 34s\n",
      "170:\ttest: 0.6366747\tbest: 0.6366747 (170)\ttotal: 1m 21s\tremaining: 6m 34s\n",
      "171:\ttest: 0.6367749\tbest: 0.6367749 (171)\ttotal: 1m 21s\tremaining: 6m 33s\n",
      "172:\ttest: 0.6367828\tbest: 0.6367828 (172)\ttotal: 1m 22s\tremaining: 6m 33s\n",
      "173:\ttest: 0.6367859\tbest: 0.6367859 (173)\ttotal: 1m 22s\tremaining: 6m 33s\n",
      "174:\ttest: 0.6367641\tbest: 0.6367859 (173)\ttotal: 1m 23s\tremaining: 6m 32s\n",
      "175:\ttest: 0.6367946\tbest: 0.6367946 (175)\ttotal: 1m 23s\tremaining: 6m 32s\n",
      "176:\ttest: 0.6367506\tbest: 0.6367946 (175)\ttotal: 1m 24s\tremaining: 6m 32s\n",
      "177:\ttest: 0.6367772\tbest: 0.6367946 (175)\ttotal: 1m 25s\tremaining: 6m 32s\n",
      "178:\ttest: 0.6367776\tbest: 0.6367946 (175)\ttotal: 1m 25s\tremaining: 6m 32s\n",
      "179:\ttest: 0.6368389\tbest: 0.6368389 (179)\ttotal: 1m 26s\tremaining: 6m 32s\n",
      "180:\ttest: 0.6368547\tbest: 0.6368547 (180)\ttotal: 1m 26s\tremaining: 6m 31s\n",
      "181:\ttest: 0.6368559\tbest: 0.6368559 (181)\ttotal: 1m 27s\tremaining: 6m 31s\n",
      "182:\ttest: 0.6368207\tbest: 0.6368559 (181)\ttotal: 1m 27s\tremaining: 6m 31s\n",
      "183:\ttest: 0.6368139\tbest: 0.6368559 (181)\ttotal: 1m 28s\tremaining: 6m 31s\n",
      "184:\ttest: 0.6367864\tbest: 0.6368559 (181)\ttotal: 1m 28s\tremaining: 6m 31s\n",
      "185:\ttest: 0.6367908\tbest: 0.6368559 (181)\ttotal: 1m 29s\tremaining: 6m 30s\n",
      "186:\ttest: 0.6367783\tbest: 0.6368559 (181)\ttotal: 1m 29s\tremaining: 6m 30s\n",
      "187:\ttest: 0.6368015\tbest: 0.6368559 (181)\ttotal: 1m 30s\tremaining: 6m 30s\n",
      "188:\ttest: 0.6368048\tbest: 0.6368559 (181)\ttotal: 1m 30s\tremaining: 6m 30s\n",
      "189:\ttest: 0.6368172\tbest: 0.6368559 (181)\ttotal: 1m 31s\tremaining: 6m 29s\n",
      "190:\ttest: 0.6368072\tbest: 0.6368559 (181)\ttotal: 1m 32s\tremaining: 6m 29s\n",
      "191:\ttest: 0.6368188\tbest: 0.6368559 (181)\ttotal: 1m 32s\tremaining: 6m 29s\n",
      "192:\ttest: 0.6368106\tbest: 0.6368559 (181)\ttotal: 1m 33s\tremaining: 6m 29s\n",
      "193:\ttest: 0.6367960\tbest: 0.6368559 (181)\ttotal: 1m 33s\tremaining: 6m 29s\n",
      "194:\ttest: 0.6367760\tbest: 0.6368559 (181)\ttotal: 1m 34s\tremaining: 6m 29s\n",
      "195:\ttest: 0.6367465\tbest: 0.6368559 (181)\ttotal: 1m 34s\tremaining: 6m 28s\n",
      "196:\ttest: 0.6367970\tbest: 0.6368559 (181)\ttotal: 1m 35s\tremaining: 6m 28s\n",
      "197:\ttest: 0.6368061\tbest: 0.6368559 (181)\ttotal: 1m 35s\tremaining: 6m 28s\n",
      "198:\ttest: 0.6368254\tbest: 0.6368559 (181)\ttotal: 1m 36s\tremaining: 6m 28s\n",
      "199:\ttest: 0.6368464\tbest: 0.6368559 (181)\ttotal: 1m 37s\tremaining: 6m 28s\n",
      "200:\ttest: 0.6368685\tbest: 0.6368685 (200)\ttotal: 1m 37s\tremaining: 6m 27s\n",
      "201:\ttest: 0.6368703\tbest: 0.6368703 (201)\ttotal: 1m 38s\tremaining: 6m 27s\n",
      "202:\ttest: 0.6369382\tbest: 0.6369382 (202)\ttotal: 1m 38s\tremaining: 6m 27s\n",
      "203:\ttest: 0.6369406\tbest: 0.6369406 (203)\ttotal: 1m 39s\tremaining: 6m 26s\n",
      "204:\ttest: 0.6369630\tbest: 0.6369630 (204)\ttotal: 1m 39s\tremaining: 6m 26s\n",
      "205:\ttest: 0.6369798\tbest: 0.6369798 (205)\ttotal: 1m 40s\tremaining: 6m 25s\n",
      "206:\ttest: 0.6370002\tbest: 0.6370002 (206)\ttotal: 1m 40s\tremaining: 6m 25s\n",
      "207:\ttest: 0.6369844\tbest: 0.6370002 (206)\ttotal: 1m 41s\tremaining: 6m 24s\n",
      "208:\ttest: 0.6370330\tbest: 0.6370330 (208)\ttotal: 1m 41s\tremaining: 6m 24s\n",
      "209:\ttest: 0.6370227\tbest: 0.6370330 (208)\ttotal: 1m 42s\tremaining: 6m 24s\n",
      "210:\ttest: 0.6370263\tbest: 0.6370330 (208)\ttotal: 1m 42s\tremaining: 6m 23s\n",
      "211:\ttest: 0.6370472\tbest: 0.6370472 (211)\ttotal: 1m 43s\tremaining: 6m 23s\n",
      "212:\ttest: 0.6370823\tbest: 0.6370823 (212)\ttotal: 1m 43s\tremaining: 6m 23s\n",
      "213:\ttest: 0.6371358\tbest: 0.6371358 (213)\ttotal: 1m 44s\tremaining: 6m 22s\n",
      "214:\ttest: 0.6371555\tbest: 0.6371555 (214)\ttotal: 1m 44s\tremaining: 6m 22s\n",
      "215:\ttest: 0.6371694\tbest: 0.6371694 (215)\ttotal: 1m 45s\tremaining: 6m 22s\n",
      "216:\ttest: 0.6371748\tbest: 0.6371748 (216)\ttotal: 1m 45s\tremaining: 6m 21s\n",
      "217:\ttest: 0.6371837\tbest: 0.6371837 (217)\ttotal: 1m 46s\tremaining: 6m 21s\n",
      "218:\ttest: 0.6371785\tbest: 0.6371837 (217)\ttotal: 1m 46s\tremaining: 6m 21s\n",
      "219:\ttest: 0.6371564\tbest: 0.6371837 (217)\ttotal: 1m 47s\tremaining: 6m 20s\n",
      "220:\ttest: 0.6371872\tbest: 0.6371872 (220)\ttotal: 1m 47s\tremaining: 6m 20s\n",
      "221:\ttest: 0.6371538\tbest: 0.6371872 (220)\ttotal: 1m 48s\tremaining: 6m 20s\n",
      "222:\ttest: 0.6371093\tbest: 0.6371872 (220)\ttotal: 1m 49s\tremaining: 6m 20s\n",
      "223:\ttest: 0.6370940\tbest: 0.6371872 (220)\ttotal: 1m 49s\tremaining: 6m 20s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "224:\ttest: 0.6370319\tbest: 0.6371872 (220)\ttotal: 1m 50s\tremaining: 6m 19s\n",
      "225:\ttest: 0.6370104\tbest: 0.6371872 (220)\ttotal: 1m 50s\tremaining: 6m 19s\n",
      "226:\ttest: 0.6370166\tbest: 0.6371872 (220)\ttotal: 1m 51s\tremaining: 6m 18s\n",
      "227:\ttest: 0.6370095\tbest: 0.6371872 (220)\ttotal: 1m 51s\tremaining: 6m 18s\n",
      "228:\ttest: 0.6369749\tbest: 0.6371872 (220)\ttotal: 1m 52s\tremaining: 6m 18s\n",
      "229:\ttest: 0.6370254\tbest: 0.6371872 (220)\ttotal: 1m 52s\tremaining: 6m 17s\n",
      "230:\ttest: 0.6370318\tbest: 0.6371872 (220)\ttotal: 1m 53s\tremaining: 6m 17s\n",
      "231:\ttest: 0.6369837\tbest: 0.6371872 (220)\ttotal: 1m 53s\tremaining: 6m 16s\n",
      "232:\ttest: 0.6370065\tbest: 0.6371872 (220)\ttotal: 1m 54s\tremaining: 6m 16s\n",
      "233:\ttest: 0.6370234\tbest: 0.6371872 (220)\ttotal: 1m 55s\tremaining: 6m 16s\n",
      "234:\ttest: 0.6370030\tbest: 0.6371872 (220)\ttotal: 1m 55s\tremaining: 6m 15s\n",
      "235:\ttest: 0.6370237\tbest: 0.6371872 (220)\ttotal: 1m 55s\tremaining: 6m 15s\n",
      "236:\ttest: 0.6370700\tbest: 0.6371872 (220)\ttotal: 1m 56s\tremaining: 6m 15s\n",
      "237:\ttest: 0.6370593\tbest: 0.6371872 (220)\ttotal: 1m 57s\tremaining: 6m 14s\n",
      "238:\ttest: 0.6370868\tbest: 0.6371872 (220)\ttotal: 1m 57s\tremaining: 6m 14s\n",
      "239:\ttest: 0.6370998\tbest: 0.6371872 (220)\ttotal: 1m 58s\tremaining: 6m 14s\n",
      "240:\ttest: 0.6370334\tbest: 0.6371872 (220)\ttotal: 1m 58s\tremaining: 6m 14s\n",
      "241:\ttest: 0.6370390\tbest: 0.6371872 (220)\ttotal: 1m 59s\tremaining: 6m 14s\n",
      "242:\ttest: 0.6370094\tbest: 0.6371872 (220)\ttotal: 2m\tremaining: 6m 13s\n",
      "243:\ttest: 0.6369808\tbest: 0.6371872 (220)\ttotal: 2m\tremaining: 6m 13s\n",
      "244:\ttest: 0.6370554\tbest: 0.6371872 (220)\ttotal: 2m 1s\tremaining: 6m 13s\n",
      "245:\ttest: 0.6370024\tbest: 0.6371872 (220)\ttotal: 2m 1s\tremaining: 6m 12s\n",
      "246:\ttest: 0.6369963\tbest: 0.6371872 (220)\ttotal: 2m 2s\tremaining: 6m 12s\n",
      "247:\ttest: 0.6370002\tbest: 0.6371872 (220)\ttotal: 2m 2s\tremaining: 6m 12s\n",
      "248:\ttest: 0.6369712\tbest: 0.6371872 (220)\ttotal: 2m 3s\tremaining: 6m 11s\n",
      "249:\ttest: 0.6369633\tbest: 0.6371872 (220)\ttotal: 2m 3s\tremaining: 6m 11s\n",
      "250:\ttest: 0.6369426\tbest: 0.6371872 (220)\ttotal: 2m 4s\tremaining: 6m 11s\n",
      "251:\ttest: 0.6369218\tbest: 0.6371872 (220)\ttotal: 2m 4s\tremaining: 6m 10s\n",
      "252:\ttest: 0.6369192\tbest: 0.6371872 (220)\ttotal: 2m 5s\tremaining: 6m 10s\n",
      "253:\ttest: 0.6369258\tbest: 0.6371872 (220)\ttotal: 2m 6s\tremaining: 6m 10s\n",
      "254:\ttest: 0.6368909\tbest: 0.6371872 (220)\ttotal: 2m 6s\tremaining: 6m 10s\n",
      "255:\ttest: 0.6369092\tbest: 0.6371872 (220)\ttotal: 2m 7s\tremaining: 6m 9s\n",
      "256:\ttest: 0.6369400\tbest: 0.6371872 (220)\ttotal: 2m 7s\tremaining: 6m 9s\n",
      "257:\ttest: 0.6369537\tbest: 0.6371872 (220)\ttotal: 2m 8s\tremaining: 6m 9s\n",
      "258:\ttest: 0.6369350\tbest: 0.6371872 (220)\ttotal: 2m 9s\tremaining: 6m 9s\n",
      "259:\ttest: 0.6369912\tbest: 0.6371872 (220)\ttotal: 2m 9s\tremaining: 6m 8s\n",
      "260:\ttest: 0.6369881\tbest: 0.6371872 (220)\ttotal: 2m 10s\tremaining: 6m 8s\n",
      "261:\ttest: 0.6369691\tbest: 0.6371872 (220)\ttotal: 2m 10s\tremaining: 6m 7s\n",
      "262:\ttest: 0.6369784\tbest: 0.6371872 (220)\ttotal: 2m 11s\tremaining: 6m 7s\n",
      "263:\ttest: 0.6369867\tbest: 0.6371872 (220)\ttotal: 2m 11s\tremaining: 6m 6s\n",
      "264:\ttest: 0.6369692\tbest: 0.6371872 (220)\ttotal: 2m 12s\tremaining: 6m 6s\n",
      "265:\ttest: 0.6369523\tbest: 0.6371872 (220)\ttotal: 2m 12s\tremaining: 6m 6s\n",
      "266:\ttest: 0.6369809\tbest: 0.6371872 (220)\ttotal: 2m 13s\tremaining: 6m 5s\n",
      "267:\ttest: 0.6369791\tbest: 0.6371872 (220)\ttotal: 2m 13s\tremaining: 6m 5s\n",
      "268:\ttest: 0.6369653\tbest: 0.6371872 (220)\ttotal: 2m 14s\tremaining: 6m 5s\n",
      "269:\ttest: 0.6369245\tbest: 0.6371872 (220)\ttotal: 2m 14s\tremaining: 6m 4s\n",
      "270:\ttest: 0.6368896\tbest: 0.6371872 (220)\ttotal: 2m 15s\tremaining: 6m 4s\n",
      "271:\ttest: 0.6368945\tbest: 0.6371872 (220)\ttotal: 2m 15s\tremaining: 6m 3s\n",
      "272:\ttest: 0.6368935\tbest: 0.6371872 (220)\ttotal: 2m 16s\tremaining: 6m 3s\n",
      "273:\ttest: 0.6369156\tbest: 0.6371872 (220)\ttotal: 2m 17s\tremaining: 6m 3s\n",
      "274:\ttest: 0.6369388\tbest: 0.6371872 (220)\ttotal: 2m 17s\tremaining: 6m 2s\n",
      "275:\ttest: 0.6369715\tbest: 0.6371872 (220)\ttotal: 2m 18s\tremaining: 6m 2s\n",
      "276:\ttest: 0.6369754\tbest: 0.6371872 (220)\ttotal: 2m 18s\tremaining: 6m 2s\n",
      "277:\ttest: 0.6369414\tbest: 0.6371872 (220)\ttotal: 2m 19s\tremaining: 6m 1s\n",
      "278:\ttest: 0.6369674\tbest: 0.6371872 (220)\ttotal: 2m 19s\tremaining: 6m 1s\n",
      "279:\ttest: 0.6369663\tbest: 0.6371872 (220)\ttotal: 2m 20s\tremaining: 6m\n",
      "280:\ttest: 0.6369401\tbest: 0.6371872 (220)\ttotal: 2m 20s\tremaining: 6m\n",
      "281:\ttest: 0.6369399\tbest: 0.6371872 (220)\ttotal: 2m 21s\tremaining: 6m\n",
      "282:\ttest: 0.6369685\tbest: 0.6371872 (220)\ttotal: 2m 21s\tremaining: 5m 59s\n",
      "283:\ttest: 0.6369338\tbest: 0.6371872 (220)\ttotal: 2m 22s\tremaining: 5m 59s\n",
      "284:\ttest: 0.6369302\tbest: 0.6371872 (220)\ttotal: 2m 23s\tremaining: 5m 58s\n",
      "285:\ttest: 0.6369330\tbest: 0.6371872 (220)\ttotal: 2m 23s\tremaining: 5m 58s\n",
      "286:\ttest: 0.6369234\tbest: 0.6371872 (220)\ttotal: 2m 24s\tremaining: 5m 58s\n",
      "287:\ttest: 0.6369113\tbest: 0.6371872 (220)\ttotal: 2m 24s\tremaining: 5m 57s\n",
      "288:\ttest: 0.6369308\tbest: 0.6371872 (220)\ttotal: 2m 25s\tremaining: 5m 57s\n",
      "289:\ttest: 0.6369703\tbest: 0.6371872 (220)\ttotal: 2m 25s\tremaining: 5m 56s\n",
      "290:\ttest: 0.6369896\tbest: 0.6371872 (220)\ttotal: 2m 26s\tremaining: 5m 56s\n",
      "291:\ttest: 0.6370151\tbest: 0.6371872 (220)\ttotal: 2m 26s\tremaining: 5m 55s\n",
      "292:\ttest: 0.6369860\tbest: 0.6371872 (220)\ttotal: 2m 27s\tremaining: 5m 55s\n",
      "293:\ttest: 0.6369857\tbest: 0.6371872 (220)\ttotal: 2m 27s\tremaining: 5m 55s\n",
      "294:\ttest: 0.6369044\tbest: 0.6371872 (220)\ttotal: 2m 28s\tremaining: 5m 54s\n",
      "295:\ttest: 0.6368564\tbest: 0.6371872 (220)\ttotal: 2m 29s\tremaining: 5m 54s\n",
      "296:\ttest: 0.6368672\tbest: 0.6371872 (220)\ttotal: 2m 29s\tremaining: 5m 54s\n",
      "297:\ttest: 0.6368616\tbest: 0.6371872 (220)\ttotal: 2m 30s\tremaining: 5m 53s\n",
      "298:\ttest: 0.6369033\tbest: 0.6371872 (220)\ttotal: 2m 30s\tremaining: 5m 53s\n",
      "299:\ttest: 0.6369403\tbest: 0.6371872 (220)\ttotal: 2m 31s\tremaining: 5m 52s\n",
      "300:\ttest: 0.6369361\tbest: 0.6371872 (220)\ttotal: 2m 31s\tremaining: 5m 52s\n",
      "301:\ttest: 0.6369290\tbest: 0.6371872 (220)\ttotal: 2m 32s\tremaining: 5m 51s\n",
      "302:\ttest: 0.6369415\tbest: 0.6371872 (220)\ttotal: 2m 32s\tremaining: 5m 51s\n",
      "303:\ttest: 0.6369477\tbest: 0.6371872 (220)\ttotal: 2m 33s\tremaining: 5m 51s\n",
      "304:\ttest: 0.6369548\tbest: 0.6371872 (220)\ttotal: 2m 34s\tremaining: 5m 50s\n",
      "305:\ttest: 0.6369580\tbest: 0.6371872 (220)\ttotal: 2m 34s\tremaining: 5m 50s\n",
      "306:\ttest: 0.6369571\tbest: 0.6371872 (220)\ttotal: 2m 35s\tremaining: 5m 50s\n",
      "307:\ttest: 0.6369077\tbest: 0.6371872 (220)\ttotal: 2m 35s\tremaining: 5m 49s\n",
      "308:\ttest: 0.6368996\tbest: 0.6371872 (220)\ttotal: 2m 36s\tremaining: 5m 49s\n",
      "309:\ttest: 0.6369250\tbest: 0.6371872 (220)\ttotal: 2m 36s\tremaining: 5m 49s\n",
      "310:\ttest: 0.6369206\tbest: 0.6371872 (220)\ttotal: 2m 37s\tremaining: 5m 48s\n",
      "311:\ttest: 0.6369157\tbest: 0.6371872 (220)\ttotal: 2m 37s\tremaining: 5m 48s\n",
      "312:\ttest: 0.6369475\tbest: 0.6371872 (220)\ttotal: 2m 38s\tremaining: 5m 47s\n",
      "313:\ttest: 0.6368667\tbest: 0.6371872 (220)\ttotal: 2m 38s\tremaining: 5m 47s\n",
      "314:\ttest: 0.6368265\tbest: 0.6371872 (220)\ttotal: 2m 39s\tremaining: 5m 46s\n",
      "315:\ttest: 0.6368223\tbest: 0.6371872 (220)\ttotal: 2m 40s\tremaining: 5m 46s\n",
      "316:\ttest: 0.6367579\tbest: 0.6371872 (220)\ttotal: 2m 40s\tremaining: 5m 46s\n",
      "317:\ttest: 0.6367845\tbest: 0.6371872 (220)\ttotal: 2m 41s\tremaining: 5m 45s\n",
      "318:\ttest: 0.6367584\tbest: 0.6371872 (220)\ttotal: 2m 41s\tremaining: 5m 45s\n",
      "319:\ttest: 0.6368345\tbest: 0.6371872 (220)\ttotal: 2m 42s\tremaining: 5m 44s\n",
      "320:\ttest: 0.6368892\tbest: 0.6371872 (220)\ttotal: 2m 42s\tremaining: 5m 44s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.637187165\n",
      "bestIteration = 220\n",
      "\n",
      "Shrink model to first 221 iterations.\n",
      "err:  0.6371871650219586\n",
      "0:\ttest: 0.5854215\tbest: 0.5854215 (0)\ttotal: 482ms\tremaining: 8m 1s\n",
      "1:\ttest: 0.5867052\tbest: 0.5867052 (1)\ttotal: 936ms\tremaining: 7m 47s\n",
      "2:\ttest: 0.5875193\tbest: 0.5875193 (2)\ttotal: 1.23s\tremaining: 6m 49s\n",
      "3:\ttest: 0.5880230\tbest: 0.5880230 (3)\ttotal: 1.74s\tremaining: 7m 13s\n",
      "4:\ttest: 0.5882019\tbest: 0.5882019 (4)\ttotal: 2.26s\tremaining: 7m 29s\n",
      "5:\ttest: 0.5884244\tbest: 0.5884244 (5)\ttotal: 2.69s\tremaining: 7m 25s\n",
      "6:\ttest: 0.5897016\tbest: 0.5897016 (6)\ttotal: 3.21s\tremaining: 7m 35s\n",
      "7:\ttest: 0.5908649\tbest: 0.5908649 (7)\ttotal: 3.72s\tremaining: 7m 41s\n",
      "8:\ttest: 0.5917256\tbest: 0.5917256 (8)\ttotal: 4.31s\tremaining: 7m 54s\n",
      "9:\ttest: 0.5921229\tbest: 0.5921229 (9)\ttotal: 4.9s\tremaining: 8m 5s\n",
      "10:\ttest: 0.5921283\tbest: 0.5921283 (10)\ttotal: 5.44s\tremaining: 8m 9s\n",
      "11:\ttest: 0.5918702\tbest: 0.5921283 (10)\ttotal: 5.97s\tremaining: 8m 11s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12:\ttest: 0.5919156\tbest: 0.5921283 (10)\ttotal: 6.51s\tremaining: 8m 14s\n",
      "13:\ttest: 0.5915985\tbest: 0.5921283 (10)\ttotal: 7.07s\tremaining: 8m 17s\n",
      "14:\ttest: 0.5917690\tbest: 0.5921283 (10)\ttotal: 7.59s\tremaining: 8m 18s\n",
      "15:\ttest: 0.5915104\tbest: 0.5921283 (10)\ttotal: 8.1s\tremaining: 8m 18s\n",
      "16:\ttest: 0.5913704\tbest: 0.5921283 (10)\ttotal: 8.62s\tremaining: 8m 18s\n",
      "17:\ttest: 0.5914344\tbest: 0.5921283 (10)\ttotal: 9.09s\tremaining: 8m 15s\n",
      "18:\ttest: 0.5922117\tbest: 0.5922117 (18)\ttotal: 9.68s\tremaining: 8m 19s\n",
      "19:\ttest: 0.5921200\tbest: 0.5922117 (18)\ttotal: 10.2s\tremaining: 8m 20s\n",
      "20:\ttest: 0.5920528\tbest: 0.5922117 (18)\ttotal: 10.8s\tremaining: 8m 23s\n",
      "21:\ttest: 0.5919977\tbest: 0.5922117 (18)\ttotal: 11.3s\tremaining: 8m 22s\n",
      "22:\ttest: 0.5921669\tbest: 0.5922117 (18)\ttotal: 11.8s\tremaining: 8m 23s\n",
      "23:\ttest: 0.5922783\tbest: 0.5922783 (23)\ttotal: 12.4s\tremaining: 8m 23s\n",
      "24:\ttest: 0.5925798\tbest: 0.5925798 (24)\ttotal: 13s\tremaining: 8m 26s\n",
      "25:\ttest: 0.5924744\tbest: 0.5925798 (24)\ttotal: 13.6s\tremaining: 8m 28s\n",
      "26:\ttest: 0.5925709\tbest: 0.5925798 (24)\ttotal: 14.1s\tremaining: 8m 26s\n",
      "27:\ttest: 0.5927240\tbest: 0.5927240 (27)\ttotal: 14.6s\tremaining: 8m 26s\n",
      "28:\ttest: 0.5929720\tbest: 0.5929720 (28)\ttotal: 15.2s\tremaining: 8m 27s\n",
      "29:\ttest: 0.5928913\tbest: 0.5929720 (28)\ttotal: 15.7s\tremaining: 8m 28s\n",
      "30:\ttest: 0.5931519\tbest: 0.5931519 (30)\ttotal: 16.3s\tremaining: 8m 28s\n",
      "31:\ttest: 0.5931908\tbest: 0.5931908 (31)\ttotal: 16.9s\tremaining: 8m 30s\n",
      "32:\ttest: 0.5932291\tbest: 0.5932291 (32)\ttotal: 17.5s\tremaining: 8m 33s\n",
      "33:\ttest: 0.5932689\tbest: 0.5932689 (33)\ttotal: 18.1s\tremaining: 8m 33s\n",
      "34:\ttest: 0.5938098\tbest: 0.5938098 (34)\ttotal: 18.7s\tremaining: 8m 35s\n",
      "35:\ttest: 0.5944117\tbest: 0.5944117 (35)\ttotal: 19.3s\tremaining: 8m 37s\n",
      "36:\ttest: 0.5945795\tbest: 0.5945795 (36)\ttotal: 19.8s\tremaining: 8m 36s\n",
      "37:\ttest: 0.5946885\tbest: 0.5946885 (37)\ttotal: 20.4s\tremaining: 8m 37s\n",
      "38:\ttest: 0.5948741\tbest: 0.5948741 (38)\ttotal: 21s\tremaining: 8m 36s\n",
      "39:\ttest: 0.5948506\tbest: 0.5948741 (38)\ttotal: 21.5s\tremaining: 8m 37s\n",
      "40:\ttest: 0.5950610\tbest: 0.5950610 (40)\ttotal: 22.1s\tremaining: 8m 36s\n",
      "41:\ttest: 0.5949969\tbest: 0.5950610 (40)\ttotal: 22.7s\tremaining: 8m 38s\n",
      "42:\ttest: 0.5952154\tbest: 0.5952154 (42)\ttotal: 23.3s\tremaining: 8m 39s\n",
      "43:\ttest: 0.5953620\tbest: 0.5953620 (43)\ttotal: 23.9s\tremaining: 8m 39s\n",
      "44:\ttest: 0.5953079\tbest: 0.5953620 (43)\ttotal: 24.4s\tremaining: 8m 37s\n",
      "45:\ttest: 0.5953612\tbest: 0.5953620 (43)\ttotal: 25s\tremaining: 8m 38s\n",
      "46:\ttest: 0.5954339\tbest: 0.5954339 (46)\ttotal: 25.5s\tremaining: 8m 37s\n",
      "47:\ttest: 0.5954500\tbest: 0.5954500 (47)\ttotal: 26.1s\tremaining: 8m 37s\n",
      "48:\ttest: 0.5954876\tbest: 0.5954876 (48)\ttotal: 26.7s\tremaining: 8m 37s\n",
      "49:\ttest: 0.6004265\tbest: 0.6004265 (49)\ttotal: 27.3s\tremaining: 8m 38s\n",
      "50:\ttest: 0.6057170\tbest: 0.6057170 (50)\ttotal: 27.8s\tremaining: 8m 37s\n",
      "51:\ttest: 0.6102441\tbest: 0.6102441 (51)\ttotal: 28.2s\tremaining: 8m 34s\n",
      "52:\ttest: 0.6135748\tbest: 0.6135748 (52)\ttotal: 28.7s\tremaining: 8m 33s\n",
      "53:\ttest: 0.6167279\tbest: 0.6167279 (53)\ttotal: 29.1s\tremaining: 8m 30s\n",
      "54:\ttest: 0.6192057\tbest: 0.6192057 (54)\ttotal: 29.6s\tremaining: 8m 27s\n",
      "55:\ttest: 0.6214196\tbest: 0.6214196 (55)\ttotal: 29.9s\tremaining: 8m 24s\n",
      "56:\ttest: 0.6226711\tbest: 0.6226711 (56)\ttotal: 30.4s\tremaining: 8m 22s\n",
      "57:\ttest: 0.6236022\tbest: 0.6236022 (57)\ttotal: 30.8s\tremaining: 8m 20s\n",
      "58:\ttest: 0.6243655\tbest: 0.6243655 (58)\ttotal: 31.2s\tremaining: 8m 17s\n",
      "59:\ttest: 0.6252549\tbest: 0.6252549 (59)\ttotal: 31.7s\tremaining: 8m 16s\n",
      "60:\ttest: 0.6263957\tbest: 0.6263957 (60)\ttotal: 32.2s\tremaining: 8m 15s\n",
      "61:\ttest: 0.6275180\tbest: 0.6275180 (61)\ttotal: 32.5s\tremaining: 8m 12s\n",
      "62:\ttest: 0.6280982\tbest: 0.6280982 (62)\ttotal: 32.9s\tremaining: 8m 9s\n",
      "63:\ttest: 0.6285121\tbest: 0.6285121 (63)\ttotal: 33.4s\tremaining: 8m 8s\n",
      "64:\ttest: 0.6288715\tbest: 0.6288715 (64)\ttotal: 34s\tremaining: 8m 8s\n",
      "65:\ttest: 0.6291137\tbest: 0.6291137 (65)\ttotal: 34.4s\tremaining: 8m 6s\n",
      "66:\ttest: 0.6295648\tbest: 0.6295648 (66)\ttotal: 34.7s\tremaining: 8m 3s\n",
      "67:\ttest: 0.6300415\tbest: 0.6300415 (67)\ttotal: 35.1s\tremaining: 8m 1s\n",
      "68:\ttest: 0.6302250\tbest: 0.6302250 (68)\ttotal: 35.6s\tremaining: 8m\n",
      "69:\ttest: 0.6303415\tbest: 0.6303415 (69)\ttotal: 36s\tremaining: 7m 57s\n",
      "70:\ttest: 0.6304965\tbest: 0.6304965 (70)\ttotal: 36.4s\tremaining: 7m 56s\n",
      "71:\ttest: 0.6304873\tbest: 0.6304965 (70)\ttotal: 36.9s\tremaining: 7m 55s\n",
      "72:\ttest: 0.6305494\tbest: 0.6305494 (72)\ttotal: 37.4s\tremaining: 7m 55s\n",
      "73:\ttest: 0.6308046\tbest: 0.6308046 (73)\ttotal: 37.9s\tremaining: 7m 53s\n",
      "74:\ttest: 0.6308725\tbest: 0.6308725 (74)\ttotal: 38.3s\tremaining: 7m 52s\n",
      "75:\ttest: 0.6313405\tbest: 0.6313405 (75)\ttotal: 38.7s\tremaining: 7m 50s\n",
      "76:\ttest: 0.6314947\tbest: 0.6314947 (76)\ttotal: 39.1s\tremaining: 7m 48s\n",
      "77:\ttest: 0.6317071\tbest: 0.6317071 (77)\ttotal: 39.6s\tremaining: 7m 47s\n",
      "78:\ttest: 0.6317300\tbest: 0.6317300 (78)\ttotal: 40.1s\tremaining: 7m 47s\n",
      "79:\ttest: 0.6317784\tbest: 0.6317784 (79)\ttotal: 40.5s\tremaining: 7m 45s\n",
      "80:\ttest: 0.6318514\tbest: 0.6318514 (80)\ttotal: 40.9s\tremaining: 7m 44s\n",
      "81:\ttest: 0.6318238\tbest: 0.6318514 (80)\ttotal: 41.3s\tremaining: 7m 42s\n",
      "82:\ttest: 0.6320701\tbest: 0.6320701 (82)\ttotal: 41.8s\tremaining: 7m 42s\n",
      "83:\ttest: 0.6323704\tbest: 0.6323704 (83)\ttotal: 42.3s\tremaining: 7m 41s\n",
      "84:\ttest: 0.6327081\tbest: 0.6327081 (84)\ttotal: 42.7s\tremaining: 7m 39s\n",
      "85:\ttest: 0.6328457\tbest: 0.6328457 (85)\ttotal: 43.1s\tremaining: 7m 38s\n",
      "86:\ttest: 0.6330210\tbest: 0.6330210 (86)\ttotal: 43.5s\tremaining: 7m 36s\n",
      "87:\ttest: 0.6331568\tbest: 0.6331568 (87)\ttotal: 44s\tremaining: 7m 35s\n",
      "88:\ttest: 0.6331523\tbest: 0.6331568 (87)\ttotal: 44.5s\tremaining: 7m 35s\n",
      "89:\ttest: 0.6332559\tbest: 0.6332559 (89)\ttotal: 45s\tremaining: 7m 35s\n",
      "90:\ttest: 0.6332744\tbest: 0.6332744 (90)\ttotal: 45.5s\tremaining: 7m 34s\n",
      "91:\ttest: 0.6333004\tbest: 0.6333004 (91)\ttotal: 46.1s\tremaining: 7m 35s\n",
      "92:\ttest: 0.6334452\tbest: 0.6334452 (92)\ttotal: 46.5s\tremaining: 7m 33s\n",
      "93:\ttest: 0.6334821\tbest: 0.6334821 (93)\ttotal: 47.1s\tremaining: 7m 33s\n",
      "94:\ttest: 0.6336085\tbest: 0.6336085 (94)\ttotal: 47.4s\tremaining: 7m 31s\n",
      "95:\ttest: 0.6336502\tbest: 0.6336502 (95)\ttotal: 47.9s\tremaining: 7m 31s\n",
      "96:\ttest: 0.6336991\tbest: 0.6336991 (96)\ttotal: 48.4s\tremaining: 7m 30s\n",
      "97:\ttest: 0.6337428\tbest: 0.6337428 (97)\ttotal: 48.8s\tremaining: 7m 29s\n",
      "98:\ttest: 0.6338262\tbest: 0.6338262 (98)\ttotal: 49.3s\tremaining: 7m 29s\n",
      "99:\ttest: 0.6339204\tbest: 0.6339204 (99)\ttotal: 49.9s\tremaining: 7m 28s\n",
      "100:\ttest: 0.6339750\tbest: 0.6339750 (100)\ttotal: 50.3s\tremaining: 7m 27s\n",
      "101:\ttest: 0.6339594\tbest: 0.6339750 (100)\ttotal: 50.7s\tremaining: 7m 26s\n",
      "102:\ttest: 0.6341096\tbest: 0.6341096 (102)\ttotal: 51.2s\tremaining: 7m 26s\n",
      "103:\ttest: 0.6341843\tbest: 0.6341843 (103)\ttotal: 51.8s\tremaining: 7m 26s\n",
      "104:\ttest: 0.6342465\tbest: 0.6342465 (104)\ttotal: 52.3s\tremaining: 7m 25s\n",
      "105:\ttest: 0.6341846\tbest: 0.6342465 (104)\ttotal: 52.8s\tremaining: 7m 25s\n",
      "106:\ttest: 0.6342102\tbest: 0.6342465 (104)\ttotal: 53.3s\tremaining: 7m 24s\n",
      "107:\ttest: 0.6342347\tbest: 0.6342465 (104)\ttotal: 53.8s\tremaining: 7m 24s\n",
      "108:\ttest: 0.6342557\tbest: 0.6342557 (108)\ttotal: 54.2s\tremaining: 7m 23s\n",
      "109:\ttest: 0.6342998\tbest: 0.6342998 (109)\ttotal: 54.8s\tremaining: 7m 23s\n",
      "110:\ttest: 0.6342962\tbest: 0.6342998 (109)\ttotal: 55.3s\tremaining: 7m 23s\n",
      "111:\ttest: 0.6343151\tbest: 0.6343151 (111)\ttotal: 55.9s\tremaining: 7m 22s\n",
      "112:\ttest: 0.6343486\tbest: 0.6343486 (112)\ttotal: 56.4s\tremaining: 7m 23s\n",
      "113:\ttest: 0.6344009\tbest: 0.6344009 (113)\ttotal: 57s\tremaining: 7m 22s\n",
      "114:\ttest: 0.6343485\tbest: 0.6344009 (113)\ttotal: 57.3s\tremaining: 7m 21s\n",
      "115:\ttest: 0.6343996\tbest: 0.6344009 (113)\ttotal: 57.8s\tremaining: 7m 20s\n",
      "116:\ttest: 0.6344913\tbest: 0.6344913 (116)\ttotal: 58.3s\tremaining: 7m 19s\n",
      "117:\ttest: 0.6344855\tbest: 0.6344913 (116)\ttotal: 58.8s\tremaining: 7m 19s\n",
      "118:\ttest: 0.6344804\tbest: 0.6344913 (116)\ttotal: 59.2s\tremaining: 7m 18s\n",
      "119:\ttest: 0.6344957\tbest: 0.6344957 (119)\ttotal: 59.8s\tremaining: 7m 18s\n",
      "120:\ttest: 0.6345381\tbest: 0.6345381 (120)\ttotal: 1m\tremaining: 7m 18s\n",
      "121:\ttest: 0.6345450\tbest: 0.6345450 (121)\ttotal: 1m\tremaining: 7m 18s\n",
      "122:\ttest: 0.6345490\tbest: 0.6345490 (122)\ttotal: 1m 1s\tremaining: 7m 18s\n",
      "123:\ttest: 0.6345292\tbest: 0.6345490 (122)\ttotal: 1m 1s\tremaining: 7m 17s\n",
      "124:\ttest: 0.6345396\tbest: 0.6345490 (122)\ttotal: 1m 2s\tremaining: 7m 16s\n",
      "125:\ttest: 0.6345845\tbest: 0.6345845 (125)\ttotal: 1m 2s\tremaining: 7m 15s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "126:\ttest: 0.6346118\tbest: 0.6346118 (126)\ttotal: 1m 3s\tremaining: 7m 15s\n",
      "127:\ttest: 0.6346255\tbest: 0.6346255 (127)\ttotal: 1m 3s\tremaining: 7m 14s\n",
      "128:\ttest: 0.6346328\tbest: 0.6346328 (128)\ttotal: 1m 4s\tremaining: 7m 14s\n",
      "129:\ttest: 0.6345750\tbest: 0.6346328 (128)\ttotal: 1m 4s\tremaining: 7m 13s\n",
      "130:\ttest: 0.6345796\tbest: 0.6346328 (128)\ttotal: 1m 5s\tremaining: 7m 13s\n",
      "131:\ttest: 0.6346325\tbest: 0.6346328 (128)\ttotal: 1m 5s\tremaining: 7m 13s\n",
      "132:\ttest: 0.6346247\tbest: 0.6346328 (128)\ttotal: 1m 6s\tremaining: 7m 13s\n",
      "133:\ttest: 0.6346621\tbest: 0.6346621 (133)\ttotal: 1m 7s\tremaining: 7m 13s\n",
      "134:\ttest: 0.6347252\tbest: 0.6347252 (134)\ttotal: 1m 7s\tremaining: 7m 13s\n",
      "135:\ttest: 0.6347004\tbest: 0.6347252 (134)\ttotal: 1m 8s\tremaining: 7m 12s\n",
      "136:\ttest: 0.6347232\tbest: 0.6347252 (134)\ttotal: 1m 8s\tremaining: 7m 12s\n",
      "137:\ttest: 0.6347878\tbest: 0.6347878 (137)\ttotal: 1m 9s\tremaining: 7m 11s\n",
      "138:\ttest: 0.6347942\tbest: 0.6347942 (138)\ttotal: 1m 9s\tremaining: 7m 11s\n",
      "139:\ttest: 0.6347655\tbest: 0.6347942 (138)\ttotal: 1m 10s\tremaining: 7m 11s\n",
      "140:\ttest: 0.6348951\tbest: 0.6348951 (140)\ttotal: 1m 10s\tremaining: 7m 10s\n",
      "141:\ttest: 0.6349284\tbest: 0.6349284 (141)\ttotal: 1m 11s\tremaining: 7m 9s\n",
      "142:\ttest: 0.6349654\tbest: 0.6349654 (142)\ttotal: 1m 11s\tremaining: 7m 8s\n",
      "143:\ttest: 0.6349770\tbest: 0.6349770 (143)\ttotal: 1m 11s\tremaining: 7m 7s\n",
      "144:\ttest: 0.6350356\tbest: 0.6350356 (144)\ttotal: 1m 12s\tremaining: 7m 7s\n",
      "145:\ttest: 0.6350523\tbest: 0.6350523 (145)\ttotal: 1m 12s\tremaining: 7m 6s\n",
      "146:\ttest: 0.6350132\tbest: 0.6350523 (145)\ttotal: 1m 13s\tremaining: 7m 6s\n",
      "147:\ttest: 0.6350766\tbest: 0.6350766 (147)\ttotal: 1m 14s\tremaining: 7m 6s\n",
      "148:\ttest: 0.6351039\tbest: 0.6351039 (148)\ttotal: 1m 14s\tremaining: 7m 6s\n",
      "149:\ttest: 0.6350901\tbest: 0.6351039 (148)\ttotal: 1m 15s\tremaining: 7m 6s\n",
      "150:\ttest: 0.6351457\tbest: 0.6351457 (150)\ttotal: 1m 15s\tremaining: 7m 5s\n",
      "151:\ttest: 0.6351741\tbest: 0.6351741 (151)\ttotal: 1m 16s\tremaining: 7m 4s\n",
      "152:\ttest: 0.6351894\tbest: 0.6351894 (152)\ttotal: 1m 16s\tremaining: 7m 3s\n",
      "153:\ttest: 0.6352178\tbest: 0.6352178 (153)\ttotal: 1m 16s\tremaining: 7m 2s\n",
      "154:\ttest: 0.6352764\tbest: 0.6352764 (154)\ttotal: 1m 17s\tremaining: 7m 1s\n",
      "155:\ttest: 0.6352780\tbest: 0.6352780 (155)\ttotal: 1m 17s\tremaining: 7m 1s\n",
      "156:\ttest: 0.6353133\tbest: 0.6353133 (156)\ttotal: 1m 18s\tremaining: 7m 1s\n",
      "157:\ttest: 0.6352996\tbest: 0.6353133 (156)\ttotal: 1m 18s\tremaining: 7m\n",
      "158:\ttest: 0.6353639\tbest: 0.6353639 (158)\ttotal: 1m 19s\tremaining: 7m\n",
      "159:\ttest: 0.6354321\tbest: 0.6354321 (159)\ttotal: 1m 19s\tremaining: 6m 59s\n",
      "160:\ttest: 0.6355100\tbest: 0.6355100 (160)\ttotal: 1m 20s\tremaining: 6m 59s\n",
      "161:\ttest: 0.6355232\tbest: 0.6355232 (161)\ttotal: 1m 20s\tremaining: 6m 58s\n",
      "162:\ttest: 0.6354924\tbest: 0.6355232 (161)\ttotal: 1m 21s\tremaining: 6m 58s\n",
      "163:\ttest: 0.6355098\tbest: 0.6355232 (161)\ttotal: 1m 21s\tremaining: 6m 57s\n",
      "164:\ttest: 0.6354725\tbest: 0.6355232 (161)\ttotal: 1m 22s\tremaining: 6m 57s\n",
      "165:\ttest: 0.6354759\tbest: 0.6355232 (161)\ttotal: 1m 23s\tremaining: 6m 57s\n",
      "166:\ttest: 0.6355004\tbest: 0.6355232 (161)\ttotal: 1m 23s\tremaining: 6m 56s\n",
      "167:\ttest: 0.6355213\tbest: 0.6355232 (161)\ttotal: 1m 23s\tremaining: 6m 55s\n",
      "168:\ttest: 0.6355602\tbest: 0.6355602 (168)\ttotal: 1m 24s\tremaining: 6m 55s\n",
      "169:\ttest: 0.6355888\tbest: 0.6355888 (169)\ttotal: 1m 24s\tremaining: 6m 54s\n",
      "170:\ttest: 0.6355792\tbest: 0.6355888 (169)\ttotal: 1m 25s\tremaining: 6m 54s\n",
      "171:\ttest: 0.6355952\tbest: 0.6355952 (171)\ttotal: 1m 25s\tremaining: 6m 53s\n",
      "172:\ttest: 0.6358544\tbest: 0.6358544 (172)\ttotal: 1m 26s\tremaining: 6m 53s\n",
      "173:\ttest: 0.6358548\tbest: 0.6358548 (173)\ttotal: 1m 26s\tremaining: 6m 52s\n",
      "174:\ttest: 0.6358201\tbest: 0.6358548 (173)\ttotal: 1m 27s\tremaining: 6m 52s\n",
      "175:\ttest: 0.6358044\tbest: 0.6358548 (173)\ttotal: 1m 27s\tremaining: 6m 51s\n",
      "176:\ttest: 0.6358997\tbest: 0.6358997 (176)\ttotal: 1m 28s\tremaining: 6m 51s\n",
      "177:\ttest: 0.6358884\tbest: 0.6358997 (176)\ttotal: 1m 29s\tremaining: 6m 51s\n",
      "178:\ttest: 0.6358755\tbest: 0.6358997 (176)\ttotal: 1m 29s\tremaining: 6m 50s\n",
      "179:\ttest: 0.6358842\tbest: 0.6358997 (176)\ttotal: 1m 30s\tremaining: 6m 50s\n",
      "180:\ttest: 0.6358934\tbest: 0.6358997 (176)\ttotal: 1m 30s\tremaining: 6m 49s\n",
      "181:\ttest: 0.6359046\tbest: 0.6359046 (181)\ttotal: 1m 31s\tremaining: 6m 49s\n",
      "182:\ttest: 0.6359436\tbest: 0.6359436 (182)\ttotal: 1m 31s\tremaining: 6m 48s\n",
      "183:\ttest: 0.6359659\tbest: 0.6359659 (183)\ttotal: 1m 32s\tremaining: 6m 48s\n",
      "184:\ttest: 0.6359734\tbest: 0.6359734 (184)\ttotal: 1m 32s\tremaining: 6m 48s\n",
      "185:\ttest: 0.6359885\tbest: 0.6359885 (185)\ttotal: 1m 33s\tremaining: 6m 47s\n",
      "186:\ttest: 0.6359853\tbest: 0.6359885 (185)\ttotal: 1m 33s\tremaining: 6m 47s\n",
      "187:\ttest: 0.6359906\tbest: 0.6359906 (187)\ttotal: 1m 34s\tremaining: 6m 47s\n",
      "188:\ttest: 0.6360027\tbest: 0.6360027 (188)\ttotal: 1m 34s\tremaining: 6m 47s\n",
      "189:\ttest: 0.6359924\tbest: 0.6360027 (188)\ttotal: 1m 35s\tremaining: 6m 46s\n",
      "190:\ttest: 0.6360023\tbest: 0.6360027 (188)\ttotal: 1m 35s\tremaining: 6m 46s\n",
      "191:\ttest: 0.6359933\tbest: 0.6360027 (188)\ttotal: 1m 36s\tremaining: 6m 46s\n",
      "192:\ttest: 0.6359631\tbest: 0.6360027 (188)\ttotal: 1m 37s\tremaining: 6m 45s\n",
      "193:\ttest: 0.6360698\tbest: 0.6360698 (193)\ttotal: 1m 37s\tremaining: 6m 45s\n",
      "194:\ttest: 0.6360727\tbest: 0.6360727 (194)\ttotal: 1m 38s\tremaining: 6m 44s\n",
      "195:\ttest: 0.6360658\tbest: 0.6360727 (194)\ttotal: 1m 38s\tremaining: 6m 44s\n",
      "196:\ttest: 0.6360789\tbest: 0.6360789 (196)\ttotal: 1m 39s\tremaining: 6m 44s\n",
      "197:\ttest: 0.6360728\tbest: 0.6360789 (196)\ttotal: 1m 39s\tremaining: 6m 43s\n",
      "198:\ttest: 0.6361089\tbest: 0.6361089 (198)\ttotal: 1m 40s\tremaining: 6m 43s\n",
      "199:\ttest: 0.6361233\tbest: 0.6361233 (199)\ttotal: 1m 40s\tremaining: 6m 43s\n",
      "200:\ttest: 0.6360816\tbest: 0.6361233 (199)\ttotal: 1m 41s\tremaining: 6m 42s\n",
      "201:\ttest: 0.6360536\tbest: 0.6361233 (199)\ttotal: 1m 41s\tremaining: 6m 42s\n",
      "202:\ttest: 0.6360663\tbest: 0.6361233 (199)\ttotal: 1m 42s\tremaining: 6m 41s\n",
      "203:\ttest: 0.6360528\tbest: 0.6361233 (199)\ttotal: 1m 42s\tremaining: 6m 41s\n",
      "204:\ttest: 0.6360194\tbest: 0.6361233 (199)\ttotal: 1m 43s\tremaining: 6m 40s\n",
      "205:\ttest: 0.6359326\tbest: 0.6361233 (199)\ttotal: 1m 43s\tremaining: 6m 40s\n",
      "206:\ttest: 0.6359427\tbest: 0.6361233 (199)\ttotal: 1m 44s\tremaining: 6m 39s\n",
      "207:\ttest: 0.6359212\tbest: 0.6361233 (199)\ttotal: 1m 44s\tremaining: 6m 39s\n",
      "208:\ttest: 0.6359257\tbest: 0.6361233 (199)\ttotal: 1m 45s\tremaining: 6m 38s\n",
      "209:\ttest: 0.6359467\tbest: 0.6361233 (199)\ttotal: 1m 45s\tremaining: 6m 38s\n",
      "210:\ttest: 0.6359418\tbest: 0.6361233 (199)\ttotal: 1m 46s\tremaining: 6m 37s\n",
      "211:\ttest: 0.6359388\tbest: 0.6361233 (199)\ttotal: 1m 46s\tremaining: 6m 37s\n",
      "212:\ttest: 0.6359459\tbest: 0.6361233 (199)\ttotal: 1m 47s\tremaining: 6m 36s\n",
      "213:\ttest: 0.6359054\tbest: 0.6361233 (199)\ttotal: 1m 47s\tremaining: 6m 36s\n",
      "214:\ttest: 0.6358935\tbest: 0.6361233 (199)\ttotal: 1m 48s\tremaining: 6m 35s\n",
      "215:\ttest: 0.6358794\tbest: 0.6361233 (199)\ttotal: 1m 48s\tremaining: 6m 35s\n",
      "216:\ttest: 0.6358716\tbest: 0.6361233 (199)\ttotal: 1m 49s\tremaining: 6m 34s\n",
      "217:\ttest: 0.6358700\tbest: 0.6361233 (199)\ttotal: 1m 49s\tremaining: 6m 34s\n",
      "218:\ttest: 0.6358031\tbest: 0.6361233 (199)\ttotal: 1m 50s\tremaining: 6m 33s\n",
      "219:\ttest: 0.6358009\tbest: 0.6361233 (199)\ttotal: 1m 50s\tremaining: 6m 33s\n",
      "220:\ttest: 0.6357911\tbest: 0.6361233 (199)\ttotal: 1m 51s\tremaining: 6m 32s\n",
      "221:\ttest: 0.6357880\tbest: 0.6361233 (199)\ttotal: 1m 51s\tremaining: 6m 32s\n",
      "222:\ttest: 0.6358062\tbest: 0.6361233 (199)\ttotal: 1m 52s\tremaining: 6m 32s\n",
      "223:\ttest: 0.6358684\tbest: 0.6361233 (199)\ttotal: 1m 53s\tremaining: 6m 31s\n",
      "224:\ttest: 0.6358529\tbest: 0.6361233 (199)\ttotal: 1m 53s\tremaining: 6m 31s\n",
      "225:\ttest: 0.6358379\tbest: 0.6361233 (199)\ttotal: 1m 54s\tremaining: 6m 31s\n",
      "226:\ttest: 0.6358607\tbest: 0.6361233 (199)\ttotal: 1m 54s\tremaining: 6m 30s\n",
      "227:\ttest: 0.6358608\tbest: 0.6361233 (199)\ttotal: 1m 55s\tremaining: 6m 30s\n",
      "228:\ttest: 0.6358609\tbest: 0.6361233 (199)\ttotal: 1m 55s\tremaining: 6m 29s\n",
      "229:\ttest: 0.6358741\tbest: 0.6361233 (199)\ttotal: 1m 56s\tremaining: 6m 29s\n",
      "230:\ttest: 0.6358828\tbest: 0.6361233 (199)\ttotal: 1m 56s\tremaining: 6m 29s\n",
      "231:\ttest: 0.6358809\tbest: 0.6361233 (199)\ttotal: 1m 57s\tremaining: 6m 29s\n",
      "232:\ttest: 0.6358678\tbest: 0.6361233 (199)\ttotal: 1m 58s\tremaining: 6m 28s\n",
      "233:\ttest: 0.6358959\tbest: 0.6361233 (199)\ttotal: 1m 58s\tremaining: 6m 27s\n",
      "234:\ttest: 0.6358973\tbest: 0.6361233 (199)\ttotal: 1m 59s\tremaining: 6m 27s\n",
      "235:\ttest: 0.6358594\tbest: 0.6361233 (199)\ttotal: 1m 59s\tremaining: 6m 27s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "236:\ttest: 0.6358980\tbest: 0.6361233 (199)\ttotal: 2m\tremaining: 6m 26s\n",
      "237:\ttest: 0.6358824\tbest: 0.6361233 (199)\ttotal: 2m\tremaining: 6m 26s\n",
      "238:\ttest: 0.6358496\tbest: 0.6361233 (199)\ttotal: 2m 1s\tremaining: 6m 26s\n",
      "239:\ttest: 0.6358702\tbest: 0.6361233 (199)\ttotal: 2m 1s\tremaining: 6m 26s\n",
      "240:\ttest: 0.6358671\tbest: 0.6361233 (199)\ttotal: 2m 2s\tremaining: 6m 25s\n",
      "241:\ttest: 0.6358728\tbest: 0.6361233 (199)\ttotal: 2m 3s\tremaining: 6m 25s\n",
      "242:\ttest: 0.6358320\tbest: 0.6361233 (199)\ttotal: 2m 3s\tremaining: 6m 24s\n",
      "243:\ttest: 0.6358370\tbest: 0.6361233 (199)\ttotal: 2m 4s\tremaining: 6m 24s\n",
      "244:\ttest: 0.6358281\tbest: 0.6361233 (199)\ttotal: 2m 4s\tremaining: 6m 24s\n",
      "245:\ttest: 0.6358361\tbest: 0.6361233 (199)\ttotal: 2m 5s\tremaining: 6m 23s\n",
      "246:\ttest: 0.6358676\tbest: 0.6361233 (199)\ttotal: 2m 5s\tremaining: 6m 23s\n",
      "247:\ttest: 0.6358399\tbest: 0.6361233 (199)\ttotal: 2m 6s\tremaining: 6m 23s\n",
      "248:\ttest: 0.6358686\tbest: 0.6361233 (199)\ttotal: 2m 6s\tremaining: 6m 22s\n",
      "249:\ttest: 0.6358515\tbest: 0.6361233 (199)\ttotal: 2m 7s\tremaining: 6m 22s\n",
      "250:\ttest: 0.6358770\tbest: 0.6361233 (199)\ttotal: 2m 8s\tremaining: 6m 22s\n",
      "251:\ttest: 0.6358787\tbest: 0.6361233 (199)\ttotal: 2m 8s\tremaining: 6m 21s\n",
      "252:\ttest: 0.6359120\tbest: 0.6361233 (199)\ttotal: 2m 9s\tremaining: 6m 21s\n",
      "253:\ttest: 0.6359367\tbest: 0.6361233 (199)\ttotal: 2m 9s\tremaining: 6m 20s\n",
      "254:\ttest: 0.6359063\tbest: 0.6361233 (199)\ttotal: 2m 10s\tremaining: 6m 20s\n",
      "255:\ttest: 0.6358915\tbest: 0.6361233 (199)\ttotal: 2m 10s\tremaining: 6m 20s\n",
      "256:\ttest: 0.6359012\tbest: 0.6361233 (199)\ttotal: 2m 11s\tremaining: 6m 19s\n",
      "257:\ttest: 0.6359058\tbest: 0.6361233 (199)\ttotal: 2m 11s\tremaining: 6m 19s\n",
      "258:\ttest: 0.6358758\tbest: 0.6361233 (199)\ttotal: 2m 12s\tremaining: 6m 18s\n",
      "259:\ttest: 0.6359118\tbest: 0.6361233 (199)\ttotal: 2m 12s\tremaining: 6m 18s\n",
      "260:\ttest: 0.6359837\tbest: 0.6361233 (199)\ttotal: 2m 13s\tremaining: 6m 17s\n",
      "261:\ttest: 0.6359698\tbest: 0.6361233 (199)\ttotal: 2m 13s\tremaining: 6m 17s\n",
      "262:\ttest: 0.6359780\tbest: 0.6361233 (199)\ttotal: 2m 14s\tremaining: 6m 16s\n",
      "263:\ttest: 0.6359808\tbest: 0.6361233 (199)\ttotal: 2m 15s\tremaining: 6m 16s\n",
      "264:\ttest: 0.6359714\tbest: 0.6361233 (199)\ttotal: 2m 15s\tremaining: 6m 16s\n",
      "265:\ttest: 0.6359381\tbest: 0.6361233 (199)\ttotal: 2m 16s\tremaining: 6m 15s\n",
      "266:\ttest: 0.6359354\tbest: 0.6361233 (199)\ttotal: 2m 16s\tremaining: 6m 15s\n",
      "267:\ttest: 0.6359227\tbest: 0.6361233 (199)\ttotal: 2m 17s\tremaining: 6m 14s\n",
      "268:\ttest: 0.6358982\tbest: 0.6361233 (199)\ttotal: 2m 17s\tremaining: 6m 14s\n",
      "269:\ttest: 0.6358938\tbest: 0.6361233 (199)\ttotal: 2m 18s\tremaining: 6m 13s\n",
      "270:\ttest: 0.6358862\tbest: 0.6361233 (199)\ttotal: 2m 18s\tremaining: 6m 13s\n",
      "271:\ttest: 0.6358986\tbest: 0.6361233 (199)\ttotal: 2m 19s\tremaining: 6m 13s\n",
      "272:\ttest: 0.6360094\tbest: 0.6361233 (199)\ttotal: 2m 20s\tremaining: 6m 12s\n",
      "273:\ttest: 0.6360017\tbest: 0.6361233 (199)\ttotal: 2m 20s\tremaining: 6m 12s\n",
      "274:\ttest: 0.6359690\tbest: 0.6361233 (199)\ttotal: 2m 21s\tremaining: 6m 11s\n",
      "275:\ttest: 0.6360033\tbest: 0.6361233 (199)\ttotal: 2m 21s\tremaining: 6m 11s\n",
      "276:\ttest: 0.6360226\tbest: 0.6361233 (199)\ttotal: 2m 22s\tremaining: 6m 10s\n",
      "277:\ttest: 0.6359990\tbest: 0.6361233 (199)\ttotal: 2m 22s\tremaining: 6m 10s\n",
      "278:\ttest: 0.6360239\tbest: 0.6361233 (199)\ttotal: 2m 23s\tremaining: 6m 10s\n",
      "279:\ttest: 0.6359960\tbest: 0.6361233 (199)\ttotal: 2m 23s\tremaining: 6m 9s\n",
      "280:\ttest: 0.6359956\tbest: 0.6361233 (199)\ttotal: 2m 24s\tremaining: 6m 9s\n",
      "281:\ttest: 0.6359798\tbest: 0.6361233 (199)\ttotal: 2m 24s\tremaining: 6m 8s\n",
      "282:\ttest: 0.6360216\tbest: 0.6361233 (199)\ttotal: 2m 25s\tremaining: 6m 8s\n",
      "283:\ttest: 0.6360025\tbest: 0.6361233 (199)\ttotal: 2m 26s\tremaining: 6m 8s\n",
      "284:\ttest: 0.6360030\tbest: 0.6361233 (199)\ttotal: 2m 26s\tremaining: 6m 8s\n",
      "285:\ttest: 0.6360011\tbest: 0.6361233 (199)\ttotal: 2m 27s\tremaining: 6m 7s\n",
      "286:\ttest: 0.6359758\tbest: 0.6361233 (199)\ttotal: 2m 27s\tremaining: 6m 7s\n",
      "287:\ttest: 0.6359789\tbest: 0.6361233 (199)\ttotal: 2m 28s\tremaining: 6m 6s\n",
      "288:\ttest: 0.6359599\tbest: 0.6361233 (199)\ttotal: 2m 28s\tremaining: 6m 6s\n",
      "289:\ttest: 0.6359939\tbest: 0.6361233 (199)\ttotal: 2m 29s\tremaining: 6m 5s\n",
      "290:\ttest: 0.6359892\tbest: 0.6361233 (199)\ttotal: 2m 29s\tremaining: 6m 5s\n",
      "291:\ttest: 0.6360494\tbest: 0.6361233 (199)\ttotal: 2m 30s\tremaining: 6m 4s\n",
      "292:\ttest: 0.6360392\tbest: 0.6361233 (199)\ttotal: 2m 31s\tremaining: 6m 4s\n",
      "293:\ttest: 0.6360386\tbest: 0.6361233 (199)\ttotal: 2m 31s\tremaining: 6m 4s\n",
      "294:\ttest: 0.6360431\tbest: 0.6361233 (199)\ttotal: 2m 32s\tremaining: 6m 3s\n",
      "295:\ttest: 0.6360792\tbest: 0.6361233 (199)\ttotal: 2m 32s\tremaining: 6m 2s\n",
      "296:\ttest: 0.6360820\tbest: 0.6361233 (199)\ttotal: 2m 33s\tremaining: 6m 2s\n",
      "297:\ttest: 0.6361055\tbest: 0.6361233 (199)\ttotal: 2m 33s\tremaining: 6m 2s\n",
      "298:\ttest: 0.6361324\tbest: 0.6361324 (298)\ttotal: 2m 34s\tremaining: 6m 1s\n",
      "299:\ttest: 0.6361437\tbest: 0.6361437 (299)\ttotal: 2m 34s\tremaining: 6m 1s\n",
      "300:\ttest: 0.6361698\tbest: 0.6361698 (300)\ttotal: 2m 35s\tremaining: 6m\n",
      "301:\ttest: 0.6361659\tbest: 0.6361698 (300)\ttotal: 2m 35s\tremaining: 6m\n",
      "302:\ttest: 0.6361640\tbest: 0.6361698 (300)\ttotal: 2m 36s\tremaining: 6m\n",
      "303:\ttest: 0.6361649\tbest: 0.6361698 (300)\ttotal: 2m 37s\tremaining: 5m 59s\n",
      "304:\ttest: 0.6361108\tbest: 0.6361698 (300)\ttotal: 2m 37s\tremaining: 5m 59s\n",
      "305:\ttest: 0.6360442\tbest: 0.6361698 (300)\ttotal: 2m 38s\tremaining: 5m 58s\n",
      "306:\ttest: 0.6360265\tbest: 0.6361698 (300)\ttotal: 2m 38s\tremaining: 5m 58s\n",
      "307:\ttest: 0.6360781\tbest: 0.6361698 (300)\ttotal: 2m 39s\tremaining: 5m 57s\n",
      "308:\ttest: 0.6361112\tbest: 0.6361698 (300)\ttotal: 2m 39s\tremaining: 5m 57s\n",
      "309:\ttest: 0.6360833\tbest: 0.6361698 (300)\ttotal: 2m 40s\tremaining: 5m 56s\n",
      "310:\ttest: 0.6360659\tbest: 0.6361698 (300)\ttotal: 2m 40s\tremaining: 5m 55s\n",
      "311:\ttest: 0.6360566\tbest: 0.6361698 (300)\ttotal: 2m 41s\tremaining: 5m 55s\n",
      "312:\ttest: 0.6361536\tbest: 0.6361698 (300)\ttotal: 2m 41s\tremaining: 5m 54s\n",
      "313:\ttest: 0.6361157\tbest: 0.6361698 (300)\ttotal: 2m 42s\tremaining: 5m 54s\n",
      "314:\ttest: 0.6361520\tbest: 0.6361698 (300)\ttotal: 2m 42s\tremaining: 5m 53s\n",
      "315:\ttest: 0.6361435\tbest: 0.6361698 (300)\ttotal: 2m 43s\tremaining: 5m 53s\n",
      "316:\ttest: 0.6360691\tbest: 0.6361698 (300)\ttotal: 2m 43s\tremaining: 5m 52s\n",
      "317:\ttest: 0.6360806\tbest: 0.6361698 (300)\ttotal: 2m 44s\tremaining: 5m 52s\n",
      "318:\ttest: 0.6360658\tbest: 0.6361698 (300)\ttotal: 2m 44s\tremaining: 5m 51s\n",
      "319:\ttest: 0.6360781\tbest: 0.6361698 (300)\ttotal: 2m 45s\tremaining: 5m 51s\n",
      "320:\ttest: 0.6360776\tbest: 0.6361698 (300)\ttotal: 2m 45s\tremaining: 5m 50s\n",
      "321:\ttest: 0.6360799\tbest: 0.6361698 (300)\ttotal: 2m 46s\tremaining: 5m 50s\n",
      "322:\ttest: 0.6360660\tbest: 0.6361698 (300)\ttotal: 2m 47s\tremaining: 5m 50s\n",
      "323:\ttest: 0.6360729\tbest: 0.6361698 (300)\ttotal: 2m 47s\tremaining: 5m 49s\n",
      "324:\ttest: 0.6360675\tbest: 0.6361698 (300)\ttotal: 2m 48s\tremaining: 5m 49s\n",
      "325:\ttest: 0.6360181\tbest: 0.6361698 (300)\ttotal: 2m 48s\tremaining: 5m 48s\n",
      "326:\ttest: 0.6360283\tbest: 0.6361698 (300)\ttotal: 2m 49s\tremaining: 5m 48s\n",
      "327:\ttest: 0.6360081\tbest: 0.6361698 (300)\ttotal: 2m 49s\tremaining: 5m 47s\n",
      "328:\ttest: 0.6360191\tbest: 0.6361698 (300)\ttotal: 2m 50s\tremaining: 5m 47s\n",
      "329:\ttest: 0.6360471\tbest: 0.6361698 (300)\ttotal: 2m 50s\tremaining: 5m 47s\n",
      "330:\ttest: 0.6360576\tbest: 0.6361698 (300)\ttotal: 2m 51s\tremaining: 5m 46s\n",
      "331:\ttest: 0.6360471\tbest: 0.6361698 (300)\ttotal: 2m 51s\tremaining: 5m 45s\n",
      "332:\ttest: 0.6359887\tbest: 0.6361698 (300)\ttotal: 2m 52s\tremaining: 5m 45s\n",
      "333:\ttest: 0.6359967\tbest: 0.6361698 (300)\ttotal: 2m 53s\tremaining: 5m 45s\n",
      "334:\ttest: 0.6359586\tbest: 0.6361698 (300)\ttotal: 2m 53s\tremaining: 5m 44s\n",
      "335:\ttest: 0.6359695\tbest: 0.6361698 (300)\ttotal: 2m 54s\tremaining: 5m 44s\n",
      "336:\ttest: 0.6359546\tbest: 0.6361698 (300)\ttotal: 2m 54s\tremaining: 5m 43s\n",
      "337:\ttest: 0.6359302\tbest: 0.6361698 (300)\ttotal: 2m 55s\tremaining: 5m 43s\n",
      "338:\ttest: 0.6359004\tbest: 0.6361698 (300)\ttotal: 2m 55s\tremaining: 5m 42s\n",
      "339:\ttest: 0.6358565\tbest: 0.6361698 (300)\ttotal: 2m 56s\tremaining: 5m 42s\n",
      "340:\ttest: 0.6358404\tbest: 0.6361698 (300)\ttotal: 2m 56s\tremaining: 5m 41s\n",
      "341:\ttest: 0.6358463\tbest: 0.6361698 (300)\ttotal: 2m 57s\tremaining: 5m 40s\n",
      "342:\ttest: 0.6358135\tbest: 0.6361698 (300)\ttotal: 2m 57s\tremaining: 5m 40s\n",
      "343:\ttest: 0.6358398\tbest: 0.6361698 (300)\ttotal: 2m 58s\tremaining: 5m 40s\n",
      "344:\ttest: 0.6358507\tbest: 0.6361698 (300)\ttotal: 2m 58s\tremaining: 5m 39s\n",
      "345:\ttest: 0.6358755\tbest: 0.6361698 (300)\ttotal: 2m 59s\tremaining: 5m 39s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "346:\ttest: 0.6359145\tbest: 0.6361698 (300)\ttotal: 2m 59s\tremaining: 5m 38s\n",
      "347:\ttest: 0.6359015\tbest: 0.6361698 (300)\ttotal: 3m\tremaining: 5m 38s\n",
      "348:\ttest: 0.6358892\tbest: 0.6361698 (300)\ttotal: 3m 1s\tremaining: 5m 37s\n",
      "349:\ttest: 0.6358976\tbest: 0.6361698 (300)\ttotal: 3m 1s\tremaining: 5m 37s\n",
      "350:\ttest: 0.6358999\tbest: 0.6361698 (300)\ttotal: 3m 2s\tremaining: 5m 37s\n",
      "351:\ttest: 0.6359095\tbest: 0.6361698 (300)\ttotal: 3m 2s\tremaining: 5m 36s\n",
      "352:\ttest: 0.6359189\tbest: 0.6361698 (300)\ttotal: 3m 3s\tremaining: 5m 36s\n",
      "353:\ttest: 0.6359228\tbest: 0.6361698 (300)\ttotal: 3m 3s\tremaining: 5m 35s\n",
      "354:\ttest: 0.6359077\tbest: 0.6361698 (300)\ttotal: 3m 4s\tremaining: 5m 35s\n",
      "355:\ttest: 0.6359481\tbest: 0.6361698 (300)\ttotal: 3m 5s\tremaining: 5m 34s\n",
      "356:\ttest: 0.6359743\tbest: 0.6361698 (300)\ttotal: 3m 5s\tremaining: 5m 34s\n",
      "357:\ttest: 0.6359016\tbest: 0.6361698 (300)\ttotal: 3m 6s\tremaining: 5m 33s\n",
      "358:\ttest: 0.6359380\tbest: 0.6361698 (300)\ttotal: 3m 6s\tremaining: 5m 33s\n",
      "359:\ttest: 0.6359277\tbest: 0.6361698 (300)\ttotal: 3m 7s\tremaining: 5m 33s\n",
      "360:\ttest: 0.6359283\tbest: 0.6361698 (300)\ttotal: 3m 7s\tremaining: 5m 32s\n",
      "361:\ttest: 0.6359272\tbest: 0.6361698 (300)\ttotal: 3m 8s\tremaining: 5m 32s\n",
      "362:\ttest: 0.6359503\tbest: 0.6361698 (300)\ttotal: 3m 9s\tremaining: 5m 31s\n",
      "363:\ttest: 0.6359574\tbest: 0.6361698 (300)\ttotal: 3m 9s\tremaining: 5m 31s\n",
      "364:\ttest: 0.6359349\tbest: 0.6361698 (300)\ttotal: 3m 10s\tremaining: 5m 30s\n",
      "365:\ttest: 0.6359200\tbest: 0.6361698 (300)\ttotal: 3m 10s\tremaining: 5m 30s\n",
      "366:\ttest: 0.6359678\tbest: 0.6361698 (300)\ttotal: 3m 11s\tremaining: 5m 29s\n",
      "367:\ttest: 0.6359863\tbest: 0.6361698 (300)\ttotal: 3m 11s\tremaining: 5m 29s\n",
      "368:\ttest: 0.6359657\tbest: 0.6361698 (300)\ttotal: 3m 12s\tremaining: 5m 28s\n",
      "369:\ttest: 0.6360358\tbest: 0.6361698 (300)\ttotal: 3m 12s\tremaining: 5m 28s\n",
      "370:\ttest: 0.6360466\tbest: 0.6361698 (300)\ttotal: 3m 13s\tremaining: 5m 27s\n",
      "371:\ttest: 0.6360211\tbest: 0.6361698 (300)\ttotal: 3m 13s\tremaining: 5m 27s\n",
      "372:\ttest: 0.6360081\tbest: 0.6361698 (300)\ttotal: 3m 14s\tremaining: 5m 27s\n",
      "373:\ttest: 0.6359875\tbest: 0.6361698 (300)\ttotal: 3m 15s\tremaining: 5m 26s\n",
      "374:\ttest: 0.6359663\tbest: 0.6361698 (300)\ttotal: 3m 15s\tremaining: 5m 26s\n",
      "375:\ttest: 0.6360006\tbest: 0.6361698 (300)\ttotal: 3m 16s\tremaining: 5m 25s\n",
      "376:\ttest: 0.6360079\tbest: 0.6361698 (300)\ttotal: 3m 16s\tremaining: 5m 25s\n",
      "377:\ttest: 0.6360162\tbest: 0.6361698 (300)\ttotal: 3m 17s\tremaining: 5m 24s\n",
      "378:\ttest: 0.6359849\tbest: 0.6361698 (300)\ttotal: 3m 17s\tremaining: 5m 24s\n",
      "379:\ttest: 0.6359924\tbest: 0.6361698 (300)\ttotal: 3m 18s\tremaining: 5m 24s\n",
      "380:\ttest: 0.6359687\tbest: 0.6361698 (300)\ttotal: 3m 19s\tremaining: 5m 23s\n",
      "381:\ttest: 0.6360032\tbest: 0.6361698 (300)\ttotal: 3m 19s\tremaining: 5m 23s\n",
      "382:\ttest: 0.6360057\tbest: 0.6361698 (300)\ttotal: 3m 20s\tremaining: 5m 22s\n",
      "383:\ttest: 0.6359664\tbest: 0.6361698 (300)\ttotal: 3m 20s\tremaining: 5m 22s\n",
      "384:\ttest: 0.6359510\tbest: 0.6361698 (300)\ttotal: 3m 21s\tremaining: 5m 21s\n",
      "385:\ttest: 0.6359241\tbest: 0.6361698 (300)\ttotal: 3m 22s\tremaining: 5m 21s\n",
      "386:\ttest: 0.6359588\tbest: 0.6361698 (300)\ttotal: 3m 22s\tremaining: 5m 20s\n",
      "387:\ttest: 0.6360083\tbest: 0.6361698 (300)\ttotal: 3m 23s\tremaining: 5m 20s\n",
      "388:\ttest: 0.6360142\tbest: 0.6361698 (300)\ttotal: 3m 23s\tremaining: 5m 20s\n",
      "389:\ttest: 0.6360203\tbest: 0.6361698 (300)\ttotal: 3m 24s\tremaining: 5m 19s\n",
      "390:\ttest: 0.6360872\tbest: 0.6361698 (300)\ttotal: 3m 24s\tremaining: 5m 19s\n",
      "391:\ttest: 0.6360958\tbest: 0.6361698 (300)\ttotal: 3m 25s\tremaining: 5m 18s\n",
      "392:\ttest: 0.6360845\tbest: 0.6361698 (300)\ttotal: 3m 25s\tremaining: 5m 18s\n",
      "393:\ttest: 0.6361351\tbest: 0.6361698 (300)\ttotal: 3m 26s\tremaining: 5m 17s\n",
      "394:\ttest: 0.6361455\tbest: 0.6361698 (300)\ttotal: 3m 27s\tremaining: 5m 17s\n",
      "395:\ttest: 0.6361446\tbest: 0.6361698 (300)\ttotal: 3m 27s\tremaining: 5m 16s\n",
      "396:\ttest: 0.6361436\tbest: 0.6361698 (300)\ttotal: 3m 28s\tremaining: 5m 16s\n",
      "397:\ttest: 0.6361455\tbest: 0.6361698 (300)\ttotal: 3m 28s\tremaining: 5m 15s\n",
      "398:\ttest: 0.6361038\tbest: 0.6361698 (300)\ttotal: 3m 29s\tremaining: 5m 15s\n",
      "399:\ttest: 0.6361005\tbest: 0.6361698 (300)\ttotal: 3m 29s\tremaining: 5m 14s\n",
      "400:\ttest: 0.6361349\tbest: 0.6361698 (300)\ttotal: 3m 30s\tremaining: 5m 14s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.6361698213\n",
      "bestIteration = 300\n",
      "\n",
      "Shrink model to first 301 iterations.\n",
      "err:  0.6361698212714728\n",
      "0:\ttest: 0.5742307\tbest: 0.5742307 (0)\ttotal: 531ms\tremaining: 8m 50s\n",
      "1:\ttest: 0.5809965\tbest: 0.5809965 (1)\ttotal: 788ms\tremaining: 6m 33s\n",
      "2:\ttest: 0.5805268\tbest: 0.5809965 (1)\ttotal: 1.24s\tremaining: 6m 52s\n",
      "3:\ttest: 0.5812035\tbest: 0.5812035 (3)\ttotal: 1.77s\tremaining: 7m 20s\n",
      "4:\ttest: 0.5827024\tbest: 0.5827024 (4)\ttotal: 2.2s\tremaining: 7m 17s\n",
      "5:\ttest: 0.5822523\tbest: 0.5827024 (4)\ttotal: 2.76s\tremaining: 7m 37s\n",
      "6:\ttest: 0.5830030\tbest: 0.5830030 (6)\ttotal: 3.63s\tremaining: 8m 35s\n",
      "7:\ttest: 0.5837226\tbest: 0.5837226 (7)\ttotal: 4.25s\tremaining: 8m 47s\n",
      "8:\ttest: 0.5833688\tbest: 0.5837226 (7)\ttotal: 4.68s\tremaining: 8m 35s\n",
      "9:\ttest: 0.5838578\tbest: 0.5838578 (9)\ttotal: 5.15s\tremaining: 8m 30s\n",
      "10:\ttest: 0.5841684\tbest: 0.5841684 (10)\ttotal: 5.86s\tremaining: 8m 47s\n",
      "11:\ttest: 0.5848801\tbest: 0.5848801 (11)\ttotal: 6.61s\tremaining: 9m 4s\n",
      "12:\ttest: 0.5848297\tbest: 0.5848801 (11)\ttotal: 7.25s\tremaining: 9m 10s\n",
      "13:\ttest: 0.5848055\tbest: 0.5848801 (11)\ttotal: 7.73s\tremaining: 9m 4s\n",
      "14:\ttest: 0.5847668\tbest: 0.5848801 (11)\ttotal: 8.19s\tremaining: 8m 57s\n",
      "15:\ttest: 0.5855726\tbest: 0.5855726 (15)\ttotal: 8.69s\tremaining: 8m 54s\n",
      "16:\ttest: 0.5860668\tbest: 0.5860668 (16)\ttotal: 9.09s\tremaining: 8m 45s\n",
      "17:\ttest: 0.5862513\tbest: 0.5862513 (17)\ttotal: 9.67s\tremaining: 8m 47s\n",
      "18:\ttest: 0.5863427\tbest: 0.5863427 (18)\ttotal: 10.2s\tremaining: 8m 46s\n",
      "19:\ttest: 0.5864173\tbest: 0.5864173 (19)\ttotal: 10.7s\tremaining: 8m 42s\n",
      "20:\ttest: 0.5864792\tbest: 0.5864792 (20)\ttotal: 11.1s\tremaining: 8m 38s\n",
      "21:\ttest: 0.5863634\tbest: 0.5864792 (20)\ttotal: 11.6s\tremaining: 8m 37s\n",
      "22:\ttest: 0.5862241\tbest: 0.5864792 (20)\ttotal: 12.2s\tremaining: 8m 38s\n",
      "23:\ttest: 0.5862839\tbest: 0.5864792 (20)\ttotal: 12.8s\tremaining: 8m 41s\n",
      "24:\ttest: 0.5861577\tbest: 0.5864792 (20)\ttotal: 13.3s\tremaining: 8m 40s\n",
      "25:\ttest: 0.5860143\tbest: 0.5864792 (20)\ttotal: 13.9s\tremaining: 8m 39s\n",
      "26:\ttest: 0.5867463\tbest: 0.5867463 (26)\ttotal: 14.5s\tremaining: 8m 41s\n",
      "27:\ttest: 0.5868410\tbest: 0.5868410 (27)\ttotal: 15s\tremaining: 8m 41s\n",
      "28:\ttest: 0.5867838\tbest: 0.5868410 (27)\ttotal: 15.6s\tremaining: 8m 42s\n",
      "29:\ttest: 0.5867699\tbest: 0.5868410 (27)\ttotal: 16.1s\tremaining: 8m 41s\n",
      "30:\ttest: 0.5868976\tbest: 0.5868976 (30)\ttotal: 16.7s\tremaining: 8m 42s\n",
      "31:\ttest: 0.5873644\tbest: 0.5873644 (31)\ttotal: 17.3s\tremaining: 8m 44s\n",
      "32:\ttest: 0.5873922\tbest: 0.5873922 (32)\ttotal: 17.9s\tremaining: 8m 44s\n",
      "33:\ttest: 0.5876349\tbest: 0.5876349 (33)\ttotal: 18.5s\tremaining: 8m 44s\n",
      "34:\ttest: 0.5880104\tbest: 0.5880104 (34)\ttotal: 19.1s\tremaining: 8m 45s\n",
      "35:\ttest: 0.5882615\tbest: 0.5882615 (35)\ttotal: 19.6s\tremaining: 8m 44s\n",
      "36:\ttest: 0.5883785\tbest: 0.5883785 (36)\ttotal: 20.1s\tremaining: 8m 43s\n",
      "37:\ttest: 0.5884392\tbest: 0.5884392 (37)\ttotal: 20.7s\tremaining: 8m 44s\n",
      "38:\ttest: 0.5883547\tbest: 0.5884392 (37)\ttotal: 21.3s\tremaining: 8m 44s\n",
      "39:\ttest: 0.5881997\tbest: 0.5884392 (37)\ttotal: 21.9s\tremaining: 8m 44s\n",
      "40:\ttest: 0.5881943\tbest: 0.5884392 (37)\ttotal: 22.4s\tremaining: 8m 43s\n",
      "41:\ttest: 0.5883925\tbest: 0.5884392 (37)\ttotal: 22.9s\tremaining: 8m 43s\n",
      "42:\ttest: 0.5884309\tbest: 0.5884392 (37)\ttotal: 23.5s\tremaining: 8m 42s\n",
      "43:\ttest: 0.5887746\tbest: 0.5887746 (43)\ttotal: 24.1s\tremaining: 8m 42s\n",
      "44:\ttest: 0.5885829\tbest: 0.5887746 (43)\ttotal: 24.6s\tremaining: 8m 43s\n",
      "45:\ttest: 0.5952404\tbest: 0.5952404 (45)\ttotal: 25.2s\tremaining: 8m 41s\n",
      "46:\ttest: 0.6011115\tbest: 0.6011115 (46)\ttotal: 25.6s\tremaining: 8m 38s\n",
      "47:\ttest: 0.6059499\tbest: 0.6059499 (47)\ttotal: 26s\tremaining: 8m 35s\n",
      "48:\ttest: 0.6097756\tbest: 0.6097756 (48)\ttotal: 26.4s\tremaining: 8m 32s\n",
      "49:\ttest: 0.6131728\tbest: 0.6131728 (49)\ttotal: 26.9s\tremaining: 8m 30s\n",
      "50:\ttest: 0.6157912\tbest: 0.6157912 (50)\ttotal: 27.3s\tremaining: 8m 28s\n",
      "51:\ttest: 0.6180830\tbest: 0.6180830 (51)\ttotal: 27.8s\tremaining: 8m 25s\n",
      "52:\ttest: 0.6204888\tbest: 0.6204888 (52)\ttotal: 28.1s\tremaining: 8m 22s\n",
      "53:\ttest: 0.6219371\tbest: 0.6219371 (53)\ttotal: 28.5s\tremaining: 8m 19s\n",
      "54:\ttest: 0.6229435\tbest: 0.6229435 (54)\ttotal: 29s\tremaining: 8m 18s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "55:\ttest: 0.6239611\tbest: 0.6239611 (55)\ttotal: 29.5s\tremaining: 8m 16s\n",
      "56:\ttest: 0.6247667\tbest: 0.6247667 (56)\ttotal: 30s\tremaining: 8m 16s\n",
      "57:\ttest: 0.6257182\tbest: 0.6257182 (57)\ttotal: 30.4s\tremaining: 8m 13s\n",
      "58:\ttest: 0.6263093\tbest: 0.6263093 (58)\ttotal: 30.9s\tremaining: 8m 12s\n",
      "59:\ttest: 0.6271547\tbest: 0.6271547 (59)\ttotal: 31.2s\tremaining: 8m 9s\n",
      "60:\ttest: 0.6277454\tbest: 0.6277454 (60)\ttotal: 31.7s\tremaining: 8m 7s\n",
      "61:\ttest: 0.6283172\tbest: 0.6283172 (61)\ttotal: 32.2s\tremaining: 8m 7s\n",
      "62:\ttest: 0.6288247\tbest: 0.6288247 (62)\ttotal: 32.6s\tremaining: 8m 5s\n",
      "63:\ttest: 0.6292658\tbest: 0.6292658 (63)\ttotal: 33.1s\tremaining: 8m 3s\n",
      "64:\ttest: 0.6295486\tbest: 0.6295486 (64)\ttotal: 33.5s\tremaining: 8m 1s\n",
      "65:\ttest: 0.6297678\tbest: 0.6297678 (65)\ttotal: 34s\tremaining: 8m\n",
      "66:\ttest: 0.6300754\tbest: 0.6300754 (66)\ttotal: 34.4s\tremaining: 7m 58s\n",
      "67:\ttest: 0.6302798\tbest: 0.6302798 (67)\ttotal: 34.9s\tremaining: 7m 58s\n",
      "68:\ttest: 0.6305100\tbest: 0.6305100 (68)\ttotal: 35.3s\tremaining: 7m 56s\n",
      "69:\ttest: 0.6306711\tbest: 0.6306711 (69)\ttotal: 35.9s\tremaining: 7m 56s\n",
      "70:\ttest: 0.6309601\tbest: 0.6309601 (70)\ttotal: 36.5s\tremaining: 7m 57s\n",
      "71:\ttest: 0.6311302\tbest: 0.6311302 (71)\ttotal: 37s\tremaining: 7m 57s\n",
      "72:\ttest: 0.6312976\tbest: 0.6312976 (72)\ttotal: 37.5s\tremaining: 7m 56s\n",
      "73:\ttest: 0.6315918\tbest: 0.6315918 (73)\ttotal: 37.9s\tremaining: 7m 54s\n",
      "74:\ttest: 0.6317217\tbest: 0.6317217 (74)\ttotal: 38.3s\tremaining: 7m 52s\n",
      "75:\ttest: 0.6318236\tbest: 0.6318236 (75)\ttotal: 38.8s\tremaining: 7m 52s\n",
      "76:\ttest: 0.6319337\tbest: 0.6319337 (76)\ttotal: 39.3s\tremaining: 7m 50s\n",
      "77:\ttest: 0.6320998\tbest: 0.6320998 (77)\ttotal: 39.7s\tremaining: 7m 49s\n",
      "78:\ttest: 0.6322167\tbest: 0.6322167 (78)\ttotal: 40.2s\tremaining: 7m 48s\n",
      "79:\ttest: 0.6322292\tbest: 0.6322292 (79)\ttotal: 40.7s\tremaining: 7m 47s\n",
      "80:\ttest: 0.6323629\tbest: 0.6323629 (80)\ttotal: 41.1s\tremaining: 7m 46s\n",
      "81:\ttest: 0.6325122\tbest: 0.6325122 (81)\ttotal: 41.5s\tremaining: 7m 44s\n",
      "82:\ttest: 0.6326562\tbest: 0.6326562 (82)\ttotal: 42s\tremaining: 7m 43s\n",
      "83:\ttest: 0.6326168\tbest: 0.6326562 (82)\ttotal: 42.5s\tremaining: 7m 42s\n",
      "84:\ttest: 0.6328245\tbest: 0.6328245 (84)\ttotal: 42.9s\tremaining: 7m 42s\n",
      "85:\ttest: 0.6329724\tbest: 0.6329724 (85)\ttotal: 43.5s\tremaining: 7m 42s\n",
      "86:\ttest: 0.6330010\tbest: 0.6330010 (86)\ttotal: 44s\tremaining: 7m 41s\n",
      "87:\ttest: 0.6331451\tbest: 0.6331451 (87)\ttotal: 44.4s\tremaining: 7m 40s\n",
      "88:\ttest: 0.6333390\tbest: 0.6333390 (88)\ttotal: 44.9s\tremaining: 7m 39s\n",
      "89:\ttest: 0.6336796\tbest: 0.6336796 (89)\ttotal: 45.2s\tremaining: 7m 36s\n",
      "90:\ttest: 0.6339298\tbest: 0.6339298 (90)\ttotal: 45.6s\tremaining: 7m 35s\n",
      "91:\ttest: 0.6340380\tbest: 0.6340380 (91)\ttotal: 46s\tremaining: 7m 34s\n",
      "92:\ttest: 0.6341351\tbest: 0.6341351 (92)\ttotal: 46.5s\tremaining: 7m 33s\n",
      "93:\ttest: 0.6342795\tbest: 0.6342795 (93)\ttotal: 47s\tremaining: 7m 32s\n",
      "94:\ttest: 0.6343799\tbest: 0.6343799 (94)\ttotal: 47.4s\tremaining: 7m 31s\n",
      "95:\ttest: 0.6344507\tbest: 0.6344507 (95)\ttotal: 47.9s\tremaining: 7m 30s\n",
      "96:\ttest: 0.6344901\tbest: 0.6344901 (96)\ttotal: 48.2s\tremaining: 7m 29s\n",
      "97:\ttest: 0.6346272\tbest: 0.6346272 (97)\ttotal: 48.9s\tremaining: 7m 29s\n",
      "98:\ttest: 0.6347588\tbest: 0.6347588 (98)\ttotal: 49.2s\tremaining: 7m 27s\n",
      "99:\ttest: 0.6349182\tbest: 0.6349182 (99)\ttotal: 49.6s\tremaining: 7m 26s\n",
      "100:\ttest: 0.6349763\tbest: 0.6349763 (100)\ttotal: 50.2s\tremaining: 7m 26s\n",
      "101:\ttest: 0.6350046\tbest: 0.6350046 (101)\ttotal: 50.6s\tremaining: 7m 25s\n",
      "102:\ttest: 0.6350239\tbest: 0.6350239 (102)\ttotal: 51.1s\tremaining: 7m 25s\n",
      "103:\ttest: 0.6350558\tbest: 0.6350558 (103)\ttotal: 51.5s\tremaining: 7m 23s\n",
      "104:\ttest: 0.6351465\tbest: 0.6351465 (104)\ttotal: 52s\tremaining: 7m 23s\n",
      "105:\ttest: 0.6350756\tbest: 0.6351465 (104)\ttotal: 52.5s\tremaining: 7m 22s\n",
      "106:\ttest: 0.6351287\tbest: 0.6351465 (104)\ttotal: 53s\tremaining: 7m 22s\n",
      "107:\ttest: 0.6351735\tbest: 0.6351735 (107)\ttotal: 53.7s\tremaining: 7m 23s\n",
      "108:\ttest: 0.6351841\tbest: 0.6351841 (108)\ttotal: 54.2s\tremaining: 7m 22s\n",
      "109:\ttest: 0.6352450\tbest: 0.6352450 (109)\ttotal: 54.8s\tremaining: 7m 23s\n",
      "110:\ttest: 0.6352551\tbest: 0.6352551 (110)\ttotal: 55.5s\tremaining: 7m 24s\n",
      "111:\ttest: 0.6354158\tbest: 0.6354158 (111)\ttotal: 55.9s\tremaining: 7m 23s\n",
      "112:\ttest: 0.6354470\tbest: 0.6354470 (112)\ttotal: 56.5s\tremaining: 7m 23s\n",
      "113:\ttest: 0.6355339\tbest: 0.6355339 (113)\ttotal: 57.1s\tremaining: 7m 23s\n",
      "114:\ttest: 0.6355067\tbest: 0.6355339 (113)\ttotal: 57.7s\tremaining: 7m 24s\n",
      "115:\ttest: 0.6357091\tbest: 0.6357091 (115)\ttotal: 58.3s\tremaining: 7m 24s\n",
      "116:\ttest: 0.6356481\tbest: 0.6357091 (115)\ttotal: 58.8s\tremaining: 7m 24s\n",
      "117:\ttest: 0.6356478\tbest: 0.6357091 (115)\ttotal: 59.3s\tremaining: 7m 23s\n",
      "118:\ttest: 0.6356698\tbest: 0.6357091 (115)\ttotal: 59.9s\tremaining: 7m 23s\n",
      "119:\ttest: 0.6356883\tbest: 0.6357091 (115)\ttotal: 1m\tremaining: 7m 22s\n",
      "120:\ttest: 0.6357228\tbest: 0.6357228 (120)\ttotal: 1m\tremaining: 7m 21s\n",
      "121:\ttest: 0.6357491\tbest: 0.6357491 (121)\ttotal: 1m 1s\tremaining: 7m 21s\n",
      "122:\ttest: 0.6357345\tbest: 0.6357491 (121)\ttotal: 1m 1s\tremaining: 7m 20s\n",
      "123:\ttest: 0.6357611\tbest: 0.6357611 (123)\ttotal: 1m 2s\tremaining: 7m 19s\n",
      "124:\ttest: 0.6357597\tbest: 0.6357611 (123)\ttotal: 1m 2s\tremaining: 7m 19s\n",
      "125:\ttest: 0.6358878\tbest: 0.6358878 (125)\ttotal: 1m 3s\tremaining: 7m 18s\n",
      "126:\ttest: 0.6358475\tbest: 0.6358878 (125)\ttotal: 1m 3s\tremaining: 7m 18s\n",
      "127:\ttest: 0.6358685\tbest: 0.6358878 (125)\ttotal: 1m 4s\tremaining: 7m 17s\n",
      "128:\ttest: 0.6359544\tbest: 0.6359544 (128)\ttotal: 1m 4s\tremaining: 7m 17s\n",
      "129:\ttest: 0.6359059\tbest: 0.6359544 (128)\ttotal: 1m 5s\tremaining: 7m 16s\n",
      "130:\ttest: 0.6359285\tbest: 0.6359544 (128)\ttotal: 1m 5s\tremaining: 7m 16s\n",
      "131:\ttest: 0.6360141\tbest: 0.6360141 (131)\ttotal: 1m 6s\tremaining: 7m 15s\n",
      "132:\ttest: 0.6360807\tbest: 0.6360807 (132)\ttotal: 1m 6s\tremaining: 7m 15s\n",
      "133:\ttest: 0.6361020\tbest: 0.6361020 (133)\ttotal: 1m 7s\tremaining: 7m 14s\n",
      "134:\ttest: 0.6361896\tbest: 0.6361896 (134)\ttotal: 1m 7s\tremaining: 7m 14s\n",
      "135:\ttest: 0.6361956\tbest: 0.6361956 (135)\ttotal: 1m 8s\tremaining: 7m 14s\n",
      "136:\ttest: 0.6362350\tbest: 0.6362350 (136)\ttotal: 1m 8s\tremaining: 7m 14s\n",
      "137:\ttest: 0.6363246\tbest: 0.6363246 (137)\ttotal: 1m 9s\tremaining: 7m 13s\n",
      "138:\ttest: 0.6363570\tbest: 0.6363570 (138)\ttotal: 1m 9s\tremaining: 7m 13s\n",
      "139:\ttest: 0.6363020\tbest: 0.6363570 (138)\ttotal: 1m 10s\tremaining: 7m 12s\n",
      "140:\ttest: 0.6362879\tbest: 0.6363570 (138)\ttotal: 1m 11s\tremaining: 7m 12s\n",
      "141:\ttest: 0.6362993\tbest: 0.6363570 (138)\ttotal: 1m 11s\tremaining: 7m 11s\n",
      "142:\ttest: 0.6363247\tbest: 0.6363570 (138)\ttotal: 1m 11s\tremaining: 7m 11s\n",
      "143:\ttest: 0.6363336\tbest: 0.6363570 (138)\ttotal: 1m 12s\tremaining: 7m 10s\n",
      "144:\ttest: 0.6363093\tbest: 0.6363570 (138)\ttotal: 1m 13s\tremaining: 7m 10s\n",
      "145:\ttest: 0.6362827\tbest: 0.6363570 (138)\ttotal: 1m 13s\tremaining: 7m 10s\n",
      "146:\ttest: 0.6362890\tbest: 0.6363570 (138)\ttotal: 1m 14s\tremaining: 7m 10s\n",
      "147:\ttest: 0.6362757\tbest: 0.6363570 (138)\ttotal: 1m 14s\tremaining: 7m 10s\n",
      "148:\ttest: 0.6363173\tbest: 0.6363570 (138)\ttotal: 1m 15s\tremaining: 7m 10s\n",
      "149:\ttest: 0.6363753\tbest: 0.6363753 (149)\ttotal: 1m 15s\tremaining: 7m 10s\n",
      "150:\ttest: 0.6363985\tbest: 0.6363985 (150)\ttotal: 1m 16s\tremaining: 7m 10s\n",
      "151:\ttest: 0.6364364\tbest: 0.6364364 (151)\ttotal: 1m 17s\tremaining: 7m 9s\n",
      "152:\ttest: 0.6364562\tbest: 0.6364562 (152)\ttotal: 1m 17s\tremaining: 7m 9s\n",
      "153:\ttest: 0.6364507\tbest: 0.6364562 (152)\ttotal: 1m 18s\tremaining: 7m 9s\n",
      "154:\ttest: 0.6364651\tbest: 0.6364651 (154)\ttotal: 1m 18s\tremaining: 7m 8s\n",
      "155:\ttest: 0.6364498\tbest: 0.6364651 (154)\ttotal: 1m 19s\tremaining: 7m 8s\n",
      "156:\ttest: 0.6364530\tbest: 0.6364651 (154)\ttotal: 1m 19s\tremaining: 7m 8s\n",
      "157:\ttest: 0.6364902\tbest: 0.6364902 (157)\ttotal: 1m 20s\tremaining: 7m 7s\n",
      "158:\ttest: 0.6365131\tbest: 0.6365131 (158)\ttotal: 1m 20s\tremaining: 7m 6s\n",
      "159:\ttest: 0.6365103\tbest: 0.6365131 (158)\ttotal: 1m 21s\tremaining: 7m 5s\n",
      "160:\ttest: 0.6366734\tbest: 0.6366734 (160)\ttotal: 1m 21s\tremaining: 7m 6s\n",
      "161:\ttest: 0.6366591\tbest: 0.6366734 (160)\ttotal: 1m 22s\tremaining: 7m 5s\n",
      "162:\ttest: 0.6366470\tbest: 0.6366734 (160)\ttotal: 1m 22s\tremaining: 7m 5s\n",
      "163:\ttest: 0.6366574\tbest: 0.6366734 (160)\ttotal: 1m 23s\tremaining: 7m 4s\n",
      "164:\ttest: 0.6366984\tbest: 0.6366984 (164)\ttotal: 1m 23s\tremaining: 7m 3s\n",
      "165:\ttest: 0.6367161\tbest: 0.6367161 (165)\ttotal: 1m 24s\tremaining: 7m 3s\n",
      "166:\ttest: 0.6367197\tbest: 0.6367197 (166)\ttotal: 1m 24s\tremaining: 7m 3s\n",
      "167:\ttest: 0.6367256\tbest: 0.6367256 (167)\ttotal: 1m 25s\tremaining: 7m 3s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "168:\ttest: 0.6367022\tbest: 0.6367256 (167)\ttotal: 1m 25s\tremaining: 7m 2s\n",
      "169:\ttest: 0.6367094\tbest: 0.6367256 (167)\ttotal: 1m 26s\tremaining: 7m 1s\n",
      "170:\ttest: 0.6366978\tbest: 0.6367256 (167)\ttotal: 1m 26s\tremaining: 7m 1s\n",
      "171:\ttest: 0.6367927\tbest: 0.6367927 (171)\ttotal: 1m 27s\tremaining: 7m 1s\n",
      "172:\ttest: 0.6367933\tbest: 0.6367933 (172)\ttotal: 1m 27s\tremaining: 7m\n",
      "173:\ttest: 0.6368355\tbest: 0.6368355 (173)\ttotal: 1m 28s\tremaining: 7m\n",
      "174:\ttest: 0.6368057\tbest: 0.6368355 (173)\ttotal: 1m 28s\tremaining: 6m 59s\n",
      "175:\ttest: 0.6368122\tbest: 0.6368355 (173)\ttotal: 1m 29s\tremaining: 6m 59s\n",
      "176:\ttest: 0.6368004\tbest: 0.6368355 (173)\ttotal: 1m 29s\tremaining: 6m 57s\n",
      "177:\ttest: 0.6367920\tbest: 0.6368355 (173)\ttotal: 1m 30s\tremaining: 6m 57s\n",
      "178:\ttest: 0.6367793\tbest: 0.6368355 (173)\ttotal: 1m 30s\tremaining: 6m 56s\n",
      "179:\ttest: 0.6368485\tbest: 0.6368485 (179)\ttotal: 1m 31s\tremaining: 6m 56s\n",
      "180:\ttest: 0.6368868\tbest: 0.6368868 (180)\ttotal: 1m 31s\tremaining: 6m 56s\n",
      "181:\ttest: 0.6369059\tbest: 0.6369059 (181)\ttotal: 1m 32s\tremaining: 6m 55s\n",
      "182:\ttest: 0.6368874\tbest: 0.6369059 (181)\ttotal: 1m 33s\tremaining: 6m 55s\n",
      "183:\ttest: 0.6369015\tbest: 0.6369059 (181)\ttotal: 1m 33s\tremaining: 6m 54s\n",
      "184:\ttest: 0.6368991\tbest: 0.6369059 (181)\ttotal: 1m 33s\tremaining: 6m 54s\n",
      "185:\ttest: 0.6368953\tbest: 0.6369059 (181)\ttotal: 1m 34s\tremaining: 6m 53s\n",
      "186:\ttest: 0.6369009\tbest: 0.6369059 (181)\ttotal: 1m 35s\tremaining: 6m 53s\n",
      "187:\ttest: 0.6369098\tbest: 0.6369098 (187)\ttotal: 1m 35s\tremaining: 6m 52s\n",
      "188:\ttest: 0.6369253\tbest: 0.6369253 (188)\ttotal: 1m 36s\tremaining: 6m 52s\n",
      "189:\ttest: 0.6368624\tbest: 0.6369253 (188)\ttotal: 1m 36s\tremaining: 6m 52s\n",
      "190:\ttest: 0.6368771\tbest: 0.6369253 (188)\ttotal: 1m 37s\tremaining: 6m 51s\n",
      "191:\ttest: 0.6369059\tbest: 0.6369253 (188)\ttotal: 1m 37s\tremaining: 6m 51s\n",
      "192:\ttest: 0.6369113\tbest: 0.6369253 (188)\ttotal: 1m 38s\tremaining: 6m 51s\n",
      "193:\ttest: 0.6369356\tbest: 0.6369356 (193)\ttotal: 1m 38s\tremaining: 6m 50s\n",
      "194:\ttest: 0.6369567\tbest: 0.6369567 (194)\ttotal: 1m 39s\tremaining: 6m 50s\n",
      "195:\ttest: 0.6369263\tbest: 0.6369567 (194)\ttotal: 1m 39s\tremaining: 6m 50s\n",
      "196:\ttest: 0.6368922\tbest: 0.6369567 (194)\ttotal: 1m 40s\tremaining: 6m 49s\n",
      "197:\ttest: 0.6368875\tbest: 0.6369567 (194)\ttotal: 1m 41s\tremaining: 6m 49s\n",
      "198:\ttest: 0.6368621\tbest: 0.6369567 (194)\ttotal: 1m 41s\tremaining: 6m 49s\n",
      "199:\ttest: 0.6369227\tbest: 0.6369567 (194)\ttotal: 1m 42s\tremaining: 6m 48s\n",
      "200:\ttest: 0.6369135\tbest: 0.6369567 (194)\ttotal: 1m 42s\tremaining: 6m 48s\n",
      "201:\ttest: 0.6369910\tbest: 0.6369910 (201)\ttotal: 1m 43s\tremaining: 6m 48s\n",
      "202:\ttest: 0.6370003\tbest: 0.6370003 (202)\ttotal: 1m 43s\tremaining: 6m 47s\n",
      "203:\ttest: 0.6369769\tbest: 0.6370003 (202)\ttotal: 1m 44s\tremaining: 6m 47s\n",
      "204:\ttest: 0.6369905\tbest: 0.6370003 (202)\ttotal: 1m 44s\tremaining: 6m 47s\n",
      "205:\ttest: 0.6369406\tbest: 0.6370003 (202)\ttotal: 1m 45s\tremaining: 6m 46s\n",
      "206:\ttest: 0.6369061\tbest: 0.6370003 (202)\ttotal: 1m 46s\tremaining: 6m 46s\n",
      "207:\ttest: 0.6369061\tbest: 0.6370003 (202)\ttotal: 1m 46s\tremaining: 6m 46s\n",
      "208:\ttest: 0.6368988\tbest: 0.6370003 (202)\ttotal: 1m 47s\tremaining: 6m 45s\n",
      "209:\ttest: 0.6369017\tbest: 0.6370003 (202)\ttotal: 1m 47s\tremaining: 6m 45s\n",
      "210:\ttest: 0.6369204\tbest: 0.6370003 (202)\ttotal: 1m 48s\tremaining: 6m 44s\n",
      "211:\ttest: 0.6369022\tbest: 0.6370003 (202)\ttotal: 1m 48s\tremaining: 6m 43s\n",
      "212:\ttest: 0.6368810\tbest: 0.6370003 (202)\ttotal: 1m 49s\tremaining: 6m 43s\n",
      "213:\ttest: 0.6369006\tbest: 0.6370003 (202)\ttotal: 1m 49s\tremaining: 6m 42s\n",
      "214:\ttest: 0.6369324\tbest: 0.6370003 (202)\ttotal: 1m 50s\tremaining: 6m 42s\n",
      "215:\ttest: 0.6369205\tbest: 0.6370003 (202)\ttotal: 1m 50s\tremaining: 6m 41s\n",
      "216:\ttest: 0.6368854\tbest: 0.6370003 (202)\ttotal: 1m 51s\tremaining: 6m 40s\n",
      "217:\ttest: 0.6369144\tbest: 0.6370003 (202)\ttotal: 1m 51s\tremaining: 6m 40s\n",
      "218:\ttest: 0.6368970\tbest: 0.6370003 (202)\ttotal: 1m 52s\tremaining: 6m 40s\n",
      "219:\ttest: 0.6369031\tbest: 0.6370003 (202)\ttotal: 1m 52s\tremaining: 6m 40s\n",
      "220:\ttest: 0.6369124\tbest: 0.6370003 (202)\ttotal: 1m 53s\tremaining: 6m 39s\n",
      "221:\ttest: 0.6368623\tbest: 0.6370003 (202)\ttotal: 1m 53s\tremaining: 6m 38s\n",
      "222:\ttest: 0.6369258\tbest: 0.6370003 (202)\ttotal: 1m 54s\tremaining: 6m 38s\n",
      "223:\ttest: 0.6369091\tbest: 0.6370003 (202)\ttotal: 1m 54s\tremaining: 6m 37s\n",
      "224:\ttest: 0.6369198\tbest: 0.6370003 (202)\ttotal: 1m 55s\tremaining: 6m 37s\n",
      "225:\ttest: 0.6368996\tbest: 0.6370003 (202)\ttotal: 1m 55s\tremaining: 6m 36s\n",
      "226:\ttest: 0.6368879\tbest: 0.6370003 (202)\ttotal: 1m 56s\tremaining: 6m 36s\n",
      "227:\ttest: 0.6368281\tbest: 0.6370003 (202)\ttotal: 1m 56s\tremaining: 6m 36s\n",
      "228:\ttest: 0.6368338\tbest: 0.6370003 (202)\ttotal: 1m 57s\tremaining: 6m 35s\n",
      "229:\ttest: 0.6368133\tbest: 0.6370003 (202)\ttotal: 1m 58s\tremaining: 6m 35s\n",
      "230:\ttest: 0.6368282\tbest: 0.6370003 (202)\ttotal: 1m 58s\tremaining: 6m 34s\n",
      "231:\ttest: 0.6368386\tbest: 0.6370003 (202)\ttotal: 1m 59s\tremaining: 6m 34s\n",
      "232:\ttest: 0.6368435\tbest: 0.6370003 (202)\ttotal: 1m 59s\tremaining: 6m 34s\n",
      "233:\ttest: 0.6368251\tbest: 0.6370003 (202)\ttotal: 2m\tremaining: 6m 33s\n",
      "234:\ttest: 0.6367833\tbest: 0.6370003 (202)\ttotal: 2m\tremaining: 6m 33s\n",
      "235:\ttest: 0.6367973\tbest: 0.6370003 (202)\ttotal: 2m 1s\tremaining: 6m 32s\n",
      "236:\ttest: 0.6367852\tbest: 0.6370003 (202)\ttotal: 2m 1s\tremaining: 6m 32s\n",
      "237:\ttest: 0.6367781\tbest: 0.6370003 (202)\ttotal: 2m 2s\tremaining: 6m 32s\n",
      "238:\ttest: 0.6367965\tbest: 0.6370003 (202)\ttotal: 2m 2s\tremaining: 6m 31s\n",
      "239:\ttest: 0.6367866\tbest: 0.6370003 (202)\ttotal: 2m 3s\tremaining: 6m 31s\n",
      "240:\ttest: 0.6368051\tbest: 0.6370003 (202)\ttotal: 2m 4s\tremaining: 6m 30s\n",
      "241:\ttest: 0.6367924\tbest: 0.6370003 (202)\ttotal: 2m 4s\tremaining: 6m 30s\n",
      "242:\ttest: 0.6367872\tbest: 0.6370003 (202)\ttotal: 2m 5s\tremaining: 6m 29s\n",
      "243:\ttest: 0.6367955\tbest: 0.6370003 (202)\ttotal: 2m 5s\tremaining: 6m 29s\n",
      "244:\ttest: 0.6367997\tbest: 0.6370003 (202)\ttotal: 2m 6s\tremaining: 6m 29s\n",
      "245:\ttest: 0.6367964\tbest: 0.6370003 (202)\ttotal: 2m 6s\tremaining: 6m 28s\n",
      "246:\ttest: 0.6368062\tbest: 0.6370003 (202)\ttotal: 2m 7s\tremaining: 6m 28s\n",
      "247:\ttest: 0.6368006\tbest: 0.6370003 (202)\ttotal: 2m 8s\tremaining: 6m 28s\n",
      "248:\ttest: 0.6367869\tbest: 0.6370003 (202)\ttotal: 2m 8s\tremaining: 6m 27s\n",
      "249:\ttest: 0.6367732\tbest: 0.6370003 (202)\ttotal: 2m 9s\tremaining: 6m 27s\n",
      "250:\ttest: 0.6367372\tbest: 0.6370003 (202)\ttotal: 2m 9s\tremaining: 6m 27s\n",
      "251:\ttest: 0.6367596\tbest: 0.6370003 (202)\ttotal: 2m 10s\tremaining: 6m 26s\n",
      "252:\ttest: 0.6367690\tbest: 0.6370003 (202)\ttotal: 2m 10s\tremaining: 6m 26s\n",
      "253:\ttest: 0.6367692\tbest: 0.6370003 (202)\ttotal: 2m 11s\tremaining: 6m 26s\n",
      "254:\ttest: 0.6367615\tbest: 0.6370003 (202)\ttotal: 2m 12s\tremaining: 6m 25s\n",
      "255:\ttest: 0.6367751\tbest: 0.6370003 (202)\ttotal: 2m 12s\tremaining: 6m 25s\n",
      "256:\ttest: 0.6367877\tbest: 0.6370003 (202)\ttotal: 2m 13s\tremaining: 6m 24s\n",
      "257:\ttest: 0.6367294\tbest: 0.6370003 (202)\ttotal: 2m 13s\tremaining: 6m 24s\n",
      "258:\ttest: 0.6367864\tbest: 0.6370003 (202)\ttotal: 2m 14s\tremaining: 6m 24s\n",
      "259:\ttest: 0.6368096\tbest: 0.6370003 (202)\ttotal: 2m 14s\tremaining: 6m 23s\n",
      "260:\ttest: 0.6368303\tbest: 0.6370003 (202)\ttotal: 2m 15s\tremaining: 6m 23s\n",
      "261:\ttest: 0.6368356\tbest: 0.6370003 (202)\ttotal: 2m 15s\tremaining: 6m 22s\n",
      "262:\ttest: 0.6368628\tbest: 0.6370003 (202)\ttotal: 2m 16s\tremaining: 6m 22s\n",
      "263:\ttest: 0.6368645\tbest: 0.6370003 (202)\ttotal: 2m 16s\tremaining: 6m 21s\n",
      "264:\ttest: 0.6368776\tbest: 0.6370003 (202)\ttotal: 2m 17s\tremaining: 6m 21s\n",
      "265:\ttest: 0.6368227\tbest: 0.6370003 (202)\ttotal: 2m 17s\tremaining: 6m 20s\n",
      "266:\ttest: 0.6367255\tbest: 0.6370003 (202)\ttotal: 2m 18s\tremaining: 6m 20s\n",
      "267:\ttest: 0.6367776\tbest: 0.6370003 (202)\ttotal: 2m 19s\tremaining: 6m 19s\n",
      "268:\ttest: 0.6368183\tbest: 0.6370003 (202)\ttotal: 2m 19s\tremaining: 6m 19s\n",
      "269:\ttest: 0.6368427\tbest: 0.6370003 (202)\ttotal: 2m 20s\tremaining: 6m 19s\n",
      "270:\ttest: 0.6367913\tbest: 0.6370003 (202)\ttotal: 2m 20s\tremaining: 6m 18s\n",
      "271:\ttest: 0.6367748\tbest: 0.6370003 (202)\ttotal: 2m 21s\tremaining: 6m 18s\n",
      "272:\ttest: 0.6367778\tbest: 0.6370003 (202)\ttotal: 2m 21s\tremaining: 6m 17s\n",
      "273:\ttest: 0.6367908\tbest: 0.6370003 (202)\ttotal: 2m 22s\tremaining: 6m 17s\n",
      "274:\ttest: 0.6367772\tbest: 0.6370003 (202)\ttotal: 2m 22s\tremaining: 6m 16s\n",
      "275:\ttest: 0.6367432\tbest: 0.6370003 (202)\ttotal: 2m 23s\tremaining: 6m 16s\n",
      "276:\ttest: 0.6367591\tbest: 0.6370003 (202)\ttotal: 2m 23s\tremaining: 6m 15s\n",
      "277:\ttest: 0.6367873\tbest: 0.6370003 (202)\ttotal: 2m 24s\tremaining: 6m 15s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "278:\ttest: 0.6367700\tbest: 0.6370003 (202)\ttotal: 2m 24s\tremaining: 6m 14s\n",
      "279:\ttest: 0.6367362\tbest: 0.6370003 (202)\ttotal: 2m 25s\tremaining: 6m 14s\n",
      "280:\ttest: 0.6367099\tbest: 0.6370003 (202)\ttotal: 2m 26s\tremaining: 6m 13s\n",
      "281:\ttest: 0.6367268\tbest: 0.6370003 (202)\ttotal: 2m 26s\tremaining: 6m 13s\n",
      "282:\ttest: 0.6367598\tbest: 0.6370003 (202)\ttotal: 2m 27s\tremaining: 6m 12s\n",
      "283:\ttest: 0.6367525\tbest: 0.6370003 (202)\ttotal: 2m 27s\tremaining: 6m 12s\n",
      "284:\ttest: 0.6367581\tbest: 0.6370003 (202)\ttotal: 2m 28s\tremaining: 6m 11s\n",
      "285:\ttest: 0.6367447\tbest: 0.6370003 (202)\ttotal: 2m 28s\tremaining: 6m 11s\n",
      "286:\ttest: 0.6367358\tbest: 0.6370003 (202)\ttotal: 2m 29s\tremaining: 6m 10s\n",
      "287:\ttest: 0.6367175\tbest: 0.6370003 (202)\ttotal: 2m 29s\tremaining: 6m 10s\n",
      "288:\ttest: 0.6367253\tbest: 0.6370003 (202)\ttotal: 2m 30s\tremaining: 6m 9s\n",
      "289:\ttest: 0.6367079\tbest: 0.6370003 (202)\ttotal: 2m 30s\tremaining: 6m 9s\n",
      "290:\ttest: 0.6367053\tbest: 0.6370003 (202)\ttotal: 2m 31s\tremaining: 6m 9s\n",
      "291:\ttest: 0.6367185\tbest: 0.6370003 (202)\ttotal: 2m 32s\tremaining: 6m 8s\n",
      "292:\ttest: 0.6367318\tbest: 0.6370003 (202)\ttotal: 2m 32s\tremaining: 6m 8s\n",
      "293:\ttest: 0.6366939\tbest: 0.6370003 (202)\ttotal: 2m 33s\tremaining: 6m 7s\n",
      "294:\ttest: 0.6366799\tbest: 0.6370003 (202)\ttotal: 2m 33s\tremaining: 6m 7s\n",
      "295:\ttest: 0.6366555\tbest: 0.6370003 (202)\ttotal: 2m 34s\tremaining: 6m 6s\n",
      "296:\ttest: 0.6366724\tbest: 0.6370003 (202)\ttotal: 2m 34s\tremaining: 6m 5s\n",
      "297:\ttest: 0.6366668\tbest: 0.6370003 (202)\ttotal: 2m 35s\tremaining: 6m 5s\n",
      "298:\ttest: 0.6367079\tbest: 0.6370003 (202)\ttotal: 2m 35s\tremaining: 6m 4s\n",
      "299:\ttest: 0.6366805\tbest: 0.6370003 (202)\ttotal: 2m 36s\tremaining: 6m 4s\n",
      "300:\ttest: 0.6366833\tbest: 0.6370003 (202)\ttotal: 2m 36s\tremaining: 6m 3s\n",
      "301:\ttest: 0.6366631\tbest: 0.6370003 (202)\ttotal: 2m 37s\tremaining: 6m 3s\n",
      "302:\ttest: 0.6366869\tbest: 0.6370003 (202)\ttotal: 2m 37s\tremaining: 6m 3s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.6370003283\n",
      "bestIteration = 202\n",
      "\n",
      "Shrink model to first 203 iterations.\n",
      "err:  0.6370003283067793\n",
      "0:\ttest: 0.5836989\tbest: 0.5836989 (0)\ttotal: 525ms\tremaining: 8m 44s\n",
      "1:\ttest: 0.5883551\tbest: 0.5883551 (1)\ttotal: 988ms\tremaining: 8m 12s\n",
      "2:\ttest: 0.5899899\tbest: 0.5899899 (2)\ttotal: 1.39s\tremaining: 7m 42s\n",
      "3:\ttest: 0.5913681\tbest: 0.5913681 (3)\ttotal: 1.95s\tremaining: 8m 6s\n",
      "4:\ttest: 0.5909990\tbest: 0.5913681 (3)\ttotal: 2.83s\tremaining: 9m 23s\n",
      "5:\ttest: 0.5898538\tbest: 0.5913681 (3)\ttotal: 3.52s\tremaining: 9m 44s\n",
      "6:\ttest: 0.5891902\tbest: 0.5913681 (3)\ttotal: 4.28s\tremaining: 10m 7s\n",
      "7:\ttest: 0.5891803\tbest: 0.5913681 (3)\ttotal: 4.95s\tremaining: 10m 14s\n",
      "8:\ttest: 0.5894012\tbest: 0.5913681 (3)\ttotal: 5.39s\tremaining: 9m 54s\n",
      "9:\ttest: 0.5892248\tbest: 0.5913681 (3)\ttotal: 5.8s\tremaining: 9m 33s\n",
      "10:\ttest: 0.5898736\tbest: 0.5913681 (3)\ttotal: 6.27s\tremaining: 9m 23s\n",
      "11:\ttest: 0.5900634\tbest: 0.5913681 (3)\ttotal: 6.73s\tremaining: 9m 14s\n",
      "12:\ttest: 0.5893256\tbest: 0.5913681 (3)\ttotal: 7.24s\tremaining: 9m 9s\n",
      "13:\ttest: 0.5888257\tbest: 0.5913681 (3)\ttotal: 7.71s\tremaining: 9m 3s\n",
      "14:\ttest: 0.5895719\tbest: 0.5913681 (3)\ttotal: 8.37s\tremaining: 9m 9s\n",
      "15:\ttest: 0.5897626\tbest: 0.5913681 (3)\ttotal: 9.11s\tremaining: 9m 20s\n",
      "16:\ttest: 0.5897396\tbest: 0.5913681 (3)\ttotal: 9.59s\tremaining: 9m 14s\n",
      "17:\ttest: 0.5896205\tbest: 0.5913681 (3)\ttotal: 10.1s\tremaining: 9m 9s\n",
      "18:\ttest: 0.5898301\tbest: 0.5913681 (3)\ttotal: 10.5s\tremaining: 9m 4s\n",
      "19:\ttest: 0.5896935\tbest: 0.5913681 (3)\ttotal: 11.3s\tremaining: 9m 12s\n",
      "20:\ttest: 0.5899039\tbest: 0.5913681 (3)\ttotal: 12.2s\tremaining: 9m 27s\n",
      "21:\ttest: 0.5903810\tbest: 0.5913681 (3)\ttotal: 12.7s\tremaining: 9m 23s\n",
      "22:\ttest: 0.5904766\tbest: 0.5913681 (3)\ttotal: 13.2s\tremaining: 9m 19s\n",
      "23:\ttest: 0.5907512\tbest: 0.5913681 (3)\ttotal: 13.7s\tremaining: 9m 17s\n",
      "24:\ttest: 0.5904921\tbest: 0.5913681 (3)\ttotal: 14.2s\tremaining: 9m 15s\n",
      "25:\ttest: 0.5904957\tbest: 0.5913681 (3)\ttotal: 14.8s\tremaining: 9m 12s\n",
      "26:\ttest: 0.5907690\tbest: 0.5913681 (3)\ttotal: 15.4s\tremaining: 9m 13s\n",
      "27:\ttest: 0.5904632\tbest: 0.5913681 (3)\ttotal: 15.9s\tremaining: 9m 11s\n",
      "28:\ttest: 0.5907965\tbest: 0.5913681 (3)\ttotal: 16.4s\tremaining: 9m 10s\n",
      "29:\ttest: 0.5912837\tbest: 0.5913681 (3)\ttotal: 17s\tremaining: 9m 9s\n",
      "30:\ttest: 0.5912755\tbest: 0.5913681 (3)\ttotal: 17.5s\tremaining: 9m 8s\n",
      "31:\ttest: 0.5915979\tbest: 0.5915979 (31)\ttotal: 18.1s\tremaining: 9m 7s\n",
      "32:\ttest: 0.5916927\tbest: 0.5916927 (32)\ttotal: 18.8s\tremaining: 9m 9s\n",
      "33:\ttest: 0.5917983\tbest: 0.5917983 (33)\ttotal: 19.3s\tremaining: 9m 7s\n",
      "34:\ttest: 0.5916295\tbest: 0.5917983 (33)\ttotal: 19.8s\tremaining: 9m 5s\n",
      "35:\ttest: 0.5919137\tbest: 0.5919137 (35)\ttotal: 20.3s\tremaining: 9m 2s\n",
      "36:\ttest: 0.5919180\tbest: 0.5919180 (36)\ttotal: 20.8s\tremaining: 9m 2s\n",
      "37:\ttest: 0.5918582\tbest: 0.5919180 (36)\ttotal: 21.4s\tremaining: 9m 1s\n",
      "38:\ttest: 0.5917871\tbest: 0.5919180 (36)\ttotal: 21.9s\tremaining: 9m\n",
      "39:\ttest: 0.5917141\tbest: 0.5919180 (36)\ttotal: 22.5s\tremaining: 8m 59s\n",
      "40:\ttest: 0.5916016\tbest: 0.5919180 (36)\ttotal: 23s\tremaining: 8m 58s\n",
      "41:\ttest: 0.5916702\tbest: 0.5919180 (36)\ttotal: 23.5s\tremaining: 8m 56s\n",
      "42:\ttest: 0.5916902\tbest: 0.5919180 (36)\ttotal: 24s\tremaining: 8m 54s\n",
      "43:\ttest: 0.5918001\tbest: 0.5919180 (36)\ttotal: 24.6s\tremaining: 8m 54s\n",
      "44:\ttest: 0.5918889\tbest: 0.5919180 (36)\ttotal: 25.1s\tremaining: 8m 53s\n",
      "45:\ttest: 0.5918837\tbest: 0.5919180 (36)\ttotal: 25.7s\tremaining: 8m 52s\n",
      "46:\ttest: 0.5973983\tbest: 0.5973983 (46)\ttotal: 26.2s\tremaining: 8m 51s\n",
      "47:\ttest: 0.6031120\tbest: 0.6031120 (47)\ttotal: 26.6s\tremaining: 8m 47s\n",
      "48:\ttest: 0.6063614\tbest: 0.6063614 (48)\ttotal: 27s\tremaining: 8m 43s\n",
      "49:\ttest: 0.6103275\tbest: 0.6103275 (49)\ttotal: 27.3s\tremaining: 8m 39s\n",
      "50:\ttest: 0.6132363\tbest: 0.6132363 (50)\ttotal: 27.8s\tremaining: 8m 36s\n",
      "51:\ttest: 0.6148969\tbest: 0.6148969 (51)\ttotal: 28.2s\tremaining: 8m 34s\n",
      "52:\ttest: 0.6173169\tbest: 0.6173169 (52)\ttotal: 28.6s\tremaining: 8m 31s\n",
      "53:\ttest: 0.6190108\tbest: 0.6190108 (53)\ttotal: 29.1s\tremaining: 8m 29s\n",
      "54:\ttest: 0.6204866\tbest: 0.6204866 (54)\ttotal: 29.5s\tremaining: 8m 26s\n",
      "55:\ttest: 0.6210859\tbest: 0.6210859 (55)\ttotal: 29.9s\tremaining: 8m 23s\n",
      "56:\ttest: 0.6220035\tbest: 0.6220035 (56)\ttotal: 30.3s\tremaining: 8m 21s\n",
      "57:\ttest: 0.6225256\tbest: 0.6225256 (57)\ttotal: 30.7s\tremaining: 8m 18s\n",
      "58:\ttest: 0.6232010\tbest: 0.6232010 (58)\ttotal: 31.1s\tremaining: 8m 16s\n",
      "59:\ttest: 0.6238772\tbest: 0.6238772 (59)\ttotal: 31.5s\tremaining: 8m 13s\n",
      "60:\ttest: 0.6244297\tbest: 0.6244297 (60)\ttotal: 32s\tremaining: 8m 12s\n",
      "61:\ttest: 0.6245114\tbest: 0.6245114 (61)\ttotal: 32.4s\tremaining: 8m 10s\n",
      "62:\ttest: 0.6247019\tbest: 0.6247019 (62)\ttotal: 32.8s\tremaining: 8m 8s\n",
      "63:\ttest: 0.6251270\tbest: 0.6251270 (63)\ttotal: 33.3s\tremaining: 8m 6s\n",
      "64:\ttest: 0.6252601\tbest: 0.6252601 (64)\ttotal: 33.8s\tremaining: 8m 5s\n",
      "65:\ttest: 0.6253856\tbest: 0.6253856 (65)\ttotal: 34.3s\tremaining: 8m 4s\n",
      "66:\ttest: 0.6254826\tbest: 0.6254826 (66)\ttotal: 34.7s\tremaining: 8m 3s\n",
      "67:\ttest: 0.6255648\tbest: 0.6255648 (67)\ttotal: 35.2s\tremaining: 8m 2s\n",
      "68:\ttest: 0.6259402\tbest: 0.6259402 (68)\ttotal: 35.6s\tremaining: 8m\n",
      "69:\ttest: 0.6260577\tbest: 0.6260577 (69)\ttotal: 36.1s\tremaining: 7m 59s\n",
      "70:\ttest: 0.6262077\tbest: 0.6262077 (70)\ttotal: 36.7s\tremaining: 7m 59s\n",
      "71:\ttest: 0.6265031\tbest: 0.6265031 (71)\ttotal: 37.2s\tremaining: 7m 59s\n",
      "72:\ttest: 0.6265248\tbest: 0.6265248 (72)\ttotal: 37.6s\tremaining: 7m 57s\n",
      "73:\ttest: 0.6266494\tbest: 0.6266494 (73)\ttotal: 38.2s\tremaining: 7m 57s\n",
      "74:\ttest: 0.6267580\tbest: 0.6267580 (74)\ttotal: 38.6s\tremaining: 7m 56s\n",
      "75:\ttest: 0.6267900\tbest: 0.6267900 (75)\ttotal: 39s\tremaining: 7m 54s\n",
      "76:\ttest: 0.6271553\tbest: 0.6271553 (76)\ttotal: 39.4s\tremaining: 7m 52s\n",
      "77:\ttest: 0.6275011\tbest: 0.6275011 (77)\ttotal: 39.8s\tremaining: 7m 50s\n",
      "78:\ttest: 0.6278459\tbest: 0.6278459 (78)\ttotal: 40.3s\tremaining: 7m 49s\n",
      "79:\ttest: 0.6280769\tbest: 0.6280769 (79)\ttotal: 40.6s\tremaining: 7m 47s\n",
      "80:\ttest: 0.6282056\tbest: 0.6282056 (80)\ttotal: 41s\tremaining: 7m 45s\n",
      "81:\ttest: 0.6281790\tbest: 0.6282056 (80)\ttotal: 41.5s\tremaining: 7m 44s\n",
      "82:\ttest: 0.6283114\tbest: 0.6283114 (82)\ttotal: 41.9s\tremaining: 7m 43s\n",
      "83:\ttest: 0.6283178\tbest: 0.6283178 (83)\ttotal: 42.3s\tremaining: 7m 41s\n",
      "84:\ttest: 0.6284146\tbest: 0.6284146 (84)\ttotal: 42.7s\tremaining: 7m 39s\n",
      "85:\ttest: 0.6284105\tbest: 0.6284146 (84)\ttotal: 43.1s\tremaining: 7m 38s\n",
      "86:\ttest: 0.6285771\tbest: 0.6285771 (86)\ttotal: 43.5s\tremaining: 7m 36s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "87:\ttest: 0.6286023\tbest: 0.6286023 (87)\ttotal: 44s\tremaining: 7m 36s\n",
      "88:\ttest: 0.6286866\tbest: 0.6286866 (88)\ttotal: 44.5s\tremaining: 7m 35s\n",
      "89:\ttest: 0.6287479\tbest: 0.6287479 (89)\ttotal: 45.1s\tremaining: 7m 35s\n",
      "90:\ttest: 0.6287734\tbest: 0.6287734 (90)\ttotal: 45.5s\tremaining: 7m 34s\n",
      "91:\ttest: 0.6288190\tbest: 0.6288190 (91)\ttotal: 46s\tremaining: 7m 34s\n",
      "92:\ttest: 0.6289155\tbest: 0.6289155 (92)\ttotal: 46.6s\tremaining: 7m 34s\n",
      "93:\ttest: 0.6289106\tbest: 0.6289155 (92)\ttotal: 47s\tremaining: 7m 32s\n",
      "94:\ttest: 0.6289795\tbest: 0.6289795 (94)\ttotal: 47.4s\tremaining: 7m 31s\n",
      "95:\ttest: 0.6289552\tbest: 0.6289795 (94)\ttotal: 47.9s\tremaining: 7m 31s\n",
      "96:\ttest: 0.6289987\tbest: 0.6289987 (96)\ttotal: 48.4s\tremaining: 7m 30s\n",
      "97:\ttest: 0.6290462\tbest: 0.6290462 (97)\ttotal: 48.9s\tremaining: 7m 30s\n",
      "98:\ttest: 0.6290996\tbest: 0.6290996 (98)\ttotal: 49.4s\tremaining: 7m 29s\n",
      "99:\ttest: 0.6291599\tbest: 0.6291599 (99)\ttotal: 49.9s\tremaining: 7m 29s\n",
      "100:\ttest: 0.6290820\tbest: 0.6291599 (99)\ttotal: 50.4s\tremaining: 7m 28s\n",
      "101:\ttest: 0.6290917\tbest: 0.6291599 (99)\ttotal: 50.9s\tremaining: 7m 28s\n",
      "102:\ttest: 0.6291367\tbest: 0.6291599 (99)\ttotal: 51.5s\tremaining: 7m 28s\n",
      "103:\ttest: 0.6291767\tbest: 0.6291767 (103)\ttotal: 52s\tremaining: 7m 27s\n",
      "104:\ttest: 0.6291869\tbest: 0.6291869 (104)\ttotal: 52.4s\tremaining: 7m 26s\n",
      "105:\ttest: 0.6292082\tbest: 0.6292082 (105)\ttotal: 52.9s\tremaining: 7m 26s\n",
      "106:\ttest: 0.6292457\tbest: 0.6292457 (106)\ttotal: 53.4s\tremaining: 7m 25s\n",
      "107:\ttest: 0.6294289\tbest: 0.6294289 (107)\ttotal: 54s\tremaining: 7m 25s\n",
      "108:\ttest: 0.6295650\tbest: 0.6295650 (108)\ttotal: 54.4s\tremaining: 7m 24s\n",
      "109:\ttest: 0.6297666\tbest: 0.6297666 (109)\ttotal: 54.9s\tremaining: 7m 24s\n",
      "110:\ttest: 0.6298464\tbest: 0.6298464 (110)\ttotal: 55.4s\tremaining: 7m 23s\n",
      "111:\ttest: 0.6299163\tbest: 0.6299163 (111)\ttotal: 55.9s\tremaining: 7m 23s\n",
      "112:\ttest: 0.6300341\tbest: 0.6300341 (112)\ttotal: 56.2s\tremaining: 7m 21s\n",
      "113:\ttest: 0.6302686\tbest: 0.6302686 (113)\ttotal: 56.7s\tremaining: 7m 20s\n",
      "114:\ttest: 0.6303046\tbest: 0.6303046 (114)\ttotal: 57.3s\tremaining: 7m 20s\n",
      "115:\ttest: 0.6303410\tbest: 0.6303410 (115)\ttotal: 57.8s\tremaining: 7m 20s\n",
      "116:\ttest: 0.6304780\tbest: 0.6304780 (116)\ttotal: 58.4s\tremaining: 7m 21s\n",
      "117:\ttest: 0.6305943\tbest: 0.6305943 (117)\ttotal: 58.8s\tremaining: 7m 19s\n",
      "118:\ttest: 0.6307243\tbest: 0.6307243 (118)\ttotal: 59.3s\tremaining: 7m 19s\n",
      "119:\ttest: 0.6308007\tbest: 0.6308007 (119)\ttotal: 59.8s\tremaining: 7m 18s\n",
      "120:\ttest: 0.6308645\tbest: 0.6308645 (120)\ttotal: 1m\tremaining: 7m 17s\n",
      "121:\ttest: 0.6308992\tbest: 0.6308992 (121)\ttotal: 1m\tremaining: 7m 17s\n",
      "122:\ttest: 0.6309197\tbest: 0.6309197 (122)\ttotal: 1m 1s\tremaining: 7m 17s\n",
      "123:\ttest: 0.6310149\tbest: 0.6310149 (123)\ttotal: 1m 1s\tremaining: 7m 17s\n",
      "124:\ttest: 0.6310703\tbest: 0.6310703 (124)\ttotal: 1m 2s\tremaining: 7m 16s\n",
      "125:\ttest: 0.6310627\tbest: 0.6310703 (124)\ttotal: 1m 3s\tremaining: 7m 17s\n",
      "126:\ttest: 0.6310856\tbest: 0.6310856 (126)\ttotal: 1m 3s\tremaining: 7m 16s\n",
      "127:\ttest: 0.6313140\tbest: 0.6313140 (127)\ttotal: 1m 3s\tremaining: 7m 15s\n",
      "128:\ttest: 0.6313816\tbest: 0.6313816 (128)\ttotal: 1m 4s\tremaining: 7m 15s\n",
      "129:\ttest: 0.6314224\tbest: 0.6314224 (129)\ttotal: 1m 4s\tremaining: 7m 14s\n",
      "130:\ttest: 0.6314406\tbest: 0.6314406 (130)\ttotal: 1m 5s\tremaining: 7m 13s\n",
      "131:\ttest: 0.6314968\tbest: 0.6314968 (131)\ttotal: 1m 5s\tremaining: 7m 13s\n",
      "132:\ttest: 0.6315317\tbest: 0.6315317 (132)\ttotal: 1m 6s\tremaining: 7m 13s\n",
      "133:\ttest: 0.6316631\tbest: 0.6316631 (133)\ttotal: 1m 6s\tremaining: 7m 12s\n",
      "134:\ttest: 0.6317148\tbest: 0.6317148 (134)\ttotal: 1m 7s\tremaining: 7m 11s\n",
      "135:\ttest: 0.6317972\tbest: 0.6317972 (135)\ttotal: 1m 8s\tremaining: 7m 12s\n",
      "136:\ttest: 0.6318284\tbest: 0.6318284 (136)\ttotal: 1m 8s\tremaining: 7m 11s\n",
      "137:\ttest: 0.6318946\tbest: 0.6318946 (137)\ttotal: 1m 9s\tremaining: 7m 11s\n",
      "138:\ttest: 0.6319250\tbest: 0.6319250 (138)\ttotal: 1m 9s\tremaining: 7m 11s\n",
      "139:\ttest: 0.6319260\tbest: 0.6319260 (139)\ttotal: 1m 10s\tremaining: 7m 11s\n",
      "140:\ttest: 0.6319996\tbest: 0.6319996 (140)\ttotal: 1m 10s\tremaining: 7m 10s\n",
      "141:\ttest: 0.6320043\tbest: 0.6320043 (141)\ttotal: 1m 11s\tremaining: 7m 11s\n",
      "142:\ttest: 0.6321122\tbest: 0.6321122 (142)\ttotal: 1m 11s\tremaining: 7m 11s\n",
      "143:\ttest: 0.6320738\tbest: 0.6321122 (142)\ttotal: 1m 12s\tremaining: 7m 10s\n",
      "144:\ttest: 0.6321035\tbest: 0.6321122 (142)\ttotal: 1m 12s\tremaining: 7m 10s\n",
      "145:\ttest: 0.6321012\tbest: 0.6321122 (142)\ttotal: 1m 13s\tremaining: 7m 9s\n",
      "146:\ttest: 0.6321660\tbest: 0.6321660 (146)\ttotal: 1m 14s\tremaining: 7m 9s\n",
      "147:\ttest: 0.6322308\tbest: 0.6322308 (147)\ttotal: 1m 14s\tremaining: 7m 10s\n",
      "148:\ttest: 0.6322313\tbest: 0.6322313 (148)\ttotal: 1m 15s\tremaining: 7m 9s\n",
      "149:\ttest: 0.6322755\tbest: 0.6322755 (149)\ttotal: 1m 15s\tremaining: 7m 10s\n",
      "150:\ttest: 0.6322520\tbest: 0.6322755 (149)\ttotal: 1m 16s\tremaining: 7m 9s\n",
      "151:\ttest: 0.6323739\tbest: 0.6323739 (151)\ttotal: 1m 16s\tremaining: 7m 9s\n",
      "152:\ttest: 0.6323808\tbest: 0.6323808 (152)\ttotal: 1m 17s\tremaining: 7m 9s\n",
      "153:\ttest: 0.6324027\tbest: 0.6324027 (153)\ttotal: 1m 18s\tremaining: 7m 8s\n",
      "154:\ttest: 0.6324712\tbest: 0.6324712 (154)\ttotal: 1m 18s\tremaining: 7m 8s\n",
      "155:\ttest: 0.6324957\tbest: 0.6324957 (155)\ttotal: 1m 19s\tremaining: 7m 7s\n",
      "156:\ttest: 0.6325379\tbest: 0.6325379 (156)\ttotal: 1m 19s\tremaining: 7m 6s\n",
      "157:\ttest: 0.6326372\tbest: 0.6326372 (157)\ttotal: 1m 20s\tremaining: 7m 6s\n",
      "158:\ttest: 0.6326486\tbest: 0.6326486 (158)\ttotal: 1m 20s\tremaining: 7m 5s\n",
      "159:\ttest: 0.6326790\tbest: 0.6326790 (159)\ttotal: 1m 21s\tremaining: 7m 5s\n",
      "160:\ttest: 0.6326289\tbest: 0.6326790 (159)\ttotal: 1m 21s\tremaining: 7m 4s\n",
      "161:\ttest: 0.6326891\tbest: 0.6326891 (161)\ttotal: 1m 21s\tremaining: 7m 4s\n",
      "162:\ttest: 0.6326958\tbest: 0.6326958 (162)\ttotal: 1m 22s\tremaining: 7m 3s\n",
      "163:\ttest: 0.6326871\tbest: 0.6326958 (162)\ttotal: 1m 23s\tremaining: 7m 3s\n",
      "164:\ttest: 0.6326894\tbest: 0.6326958 (162)\ttotal: 1m 23s\tremaining: 7m 3s\n",
      "165:\ttest: 0.6327234\tbest: 0.6327234 (165)\ttotal: 1m 24s\tremaining: 7m 2s\n",
      "166:\ttest: 0.6327452\tbest: 0.6327452 (166)\ttotal: 1m 24s\tremaining: 7m 2s\n",
      "167:\ttest: 0.6328013\tbest: 0.6328013 (167)\ttotal: 1m 25s\tremaining: 7m 2s\n",
      "168:\ttest: 0.6328228\tbest: 0.6328228 (168)\ttotal: 1m 25s\tremaining: 7m 2s\n",
      "169:\ttest: 0.6328908\tbest: 0.6328908 (169)\ttotal: 1m 26s\tremaining: 7m 1s\n",
      "170:\ttest: 0.6329226\tbest: 0.6329226 (170)\ttotal: 1m 26s\tremaining: 7m 1s\n",
      "171:\ttest: 0.6329695\tbest: 0.6329695 (171)\ttotal: 1m 27s\tremaining: 7m\n",
      "172:\ttest: 0.6330457\tbest: 0.6330457 (172)\ttotal: 1m 28s\tremaining: 7m\n",
      "173:\ttest: 0.6330622\tbest: 0.6330622 (173)\ttotal: 1m 28s\tremaining: 7m\n",
      "174:\ttest: 0.6330269\tbest: 0.6330622 (173)\ttotal: 1m 29s\tremaining: 6m 59s\n",
      "175:\ttest: 0.6330295\tbest: 0.6330622 (173)\ttotal: 1m 29s\tremaining: 6m 59s\n",
      "176:\ttest: 0.6330022\tbest: 0.6330622 (173)\ttotal: 1m 30s\tremaining: 6m 59s\n",
      "177:\ttest: 0.6330120\tbest: 0.6330622 (173)\ttotal: 1m 30s\tremaining: 6m 58s\n",
      "178:\ttest: 0.6330302\tbest: 0.6330622 (173)\ttotal: 1m 31s\tremaining: 6m 58s\n",
      "179:\ttest: 0.6330332\tbest: 0.6330622 (173)\ttotal: 1m 31s\tremaining: 6m 57s\n",
      "180:\ttest: 0.6330289\tbest: 0.6330622 (173)\ttotal: 1m 32s\tremaining: 6m 57s\n",
      "181:\ttest: 0.6330405\tbest: 0.6330622 (173)\ttotal: 1m 32s\tremaining: 6m 57s\n",
      "182:\ttest: 0.6331033\tbest: 0.6331033 (182)\ttotal: 1m 33s\tremaining: 6m 57s\n",
      "183:\ttest: 0.6331077\tbest: 0.6331077 (183)\ttotal: 1m 34s\tremaining: 6m 56s\n",
      "184:\ttest: 0.6331135\tbest: 0.6331135 (184)\ttotal: 1m 34s\tremaining: 6m 56s\n",
      "185:\ttest: 0.6331883\tbest: 0.6331883 (185)\ttotal: 1m 35s\tremaining: 6m 55s\n",
      "186:\ttest: 0.6331799\tbest: 0.6331883 (185)\ttotal: 1m 35s\tremaining: 6m 55s\n",
      "187:\ttest: 0.6332031\tbest: 0.6332031 (187)\ttotal: 1m 36s\tremaining: 6m 54s\n",
      "188:\ttest: 0.6332285\tbest: 0.6332285 (188)\ttotal: 1m 36s\tremaining: 6m 54s\n",
      "189:\ttest: 0.6332478\tbest: 0.6332478 (189)\ttotal: 1m 37s\tremaining: 6m 54s\n",
      "190:\ttest: 0.6332299\tbest: 0.6332478 (189)\ttotal: 1m 37s\tremaining: 6m 53s\n",
      "191:\ttest: 0.6332434\tbest: 0.6332478 (189)\ttotal: 1m 38s\tremaining: 6m 52s\n",
      "192:\ttest: 0.6332907\tbest: 0.6332907 (192)\ttotal: 1m 38s\tremaining: 6m 52s\n",
      "193:\ttest: 0.6333015\tbest: 0.6333015 (193)\ttotal: 1m 39s\tremaining: 6m 52s\n",
      "194:\ttest: 0.6333527\tbest: 0.6333527 (194)\ttotal: 1m 39s\tremaining: 6m 52s\n",
      "195:\ttest: 0.6333591\tbest: 0.6333591 (195)\ttotal: 1m 40s\tremaining: 6m 51s\n",
      "196:\ttest: 0.6333704\tbest: 0.6333704 (196)\ttotal: 1m 40s\tremaining: 6m 50s\n",
      "197:\ttest: 0.6333804\tbest: 0.6333804 (197)\ttotal: 1m 41s\tremaining: 6m 50s\n",
      "198:\ttest: 0.6334220\tbest: 0.6334220 (198)\ttotal: 1m 41s\tremaining: 6m 50s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "199:\ttest: 0.6334101\tbest: 0.6334220 (198)\ttotal: 1m 42s\tremaining: 6m 50s\n",
      "200:\ttest: 0.6334133\tbest: 0.6334220 (198)\ttotal: 1m 43s\tremaining: 6m 49s\n",
      "201:\ttest: 0.6334201\tbest: 0.6334220 (198)\ttotal: 1m 43s\tremaining: 6m 49s\n",
      "202:\ttest: 0.6334234\tbest: 0.6334234 (202)\ttotal: 1m 44s\tremaining: 6m 48s\n",
      "203:\ttest: 0.6334391\tbest: 0.6334391 (203)\ttotal: 1m 44s\tremaining: 6m 48s\n",
      "204:\ttest: 0.6334659\tbest: 0.6334659 (204)\ttotal: 1m 45s\tremaining: 6m 48s\n",
      "205:\ttest: 0.6334569\tbest: 0.6334659 (204)\ttotal: 1m 45s\tremaining: 6m 47s\n",
      "206:\ttest: 0.6334498\tbest: 0.6334659 (204)\ttotal: 1m 46s\tremaining: 6m 47s\n",
      "207:\ttest: 0.6334575\tbest: 0.6334659 (204)\ttotal: 1m 46s\tremaining: 6m 46s\n",
      "208:\ttest: 0.6334319\tbest: 0.6334659 (204)\ttotal: 1m 47s\tremaining: 6m 46s\n",
      "209:\ttest: 0.6335206\tbest: 0.6335206 (209)\ttotal: 1m 47s\tremaining: 6m 45s\n",
      "210:\ttest: 0.6335812\tbest: 0.6335812 (210)\ttotal: 1m 48s\tremaining: 6m 44s\n",
      "211:\ttest: 0.6335942\tbest: 0.6335942 (211)\ttotal: 1m 48s\tremaining: 6m 44s\n",
      "212:\ttest: 0.6336203\tbest: 0.6336203 (212)\ttotal: 1m 49s\tremaining: 6m 43s\n",
      "213:\ttest: 0.6336207\tbest: 0.6336207 (213)\ttotal: 1m 49s\tremaining: 6m 43s\n",
      "214:\ttest: 0.6336812\tbest: 0.6336812 (214)\ttotal: 1m 50s\tremaining: 6m 42s\n",
      "215:\ttest: 0.6336807\tbest: 0.6336812 (214)\ttotal: 1m 50s\tremaining: 6m 42s\n",
      "216:\ttest: 0.6336689\tbest: 0.6336812 (214)\ttotal: 1m 51s\tremaining: 6m 41s\n",
      "217:\ttest: 0.6337152\tbest: 0.6337152 (217)\ttotal: 1m 51s\tremaining: 6m 41s\n",
      "218:\ttest: 0.6337029\tbest: 0.6337152 (217)\ttotal: 1m 52s\tremaining: 6m 40s\n",
      "219:\ttest: 0.6336864\tbest: 0.6337152 (217)\ttotal: 1m 52s\tremaining: 6m 40s\n",
      "220:\ttest: 0.6336894\tbest: 0.6337152 (217)\ttotal: 1m 53s\tremaining: 6m 39s\n",
      "221:\ttest: 0.6337156\tbest: 0.6337156 (221)\ttotal: 1m 53s\tremaining: 6m 38s\n",
      "222:\ttest: 0.6337233\tbest: 0.6337233 (222)\ttotal: 1m 54s\tremaining: 6m 38s\n",
      "223:\ttest: 0.6337368\tbest: 0.6337368 (223)\ttotal: 1m 54s\tremaining: 6m 37s\n",
      "224:\ttest: 0.6337730\tbest: 0.6337730 (224)\ttotal: 1m 55s\tremaining: 6m 36s\n",
      "225:\ttest: 0.6337831\tbest: 0.6337831 (225)\ttotal: 1m 55s\tremaining: 6m 36s\n",
      "226:\ttest: 0.6337727\tbest: 0.6337831 (225)\ttotal: 1m 56s\tremaining: 6m 36s\n",
      "227:\ttest: 0.6337961\tbest: 0.6337961 (227)\ttotal: 1m 56s\tremaining: 6m 35s\n",
      "228:\ttest: 0.6337704\tbest: 0.6337961 (227)\ttotal: 1m 57s\tremaining: 6m 35s\n",
      "229:\ttest: 0.6337708\tbest: 0.6337961 (227)\ttotal: 1m 57s\tremaining: 6m 34s\n",
      "230:\ttest: 0.6338585\tbest: 0.6338585 (230)\ttotal: 1m 58s\tremaining: 6m 34s\n",
      "231:\ttest: 0.6338707\tbest: 0.6338707 (231)\ttotal: 1m 59s\tremaining: 6m 34s\n",
      "232:\ttest: 0.6338997\tbest: 0.6338997 (232)\ttotal: 1m 59s\tremaining: 6m 33s\n",
      "233:\ttest: 0.6338818\tbest: 0.6338997 (232)\ttotal: 2m\tremaining: 6m 33s\n",
      "234:\ttest: 0.6338629\tbest: 0.6338997 (232)\ttotal: 2m\tremaining: 6m 32s\n",
      "235:\ttest: 0.6338482\tbest: 0.6338997 (232)\ttotal: 2m 1s\tremaining: 6m 32s\n",
      "236:\ttest: 0.6338920\tbest: 0.6338997 (232)\ttotal: 2m 1s\tremaining: 6m 31s\n",
      "237:\ttest: 0.6339245\tbest: 0.6339245 (237)\ttotal: 2m 2s\tremaining: 6m 31s\n",
      "238:\ttest: 0.6339555\tbest: 0.6339555 (238)\ttotal: 2m 2s\tremaining: 6m 30s\n",
      "239:\ttest: 0.6339641\tbest: 0.6339641 (239)\ttotal: 2m 3s\tremaining: 6m 30s\n",
      "240:\ttest: 0.6338824\tbest: 0.6339641 (239)\ttotal: 2m 3s\tremaining: 6m 30s\n",
      "241:\ttest: 0.6338844\tbest: 0.6339641 (239)\ttotal: 2m 4s\tremaining: 6m 29s\n",
      "242:\ttest: 0.6339576\tbest: 0.6339641 (239)\ttotal: 2m 4s\tremaining: 6m 29s\n",
      "243:\ttest: 0.6339640\tbest: 0.6339641 (239)\ttotal: 2m 5s\tremaining: 6m 29s\n",
      "244:\ttest: 0.6339666\tbest: 0.6339666 (244)\ttotal: 2m 6s\tremaining: 6m 28s\n",
      "245:\ttest: 0.6339935\tbest: 0.6339935 (245)\ttotal: 2m 6s\tremaining: 6m 28s\n",
      "246:\ttest: 0.6340033\tbest: 0.6340033 (246)\ttotal: 2m 7s\tremaining: 6m 27s\n",
      "247:\ttest: 0.6339971\tbest: 0.6340033 (246)\ttotal: 2m 7s\tremaining: 6m 27s\n",
      "248:\ttest: 0.6340320\tbest: 0.6340320 (248)\ttotal: 2m 8s\tremaining: 6m 26s\n",
      "249:\ttest: 0.6340191\tbest: 0.6340320 (248)\ttotal: 2m 8s\tremaining: 6m 26s\n",
      "250:\ttest: 0.6340319\tbest: 0.6340320 (248)\ttotal: 2m 9s\tremaining: 6m 25s\n",
      "251:\ttest: 0.6341503\tbest: 0.6341503 (251)\ttotal: 2m 9s\tremaining: 6m 24s\n",
      "252:\ttest: 0.6341532\tbest: 0.6341532 (252)\ttotal: 2m 10s\tremaining: 6m 24s\n",
      "253:\ttest: 0.6341563\tbest: 0.6341563 (253)\ttotal: 2m 10s\tremaining: 6m 24s\n",
      "254:\ttest: 0.6341558\tbest: 0.6341563 (253)\ttotal: 2m 11s\tremaining: 6m 23s\n",
      "255:\ttest: 0.6341645\tbest: 0.6341645 (255)\ttotal: 2m 12s\tremaining: 6m 23s\n",
      "256:\ttest: 0.6341632\tbest: 0.6341645 (255)\ttotal: 2m 12s\tremaining: 6m 23s\n",
      "257:\ttest: 0.6341709\tbest: 0.6341709 (257)\ttotal: 2m 13s\tremaining: 6m 22s\n",
      "258:\ttest: 0.6342998\tbest: 0.6342998 (258)\ttotal: 2m 13s\tremaining: 6m 22s\n",
      "259:\ttest: 0.6342909\tbest: 0.6342998 (258)\ttotal: 2m 14s\tremaining: 6m 21s\n",
      "260:\ttest: 0.6342547\tbest: 0.6342998 (258)\ttotal: 2m 14s\tremaining: 6m 21s\n",
      "261:\ttest: 0.6342847\tbest: 0.6342998 (258)\ttotal: 2m 15s\tremaining: 6m 20s\n",
      "262:\ttest: 0.6342608\tbest: 0.6342998 (258)\ttotal: 2m 15s\tremaining: 6m 20s\n",
      "263:\ttest: 0.6341777\tbest: 0.6342998 (258)\ttotal: 2m 16s\tremaining: 6m 20s\n",
      "264:\ttest: 0.6341624\tbest: 0.6342998 (258)\ttotal: 2m 16s\tremaining: 6m 19s\n",
      "265:\ttest: 0.6341707\tbest: 0.6342998 (258)\ttotal: 2m 17s\tremaining: 6m 18s\n",
      "266:\ttest: 0.6341771\tbest: 0.6342998 (258)\ttotal: 2m 17s\tremaining: 6m 18s\n",
      "267:\ttest: 0.6342170\tbest: 0.6342998 (258)\ttotal: 2m 18s\tremaining: 6m 18s\n",
      "268:\ttest: 0.6342404\tbest: 0.6342998 (258)\ttotal: 2m 18s\tremaining: 6m 17s\n",
      "269:\ttest: 0.6342888\tbest: 0.6342998 (258)\ttotal: 2m 19s\tremaining: 6m 17s\n",
      "270:\ttest: 0.6342820\tbest: 0.6342998 (258)\ttotal: 2m 19s\tremaining: 6m 16s\n",
      "271:\ttest: 0.6343001\tbest: 0.6343001 (271)\ttotal: 2m 20s\tremaining: 6m 16s\n",
      "272:\ttest: 0.6343409\tbest: 0.6343409 (272)\ttotal: 2m 21s\tremaining: 6m 15s\n",
      "273:\ttest: 0.6343334\tbest: 0.6343409 (272)\ttotal: 2m 21s\tremaining: 6m 15s\n",
      "274:\ttest: 0.6343199\tbest: 0.6343409 (272)\ttotal: 2m 22s\tremaining: 6m 14s\n",
      "275:\ttest: 0.6342881\tbest: 0.6343409 (272)\ttotal: 2m 22s\tremaining: 6m 14s\n",
      "276:\ttest: 0.6342515\tbest: 0.6343409 (272)\ttotal: 2m 23s\tremaining: 6m 13s\n",
      "277:\ttest: 0.6342321\tbest: 0.6343409 (272)\ttotal: 2m 23s\tremaining: 6m 13s\n",
      "278:\ttest: 0.6342493\tbest: 0.6343409 (272)\ttotal: 2m 24s\tremaining: 6m 13s\n",
      "279:\ttest: 0.6342695\tbest: 0.6343409 (272)\ttotal: 2m 24s\tremaining: 6m 12s\n",
      "280:\ttest: 0.6342614\tbest: 0.6343409 (272)\ttotal: 2m 25s\tremaining: 6m 12s\n",
      "281:\ttest: 0.6342429\tbest: 0.6343409 (272)\ttotal: 2m 26s\tremaining: 6m 11s\n",
      "282:\ttest: 0.6342396\tbest: 0.6343409 (272)\ttotal: 2m 26s\tremaining: 6m 11s\n",
      "283:\ttest: 0.6342357\tbest: 0.6343409 (272)\ttotal: 2m 26s\tremaining: 6m 10s\n",
      "284:\ttest: 0.6342459\tbest: 0.6343409 (272)\ttotal: 2m 27s\tremaining: 6m 10s\n",
      "285:\ttest: 0.6342378\tbest: 0.6343409 (272)\ttotal: 2m 28s\tremaining: 6m 9s\n",
      "286:\ttest: 0.6342354\tbest: 0.6343409 (272)\ttotal: 2m 28s\tremaining: 6m 8s\n",
      "287:\ttest: 0.6342400\tbest: 0.6343409 (272)\ttotal: 2m 28s\tremaining: 6m 8s\n",
      "288:\ttest: 0.6342613\tbest: 0.6343409 (272)\ttotal: 2m 29s\tremaining: 6m 7s\n",
      "289:\ttest: 0.6342918\tbest: 0.6343409 (272)\ttotal: 2m 29s\tremaining: 6m 7s\n",
      "290:\ttest: 0.6342939\tbest: 0.6343409 (272)\ttotal: 2m 30s\tremaining: 6m 6s\n",
      "291:\ttest: 0.6342703\tbest: 0.6343409 (272)\ttotal: 2m 31s\tremaining: 6m 6s\n",
      "292:\ttest: 0.6342674\tbest: 0.6343409 (272)\ttotal: 2m 31s\tremaining: 6m 5s\n",
      "293:\ttest: 0.6342811\tbest: 0.6343409 (272)\ttotal: 2m 32s\tremaining: 6m 5s\n",
      "294:\ttest: 0.6343905\tbest: 0.6343905 (294)\ttotal: 2m 32s\tremaining: 6m 4s\n",
      "295:\ttest: 0.6343847\tbest: 0.6343905 (294)\ttotal: 2m 33s\tremaining: 6m 4s\n",
      "296:\ttest: 0.6343772\tbest: 0.6343905 (294)\ttotal: 2m 33s\tremaining: 6m 4s\n",
      "297:\ttest: 0.6344012\tbest: 0.6344012 (297)\ttotal: 2m 34s\tremaining: 6m 3s\n",
      "298:\ttest: 0.6343978\tbest: 0.6344012 (297)\ttotal: 2m 34s\tremaining: 6m 3s\n",
      "299:\ttest: 0.6344112\tbest: 0.6344112 (299)\ttotal: 2m 35s\tremaining: 6m 2s\n",
      "300:\ttest: 0.6344201\tbest: 0.6344201 (300)\ttotal: 2m 36s\tremaining: 6m 2s\n",
      "301:\ttest: 0.6344734\tbest: 0.6344734 (301)\ttotal: 2m 36s\tremaining: 6m 1s\n",
      "302:\ttest: 0.6344772\tbest: 0.6344772 (302)\ttotal: 2m 37s\tremaining: 6m 1s\n",
      "303:\ttest: 0.6344867\tbest: 0.6344867 (303)\ttotal: 2m 37s\tremaining: 6m\n",
      "304:\ttest: 0.6345353\tbest: 0.6345353 (304)\ttotal: 2m 38s\tremaining: 6m\n",
      "305:\ttest: 0.6345159\tbest: 0.6345353 (304)\ttotal: 2m 38s\tremaining: 5m 59s\n",
      "306:\ttest: 0.6345605\tbest: 0.6345605 (306)\ttotal: 2m 39s\tremaining: 5m 59s\n",
      "307:\ttest: 0.6345819\tbest: 0.6345819 (307)\ttotal: 2m 39s\tremaining: 5m 58s\n",
      "308:\ttest: 0.6345575\tbest: 0.6345819 (307)\ttotal: 2m 40s\tremaining: 5m 57s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "309:\ttest: 0.6345715\tbest: 0.6345819 (307)\ttotal: 2m 40s\tremaining: 5m 57s\n",
      "310:\ttest: 0.6345526\tbest: 0.6345819 (307)\ttotal: 2m 40s\tremaining: 5m 56s\n",
      "311:\ttest: 0.6345376\tbest: 0.6345819 (307)\ttotal: 2m 41s\tremaining: 5m 56s\n",
      "312:\ttest: 0.6345960\tbest: 0.6345960 (312)\ttotal: 2m 42s\tremaining: 5m 55s\n",
      "313:\ttest: 0.6346067\tbest: 0.6346067 (313)\ttotal: 2m 42s\tremaining: 5m 55s\n",
      "314:\ttest: 0.6345844\tbest: 0.6346067 (313)\ttotal: 2m 43s\tremaining: 5m 54s\n",
      "315:\ttest: 0.6345916\tbest: 0.6346067 (313)\ttotal: 2m 43s\tremaining: 5m 54s\n",
      "316:\ttest: 0.6345922\tbest: 0.6346067 (313)\ttotal: 2m 44s\tremaining: 5m 53s\n",
      "317:\ttest: 0.6345905\tbest: 0.6346067 (313)\ttotal: 2m 44s\tremaining: 5m 53s\n",
      "318:\ttest: 0.6346470\tbest: 0.6346470 (318)\ttotal: 2m 45s\tremaining: 5m 52s\n",
      "319:\ttest: 0.6346411\tbest: 0.6346470 (318)\ttotal: 2m 45s\tremaining: 5m 52s\n",
      "320:\ttest: 0.6346484\tbest: 0.6346484 (320)\ttotal: 2m 46s\tremaining: 5m 52s\n",
      "321:\ttest: 0.6346559\tbest: 0.6346559 (321)\ttotal: 2m 46s\tremaining: 5m 51s\n",
      "322:\ttest: 0.6347086\tbest: 0.6347086 (322)\ttotal: 2m 47s\tremaining: 5m 51s\n",
      "323:\ttest: 0.6347037\tbest: 0.6347086 (322)\ttotal: 2m 48s\tremaining: 5m 50s\n",
      "324:\ttest: 0.6346660\tbest: 0.6347086 (322)\ttotal: 2m 48s\tremaining: 5m 50s\n",
      "325:\ttest: 0.6346454\tbest: 0.6347086 (322)\ttotal: 2m 49s\tremaining: 5m 50s\n",
      "326:\ttest: 0.6346542\tbest: 0.6347086 (322)\ttotal: 2m 49s\tremaining: 5m 49s\n",
      "327:\ttest: 0.6346391\tbest: 0.6347086 (322)\ttotal: 2m 50s\tremaining: 5m 49s\n",
      "328:\ttest: 0.6346508\tbest: 0.6347086 (322)\ttotal: 2m 51s\tremaining: 5m 48s\n",
      "329:\ttest: 0.6346470\tbest: 0.6347086 (322)\ttotal: 2m 51s\tremaining: 5m 48s\n",
      "330:\ttest: 0.6346451\tbest: 0.6347086 (322)\ttotal: 2m 51s\tremaining: 5m 47s\n",
      "331:\ttest: 0.6346314\tbest: 0.6347086 (322)\ttotal: 2m 52s\tremaining: 5m 47s\n",
      "332:\ttest: 0.6346434\tbest: 0.6347086 (322)\ttotal: 2m 53s\tremaining: 5m 46s\n",
      "333:\ttest: 0.6346416\tbest: 0.6347086 (322)\ttotal: 2m 53s\tremaining: 5m 46s\n",
      "334:\ttest: 0.6346412\tbest: 0.6347086 (322)\ttotal: 2m 54s\tremaining: 5m 45s\n",
      "335:\ttest: 0.6346252\tbest: 0.6347086 (322)\ttotal: 2m 54s\tremaining: 5m 45s\n",
      "336:\ttest: 0.6346135\tbest: 0.6347086 (322)\ttotal: 2m 55s\tremaining: 5m 44s\n",
      "337:\ttest: 0.6345775\tbest: 0.6347086 (322)\ttotal: 2m 55s\tremaining: 5m 44s\n",
      "338:\ttest: 0.6345294\tbest: 0.6347086 (322)\ttotal: 2m 56s\tremaining: 5m 43s\n",
      "339:\ttest: 0.6345359\tbest: 0.6347086 (322)\ttotal: 2m 56s\tremaining: 5m 43s\n",
      "340:\ttest: 0.6345197\tbest: 0.6347086 (322)\ttotal: 2m 57s\tremaining: 5m 42s\n",
      "341:\ttest: 0.6344758\tbest: 0.6347086 (322)\ttotal: 2m 57s\tremaining: 5m 42s\n",
      "342:\ttest: 0.6345295\tbest: 0.6347086 (322)\ttotal: 2m 58s\tremaining: 5m 41s\n",
      "343:\ttest: 0.6345131\tbest: 0.6347086 (322)\ttotal: 2m 59s\tremaining: 5m 41s\n",
      "344:\ttest: 0.6345337\tbest: 0.6347086 (322)\ttotal: 2m 59s\tremaining: 5m 41s\n",
      "345:\ttest: 0.6345390\tbest: 0.6347086 (322)\ttotal: 3m\tremaining: 5m 40s\n",
      "346:\ttest: 0.6345075\tbest: 0.6347086 (322)\ttotal: 3m\tremaining: 5m 40s\n",
      "347:\ttest: 0.6344850\tbest: 0.6347086 (322)\ttotal: 3m 1s\tremaining: 5m 39s\n",
      "348:\ttest: 0.6344913\tbest: 0.6347086 (322)\ttotal: 3m 1s\tremaining: 5m 39s\n",
      "349:\ttest: 0.6345339\tbest: 0.6347086 (322)\ttotal: 3m 2s\tremaining: 5m 38s\n",
      "350:\ttest: 0.6345214\tbest: 0.6347086 (322)\ttotal: 3m 2s\tremaining: 5m 38s\n",
      "351:\ttest: 0.6345349\tbest: 0.6347086 (322)\ttotal: 3m 3s\tremaining: 5m 37s\n",
      "352:\ttest: 0.6345145\tbest: 0.6347086 (322)\ttotal: 3m 4s\tremaining: 5m 37s\n",
      "353:\ttest: 0.6345357\tbest: 0.6347086 (322)\ttotal: 3m 4s\tremaining: 5m 36s\n",
      "354:\ttest: 0.6345802\tbest: 0.6347086 (322)\ttotal: 3m 5s\tremaining: 5m 36s\n",
      "355:\ttest: 0.6346005\tbest: 0.6347086 (322)\ttotal: 3m 5s\tremaining: 5m 35s\n",
      "356:\ttest: 0.6346483\tbest: 0.6347086 (322)\ttotal: 3m 6s\tremaining: 5m 35s\n",
      "357:\ttest: 0.6346385\tbest: 0.6347086 (322)\ttotal: 3m 6s\tremaining: 5m 34s\n",
      "358:\ttest: 0.6345791\tbest: 0.6347086 (322)\ttotal: 3m 7s\tremaining: 5m 34s\n",
      "359:\ttest: 0.6345699\tbest: 0.6347086 (322)\ttotal: 3m 7s\tremaining: 5m 34s\n",
      "360:\ttest: 0.6345514\tbest: 0.6347086 (322)\ttotal: 3m 8s\tremaining: 5m 33s\n",
      "361:\ttest: 0.6344906\tbest: 0.6347086 (322)\ttotal: 3m 9s\tremaining: 5m 33s\n",
      "362:\ttest: 0.6344852\tbest: 0.6347086 (322)\ttotal: 3m 9s\tremaining: 5m 32s\n",
      "363:\ttest: 0.6344873\tbest: 0.6347086 (322)\ttotal: 3m 10s\tremaining: 5m 32s\n",
      "364:\ttest: 0.6344592\tbest: 0.6347086 (322)\ttotal: 3m 10s\tremaining: 5m 31s\n",
      "365:\ttest: 0.6344795\tbest: 0.6347086 (322)\ttotal: 3m 11s\tremaining: 5m 31s\n",
      "366:\ttest: 0.6345171\tbest: 0.6347086 (322)\ttotal: 3m 11s\tremaining: 5m 30s\n",
      "367:\ttest: 0.6345162\tbest: 0.6347086 (322)\ttotal: 3m 12s\tremaining: 5m 30s\n",
      "368:\ttest: 0.6345127\tbest: 0.6347086 (322)\ttotal: 3m 12s\tremaining: 5m 29s\n",
      "369:\ttest: 0.6345130\tbest: 0.6347086 (322)\ttotal: 3m 13s\tremaining: 5m 29s\n",
      "370:\ttest: 0.6345177\tbest: 0.6347086 (322)\ttotal: 3m 13s\tremaining: 5m 28s\n",
      "371:\ttest: 0.6344796\tbest: 0.6347086 (322)\ttotal: 3m 14s\tremaining: 5m 28s\n",
      "372:\ttest: 0.6344614\tbest: 0.6347086 (322)\ttotal: 3m 14s\tremaining: 5m 27s\n",
      "373:\ttest: 0.6345558\tbest: 0.6347086 (322)\ttotal: 3m 15s\tremaining: 5m 27s\n",
      "374:\ttest: 0.6345374\tbest: 0.6347086 (322)\ttotal: 3m 16s\tremaining: 5m 26s\n",
      "375:\ttest: 0.6345579\tbest: 0.6347086 (322)\ttotal: 3m 16s\tremaining: 5m 26s\n",
      "376:\ttest: 0.6345501\tbest: 0.6347086 (322)\ttotal: 3m 17s\tremaining: 5m 25s\n",
      "377:\ttest: 0.6345612\tbest: 0.6347086 (322)\ttotal: 3m 17s\tremaining: 5m 25s\n",
      "378:\ttest: 0.6345228\tbest: 0.6347086 (322)\ttotal: 3m 18s\tremaining: 5m 24s\n",
      "379:\ttest: 0.6345411\tbest: 0.6347086 (322)\ttotal: 3m 18s\tremaining: 5m 24s\n",
      "380:\ttest: 0.6345783\tbest: 0.6347086 (322)\ttotal: 3m 19s\tremaining: 5m 23s\n",
      "381:\ttest: 0.6345325\tbest: 0.6347086 (322)\ttotal: 3m 19s\tremaining: 5m 23s\n",
      "382:\ttest: 0.6345443\tbest: 0.6347086 (322)\ttotal: 3m 20s\tremaining: 5m 22s\n",
      "383:\ttest: 0.6345593\tbest: 0.6347086 (322)\ttotal: 3m 20s\tremaining: 5m 22s\n",
      "384:\ttest: 0.6345279\tbest: 0.6347086 (322)\ttotal: 3m 21s\tremaining: 5m 21s\n",
      "385:\ttest: 0.6345655\tbest: 0.6347086 (322)\ttotal: 3m 22s\tremaining: 5m 21s\n",
      "386:\ttest: 0.6345487\tbest: 0.6347086 (322)\ttotal: 3m 22s\tremaining: 5m 20s\n",
      "387:\ttest: 0.6345610\tbest: 0.6347086 (322)\ttotal: 3m 23s\tremaining: 5m 20s\n",
      "388:\ttest: 0.6345440\tbest: 0.6347086 (322)\ttotal: 3m 23s\tremaining: 5m 19s\n",
      "389:\ttest: 0.6345569\tbest: 0.6347086 (322)\ttotal: 3m 24s\tremaining: 5m 19s\n",
      "390:\ttest: 0.6345619\tbest: 0.6347086 (322)\ttotal: 3m 24s\tremaining: 5m 18s\n",
      "391:\ttest: 0.6345041\tbest: 0.6347086 (322)\ttotal: 3m 25s\tremaining: 5m 18s\n",
      "392:\ttest: 0.6344471\tbest: 0.6347086 (322)\ttotal: 3m 25s\tremaining: 5m 17s\n",
      "393:\ttest: 0.6344648\tbest: 0.6347086 (322)\ttotal: 3m 26s\tremaining: 5m 17s\n",
      "394:\ttest: 0.6344523\tbest: 0.6347086 (322)\ttotal: 3m 27s\tremaining: 5m 17s\n",
      "395:\ttest: 0.6344362\tbest: 0.6347086 (322)\ttotal: 3m 27s\tremaining: 5m 16s\n",
      "396:\ttest: 0.6344919\tbest: 0.6347086 (322)\ttotal: 3m 28s\tremaining: 5m 16s\n",
      "397:\ttest: 0.6345035\tbest: 0.6347086 (322)\ttotal: 3m 28s\tremaining: 5m 15s\n",
      "398:\ttest: 0.6345292\tbest: 0.6347086 (322)\ttotal: 3m 29s\tremaining: 5m 15s\n",
      "399:\ttest: 0.6345363\tbest: 0.6347086 (322)\ttotal: 3m 29s\tremaining: 5m 14s\n",
      "400:\ttest: 0.6345084\tbest: 0.6347086 (322)\ttotal: 3m 30s\tremaining: 5m 14s\n",
      "401:\ttest: 0.6345032\tbest: 0.6347086 (322)\ttotal: 3m 30s\tremaining: 5m 13s\n",
      "402:\ttest: 0.6344875\tbest: 0.6347086 (322)\ttotal: 3m 31s\tremaining: 5m 13s\n",
      "403:\ttest: 0.6344946\tbest: 0.6347086 (322)\ttotal: 3m 32s\tremaining: 5m 12s\n",
      "404:\ttest: 0.6345281\tbest: 0.6347086 (322)\ttotal: 3m 32s\tremaining: 5m 12s\n",
      "405:\ttest: 0.6345240\tbest: 0.6347086 (322)\ttotal: 3m 33s\tremaining: 5m 12s\n",
      "406:\ttest: 0.6345123\tbest: 0.6347086 (322)\ttotal: 3m 33s\tremaining: 5m 11s\n",
      "407:\ttest: 0.6345458\tbest: 0.6347086 (322)\ttotal: 3m 34s\tremaining: 5m 11s\n",
      "408:\ttest: 0.6345303\tbest: 0.6347086 (322)\ttotal: 3m 34s\tremaining: 5m 10s\n",
      "409:\ttest: 0.6345105\tbest: 0.6347086 (322)\ttotal: 3m 35s\tremaining: 5m 10s\n",
      "410:\ttest: 0.6344887\tbest: 0.6347086 (322)\ttotal: 3m 36s\tremaining: 5m 9s\n",
      "411:\ttest: 0.6344476\tbest: 0.6347086 (322)\ttotal: 3m 36s\tremaining: 5m 9s\n",
      "412:\ttest: 0.6344322\tbest: 0.6347086 (322)\ttotal: 3m 37s\tremaining: 5m 8s\n",
      "413:\ttest: 0.6344912\tbest: 0.6347086 (322)\ttotal: 3m 37s\tremaining: 5m 8s\n",
      "414:\ttest: 0.6344902\tbest: 0.6347086 (322)\ttotal: 3m 38s\tremaining: 5m 7s\n",
      "415:\ttest: 0.6345116\tbest: 0.6347086 (322)\ttotal: 3m 38s\tremaining: 5m 7s\n",
      "416:\ttest: 0.6345260\tbest: 0.6347086 (322)\ttotal: 3m 39s\tremaining: 5m 6s\n",
      "417:\ttest: 0.6345142\tbest: 0.6347086 (322)\ttotal: 3m 39s\tremaining: 5m 6s\n",
      "418:\ttest: 0.6344725\tbest: 0.6347086 (322)\ttotal: 3m 40s\tremaining: 5m 5s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "419:\ttest: 0.6344576\tbest: 0.6347086 (322)\ttotal: 3m 40s\tremaining: 5m 5s\n",
      "420:\ttest: 0.6344495\tbest: 0.6347086 (322)\ttotal: 3m 41s\tremaining: 5m 4s\n",
      "421:\ttest: 0.6344644\tbest: 0.6347086 (322)\ttotal: 3m 42s\tremaining: 5m 4s\n",
      "422:\ttest: 0.6344877\tbest: 0.6347086 (322)\ttotal: 3m 42s\tremaining: 5m 3s\n",
      "Stopped by overfitting detector  (100 iterations wait)\n",
      "\n",
      "bestTest = 0.6347086045\n",
      "bestIteration = 322\n",
      "\n",
      "Shrink model to first 323 iterations.\n",
      "err:  0.6347086045117671\n"
     ]
    }
   ],
   "source": [
    "err=[]\n",
    "y_pred_tot=[]\n",
    "from sklearn.model_selection import KFold,StratifiedKFold\n",
    "fold=StratifiedKFold(n_splits=5,shuffle=True,random_state=1994)\n",
    "for train_index, test_index in fold.split(X,y):\n",
    "    X_train, X_test = X.iloc[train_index], X.iloc[test_index]\n",
    "    y_train, y_test = y[train_index], y[test_index]\n",
    "    cat_model5 = CatBoostClassifier(n_estimators=1000,eval_metric='AUC',random_seed=1994,learning_rate = 0.05,depth = 8)\n",
    "    cat_model5.fit(X_train.values,y_train.values,cat_features=categorical_features_indices,eval_set=(X_test, y_test),\n",
    "        plot=False,early_stopping_rounds=100,use_best_model=True) # early stopping set to 100 to prevent overfitting\n",
    "    preds = cat_model5.predict_proba(X_test)[:,1]\n",
    "    print(\"err: \",roc_auc_score(y_test,preds))\n",
    "    err.append(roc_auc_score(y_test,preds))\n",
    "    p = cat_model5.predict_proba(Xtest)[:,1]\n",
    "    y_pred_tot.append(p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6362664797779944"
      ]
     },
     "execution_count": 252,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "err_avg=np.mean(err,0)\n",
    "err_avg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 233,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.08967777, 0.02732124, 0.02771389, ..., 0.04598054, 0.06758494,\n",
       "       0.04569791])"
      ]
     },
     "execution_count": 233,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_pred=np.mean(y_pred_tot,0)\n",
    "y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 234,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 234,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(y_pred>0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Final submission file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 235,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s=pd.DataFrame({'session_id':test['session_id'],'is_click':y_pred})\n",
    "s.head()\n",
    "# s.to_csv('catboost1.csv',index=False) --main 62 max depth-8 lr-0.1\n",
    "s.to_csv('catboost8_4folds.csv',index=False) #--nope"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As having only one day to score, it was the best that i can give, some conclusions on the way\n",
    "\n",
    "* Best suited Imputing ways can help predict much better\n",
    "* its sad to see that the probabilties did'nt even reach 0.5 for any test data\n",
    "* SMOTE for handling imbalanced data can be used\n",
    "* New features can be extracted from historical log dataset\n",
    "\n",
    "# Thanks for having this competition"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}