docs/getting-started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with Xanthus\n",
    "\n",
    "## What is Xanthus?\n",
    "\n",
    "Xanthus is a Neural Recommender package written in Python. It started life as a personal project to take an academic ML paper and translate it into a 'production-ready' software package and to replicate the results of the paper along the way. It uses Tensorflow 2.0 under the hood, and makes extensive use of the Keras API. If you're interested, the original authors of [the paper that inspired this project](https://dl.acm.org/doi/10.1145/3038912.3052569) provided code for their experiments, and this proved valuable when starting this project. \n",
    "\n",
    "However, while it is great that they provided their code, the repository isn't maintained, the code uses an old versions of Keras (and Theano!), it can be a little hard for beginners to get to grips with, and it's very much tailored to produce the results in their paper. All fair enough, they wrote a great paper and published their workings. Admirable stuff. Xanthus aims to make it super easy to get started with the work of building a neural recommendation system, and to scale the techniques in the original paper (hopefully) gracefully with you as the complexity of your applications increase.\n",
    "\n",
    "This notebook will walk you through a basic example of using Xanthus to predict previously unseen movies to a set of users using the classic 'Movielens' recommender dataset. The [original paper](https://dl.acm.org/doi/10.1145/3038912.3052569) tests the architectures in this paper as part of an _implicit_ recommendation problem. You'll find out more about what this means later in the notebook. In the meantime, it is worth remembering that the examples in this notebook make the same assumption.\n",
    "\n",
    "Ready for some code?\n",
    "\n",
    "## Loading a sample dataset\n",
    "\n",
    "Ah, the beginning of a brand new ML problem. You'll need to download the dataset first. You can use the Xanthus `download.movielens` utility to download, unzip and save your Movielens data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xanthus import datasets\n",
    "\n",
    "datasets.movielens.download(version=\"ml-latest-small\", output_dir=\"data\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Time to crack out Pandas and load some CSVs. You know the drill. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "ratings = pd.read_csv(\"data/ml-latest-small/ratings.csv\")\n",
    "movies = pd.read_csv(\"data/ml-latest-small/movies.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the data we've loaded. Here's the movies dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>movieId</th>\n",
       "      <th>title</th>\n",
       "      <th>genres</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>Adventure|Animation|Children|Comedy|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Jumanji (1995)</td>\n",
       "      <td>Adventure|Children|Fantasy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>Comedy|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Waiting to Exhale (1995)</td>\n",
       "      <td>Comedy|Drama|Romance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Father of the Bride Part II (1995)</td>\n",
       "      <td>Comedy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   movieId                               title  \\\n",
       "0        1                    Toy Story (1995)   \n",
       "1        2                      Jumanji (1995)   \n",
       "2        3             Grumpier Old Men (1995)   \n",
       "3        4            Waiting to Exhale (1995)   \n",
       "4        5  Father of the Bride Part II (1995)   \n",
       "\n",
       "                                        genres  \n",
       "0  Adventure|Animation|Children|Comedy|Fantasy  \n",
       "1                   Adventure|Children|Fantasy  \n",
       "2                               Comedy|Romance  \n",
       "3                         Comedy|Drama|Romance  \n",
       "4                                       Comedy  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, you've got the unique identifier for your movies, the title of the movie in human-readable format, and then the column `genres` that has a string containing a set of associated genres for the given movie. Straightforward enough. And hey, that `genres` column might come in handy at some point...\n",
    "\n",
    "On to the `ratings` frame. Here's what is in there:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>userId</th>\n",
       "      <th>movieId</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964982703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964981247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964982224</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>5.0</td>\n",
       "      <td>964983815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>5.0</td>\n",
       "      <td>964982931</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   userId  movieId  rating  timestamp\n",
       "0       1        1     4.0  964982703\n",
       "1       1        3     4.0  964981247\n",
       "2       1        6     4.0  964982224\n",
       "3       1       47     5.0  964983815\n",
       "4       1       50     5.0  964982931"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First up, you've got a `userId` corresponding to the unique user identifier, and you've got the `movieId` corresponding to the unique movie identifier (this maps onto the `movieId` column in the `movies` frame, above). You've also got a `rating` field. This is associated with the user-assigned rating for that movie. Finally, you have the `timestamp` -- the date at which the user rated the movie. For future reference, you can convert from this timestamp to a 'human readable' date with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2000-07-30 19:45:03'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "datetime.fromtimestamp(ratings.iloc[0][\"timestamp\"]).strftime(\"%Y-%m-%d %H:%M:%S\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thats your freebie for the day. Onto getting the data ready for training your recommender model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preparation\n",
    "\n",
    "Xanthus provides a few utilities for getting your recommender up and running. One of the more ubiquitous utilities is the `Dataset` class, and its related `DatasetEncoder` class. At the time of writing, the `Dataset` class assumes your 'ratings' data is in the format `user`, `item`, `rating`. You can rename the sample data to be in this format with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings = ratings.rename(columns={\"userId\": \"user\", \"movieId\": \"item\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, you might find it helpful to re-map the movie IDs (now under the `item` column) to be the `titles` in the `movies` frame. This'll make it easier for you to see what the recommender is recommending! Don't do this for big datasets though -- it can get very expensive very quickly! Anyway, remap the `item` column with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user</th>\n",
       "      <th>item</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964982703</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Grumpier Old Men (1995)</td>\n",
       "      <td>4.0</td>\n",
       "      <td>964981247</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user                     item  rating  timestamp\n",
       "0     1         Toy Story (1995)     4.0  964982703\n",
       "1     1  Grumpier Old Men (1995)     4.0  964981247"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "title_mapping = dict(zip(movies[\"movieId\"], movies[\"title\"]))\n",
    "ratings.loc[:, \"item\"] = ratings[\"item\"].apply(lambda _: title_mapping[_])\n",
    "ratings.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A little more meaningful, eh? For this example, you are going to be looking at _implicit_ recommendations, so should also remove clearly negative rating pairs from the dataset. You can do this with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings = ratings[ratings[\"rating\"] > 3.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Leave one out protocol\n",
    "\n",
    "As with any ML model, it is important to keep a held-out sample of your dataset to evaluate your model's performance. This is naturally important for recommenders too. However, recommenders differ slightly in that we are often interested in the recommender's ability to _rank_ candidate items in order to surface the most relevant content to a user. Ultimately, the essence of recommendation problems is search, and getting relevant items in the top `n` search results is generally the name of the game -- absolute accuracy can often be a secondary consideration.\n",
    "\n",
    "One common way of evaluating the performance of a recommender model is therefore to create a test set by sampling `n` items from each user's `m` interactions (e.g. movie ratings), keeping `m-n` interactions in the training set and putting the 'left out' `n` samples in the test set. The thought process then goes that when evaluating a model on this test set, you should see the model rank the 'held' out samples more highly in the results (i.e. it has started to learn a user's preferences). \n",
    "\n",
    "The 'leave one out' protocol is a specific case of this approach where `n=1`. Concretely, when creating a test set using 'leave one out', you withold a single interaction from each user and put these in your test set. You then place all other interactions in your training set. To get you going, Xanthus provides a utility function called -- funnily enough -- `leave_one_out` under the `evaluate` subpackage. You can import it and use it as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xanthus.evaluate import leave_one_out\n",
    "\n",
    "train_df, test_df = leave_one_out(ratings, shuffle=True, deduplicate=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll notice that there's a couple of things going on here. Firstly, the function returns the input interactions frame (in this case `ratings`) and splits it into the two datasets as expected. Fair enough. We then have two keyword arguments `shuffle` and `deduplicate`. The argument `shuffle` will -- you guessed it -- shuffle your dataset before sampling interactions for your test set. This is set to `True` by default, so it is shown here for the purpose of being explicit. The second argument is `deduplicate`. This does what you might expect too -- it strips any cases where a user interacts with a specific item more than once (i.e. a given user-item pair appears more than once).\n",
    "\n",
    "As discussed above, the `leave_one_out` function is really a specific version of a more general 'leave `n` out' approach to splitting a dataset. There's also other ways you might want to split datasets for recommendation problems. For many of those circumstances, Xanthus provides a more generic `split` function. This was inspired by Azure's [_Recommender Split_](https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/split-data-using-recommender-split#:~:text=The%20Recommender%20Split%20option%20is,user%2Ditem%2Drating%20triples) method in Azure ML Studio. There are a few important tweaks in the Xanthus implementation, so make sure to check out that functions documentation if you're interested.\n",
    "\n",
    "Anyway, time to build some datasets.\n",
    "\n",
    "## Introducing the `Dataset`\n",
    "\n",
    "Like other ML problems, recommendation problems typically need to create encoded representations of a domain in order to be passed into a model for training and evaluation. However, there's a few aspects of recommendation problems that can make this problem particularly fiddly. To help you on your way, Xanthus provides a few utilities, including the `Dataset` class and the `DatasetEncoder` class. These structures are designed to take care of the fiddliness for you. They'll build your input vectors (including with metadata, if you provide it -- more on that later) and sparse matrices as required. You shouldn't need to touch a thing. \n",
    "\n",
    "Here's how it works. First, your 'train' and 'test' datasets are going to need to share the same encodings, right? Otherwise they'll disagree on whether `Batman Forever (1995)` shares the same encoding across the datasets, and that would be a terrible shame. To create your `DatasetEncoder` you can do this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xanthus.datasets.encoder.DatasetEncoder at 0x122228a20>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from xanthus.datasets import DatasetEncoder\n",
    "\n",
    "encoder = DatasetEncoder()\n",
    "encoder.fit(ratings[\"user\"], ratings[\"item\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This encoder will store all of the unique encodings of every user and item in the `ratings` set. Notice that you're passing in the `ratings` set here, as opposed to either train or test. This makes doubly sure you're creating encodings for every user-item pair in the dataset. To check this has worked, you can call the `transform` method on the encoder like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'items': array([1694], dtype=int32)}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encoder.transform(items=[\"Batman Forever (1995)\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The naming conventions on the `DatasetEncoder` are deliberately reminicent of the methods on Scikit-Learn encoders, just to help you along with using them. Now you've got your encoder, you can create your `Dataset` objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xanthus.datasets import Dataset, utils\n",
    "\n",
    "train_ds = Dataset.from_df(train_df, normalize=utils.as_implicit, encoder=encoder)\n",
    "test_ds = Dataset.from_df(test_df, normalize=utils.as_implicit, encoder=encoder)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's unpack what's going on here. The `Dataset` class provides the `from_df` class method for quickly constructing a `Dataset` from a 'raw' Pandas `DataFrame`. You want to create a train and test dataset, hence creating two separate `Dataset` objects using this method. Next, you can see that the `encoder` keyword argument is passed in to the `from_df` method. This ensures that each `Dataset` maintains a reference to the _same_ `DatasetEncoder` to ensure consistency when used. The final argument here is `normalize`. This expects a callable object (e.g. a function) that scales the `rating` column (if provided). In the case of this example, the normalization is simply to treat the ratings as an implicit recommendation problem (i.e. all zero or one). The `utils.as_implicit` function simply sets all ratings to one. Simple enough, eh?\n",
    "\n",
    "And that is it for preparing your datasets for modelling, at least for now. Time for some Neural Networks.\n",
    "\n",
    "## Getting neural\n",
    "\n",
    "With your datasets ready, you can build and fit your model. In the example, the `GeneralizedMatrixFactorization` (or `GMFModel`) is used. If you're not sure what a GMF model is, be sure to check out the original paper, and the GMF class itself in the Xanthus docs. Anyway, here's how you set it up: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xanthus.models import GeneralizedMatrixFactorization as GMFModel\n",
    "\n",
    "model = GMFModel(train_ds.user_dim, train_ds.item_dim, factors=64)\n",
    "model.compile(optimizer=\"adam\", loss=\"binary_crossentropy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So what's going on here? Well, `GMFModel` is a _subclass_ of the Keras `Model` class. Consequently, is shares the same interface. You will initialize your model with specific information (in this case information related to the size of the user and item input vectors and the size of the latent factors you're looking to compute), compile the model with a given loss and optimizer, and then train it. Straightforward enough, eh? In principle, you can use `GMFModel` however you'd use a 'normal' Keras model.\n",
    "\n",
    "You're now ready to fit your model. You can do this with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "5729/5729 [==============================] - 7s 1ms/step - loss: 0.5001\n",
      "Epoch 2/5\n",
      "5729/5729 [==============================] - 7s 1ms/step - loss: 0.3685\n",
      "Epoch 3/5\n",
      "5729/5729 [==============================] - 7s 1ms/step - loss: 0.2969\n",
      "Epoch 4/5\n",
      "5729/5729 [==============================] - 7s 1ms/step - loss: 0.2246\n",
      "Epoch 5/5\n",
      "5729/5729 [==============================] - 7s 1ms/step - loss: 0.1581\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x144717dd8>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# prepare training data\n",
    "users_x, items_x, y = train_ds.to_components(\n",
    "    negative_samples=4\n",
    ")\n",
    "model.fit([users_x, items_x], y, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Remember that (as with any ML model) you'll want to tweak your hyperparameters (e.g. `factors`, regularization, etc.) to optimize your model's performance on your given dataset. The example model here is just a quick un-tuned model to show you the ropes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating the model\n",
    "\n",
    "Now to diagnose how well your model has done. The evaluation protocol here is set up in accordance with the methodology outlined in [the original paper](). To get yourself ready to generate some scores, you'll need to run:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from xanthus.evaluate import create_rankings\n",
    "\n",
    "users, items = create_rankings(\n",
    "    test_ds, train_ds, output_dim=1, n_samples=100, unravel=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, what's going on here? First, you're importing the `create_rankings` function. This implements a sampling approach used be _He et al_ in their work. The idea is that you evaluate your model on the user-item pairs in your test set, and for each 'true' user-item pair, you sample `n_samples` negative instances for that user (i.e. items they haven't interacted with). In the case of the `create_rankings` function, this produces and array of shape `n_users, n_samples + 1`. Concretely, for each user, you'll get an array where the first element is a positive sample (something they _did_ interact with) and `n_samples` negative samples (things they _did not_ interact with). \n",
    "\n",
    "The rationale here is that by having the model rank these `n_samples + 1` items for each user, you'll be able to determine whether your model is learning an effective ranking function -- the positive sample _should_ appear higher in the recommendations than the negative results if the model is doing it's job. Here's how you can rank these sampled items:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "240/240 [==============================] - 0s 540us/step\n"
     ]
    }
   ],
   "source": [
    "from xanthus.models import utils\n",
    "test_users, test_items, _ = test_ds.to_components(shuffle=False)\n",
    "\n",
    "scores = model.predict([users, items], verbose=1, batch_size=256)\n",
    "recommended = utils.reshape_recommended(users.reshape(-1, 1), items.reshape(-1, 1), scores, 10, mode=\"array\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally for the evaluation, you can use the `score` function and the provided `metrics` in the Xanthus `evaluate` subpackage. Here's how you can use them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t-nDCG 0.4719391834962755\n",
      "HR@k 0.7351973684210527\n"
     ]
    }
   ],
   "source": [
    "from xanthus.evaluate import score, metrics\n",
    "\n",
    "print(\"t-nDCG\", score(metrics.truncated_ndcg, test_items, recommended).mean())\n",
    "print(\"HR@k\", score(metrics.precision_at_k, test_items, recommended).mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking okay. Good work. Going into detail on how the metrics presented here work is beyond the scope of this notebook. If you're interested in what is going on here, make sure to check out the docs (docstrings) in the Xanthus package itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The fun bit\n",
    "\n",
    "After all of that, it is time to see what you've won. Exciting times. You can generate recommendations for your users _from unseen items_ by using the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "240/240 [==============================] - 0s 578us/step\n"
     ]
    }
   ],
   "source": [
    "scores = model.predict([users, items], verbose=1, batch_size=256)\n",
    "recommended = utils.reshape_recommended(users.reshape(-1, 1), items.reshape(-1, 1), scores, 10, mode=\"array\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall that the first 'column' in the `items` array corresponds to positive the positive sample for a user. You can skip that here. So now you have a great big array of integers. Not as exciting as you'd hoped? Fair enough. Xanthus provides a utility to convert the outputs of your model predictions into a more readable Pandas `DataFrame`. Specifically, your `DatasetEncoder` has the handy `to_df` method for just this job. Give it a set of _encoded_ users and a list of _encoded_ items for each user, and it'll build you a nice `DataFrame`. Here's how:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>item_0</th>\n",
       "      <th>item_1</th>\n",
       "      <th>item_2</th>\n",
       "      <th>item_3</th>\n",
       "      <th>item_4</th>\n",
       "      <th>item_5</th>\n",
       "      <th>item_6</th>\n",
       "      <th>item_7</th>\n",
       "      <th>item_8</th>\n",
       "      <th>item_9</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Saint, The (1997)</td>\n",
       "      <td>Fisher King, The (1991)</td>\n",
       "      <td>Lost Boys, The (1987)</td>\n",
       "      <td>West Side Story (1961)</td>\n",
       "      <td>Harry Potter and the Sorcerer's Stone (a.k.a. ...</td>\n",
       "      <td>Courage Under Fire (1996)</td>\n",
       "      <td>Thing, The (1982)</td>\n",
       "      <td>Tin Cup (1996)</td>\n",
       "      <td>Mask of Zorro, The (1998)</td>\n",
       "      <td>On Her Majesty's Secret Service (1969)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>Seven (a.k.a. Se7en) (1995)</td>\n",
       "      <td>Django Unchained (2012)</td>\n",
       "      <td>Kung Fury (2015)</td>\n",
       "      <td>There's Something About Mary (1998)</td>\n",
       "      <td>Hanna (2011)</td>\n",
       "      <td>Crash (2004)</td>\n",
       "      <td>The Boss Baby (2017)</td>\n",
       "      <td>Unbreakable (2000)</td>\n",
       "      <td>Finding Dory (2016)</td>\n",
       "      <td>Dr. Horrible's Sing-Along Blog (2008)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Four Weddings and a Funeral (1994)</td>\n",
       "      <td>African Queen, The (1951)</td>\n",
       "      <td>Jeremiah Johnson (1972)</td>\n",
       "      <td>Fantastic Voyage (1966)</td>\n",
       "      <td>Cobra (1986)</td>\n",
       "      <td>Notorious (1946)</td>\n",
       "      <td>Monsoon Wedding (2001)</td>\n",
       "      <td>Heartbreak Ridge (1986)</td>\n",
       "      <td>Miracle on 34th Street (1947)</td>\n",
       "      <td>Raise the Titanic (1980)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Killing Fields, The (1984)</td>\n",
       "      <td>Dracula (Bram Stoker's Dracula) (1992)</td>\n",
       "      <td>Midnight Cowboy (1969)</td>\n",
       "      <td>Pi (1998)</td>\n",
       "      <td>Truman Show, The (1998)</td>\n",
       "      <td>Out of Sight (1998)</td>\n",
       "      <td>Last Emperor, The (1987)</td>\n",
       "      <td>Once Were Warriors (1994)</td>\n",
       "      <td>Everyone Says I Love You (1996)</td>\n",
       "      <td>Nightmare Before Christmas, The (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Harry Potter and the Sorcerer's Stone (a.k.a. ...</td>\n",
       "      <td>This Is Spinal Tap (1984)</td>\n",
       "      <td>Mask, The (1994)</td>\n",
       "      <td>Airplane! (1980)</td>\n",
       "      <td>Friday (1995)</td>\n",
       "      <td>Star Trek IV: The Voyage Home (1986)</td>\n",
       "      <td>Kelly's Heroes (1970)</td>\n",
       "      <td>Inception (2010)</td>\n",
       "      <td>What Dreams May Come (1998)</td>\n",
       "      <td>For Love of the Game (1999)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>Home Alone (1990)</td>\n",
       "      <td>Supercop (Police Story 3: Supercop) (Jing cha ...</td>\n",
       "      <td>Dragonheart (1996)</td>\n",
       "      <td>Funny People (2009)</td>\n",
       "      <td>Kazaam (1996)</td>\n",
       "      <td>Red Dawn (1984)</td>\n",
       "      <td>Patch Adams (1998)</td>\n",
       "      <td>Ruthless People (1986)</td>\n",
       "      <td>Footloose (1984)</td>\n",
       "      <td>Sleepers (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>There's Something About Mary (1998)</td>\n",
       "      <td>Gladiator (2000)</td>\n",
       "      <td>Ferris Bueller's Day Off (1986)</td>\n",
       "      <td>Crimson Tide (1995)</td>\n",
       "      <td>Die Hard: With a Vengeance (1995)</td>\n",
       "      <td>Shakespeare in Love (1998)</td>\n",
       "      <td>Young Frankenstein (1974)</td>\n",
       "      <td>Batman (1989)</td>\n",
       "      <td>Game, The (1997)</td>\n",
       "      <td>Christmas Story, A (1983)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>Firm, The (1993)</td>\n",
       "      <td>Dangerous Minds (1995)</td>\n",
       "      <td>Few Good Men, A (1992)</td>\n",
       "      <td>Who Framed Roger Rabbit? (1988)</td>\n",
       "      <td>Bonnie and Clyde (1967)</td>\n",
       "      <td>Superman (1978)</td>\n",
       "      <td>Carlito's Way (1993)</td>\n",
       "      <td>Rocky III (1982)</td>\n",
       "      <td>Trainspotting (1996)</td>\n",
       "      <td>What's Love Got to Do with It? (1993)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>Cinema Paradiso (Nuovo cinema Paradiso) (1989)</td>\n",
       "      <td>Lord of the Rings: The Fellowship of the Ring,...</td>\n",
       "      <td>Finding Nemo (2003)</td>\n",
       "      <td>Hangover, The (2009)</td>\n",
       "      <td>Running Man, The (1987)</td>\n",
       "      <td>Ben-Hur (1959)</td>\n",
       "      <td>Talented Mr. Ripley, The (1999)</td>\n",
       "      <td>Sliding Doors (1998)</td>\n",
       "      <td>Kagemusha (1980)</td>\n",
       "      <td>Some Like It Hot (1959)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>Young Frankenstein (1974)</td>\n",
       "      <td>Batman &amp; Robin (1997)</td>\n",
       "      <td>Tangled (2010)</td>\n",
       "      <td>Louis C.K.: Hilarious (2010)</td>\n",
       "      <td>Pacific Rim (2013)</td>\n",
       "      <td>Planet of the Apes (2001)</td>\n",
       "      <td>American Pie (1999)</td>\n",
       "      <td>Guardians of the Galaxy (2014)</td>\n",
       "      <td>X-Men (2000)</td>\n",
       "      <td>28 Days Later (2002)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>11</td>\n",
       "      <td>Heat (1995)</td>\n",
       "      <td>Analyze This (1999)</td>\n",
       "      <td>To Wong Foo, Thanks for Everything! Julie Newm...</td>\n",
       "      <td>Mystery, Alaska (1999)</td>\n",
       "      <td>I, Robot (2004)</td>\n",
       "      <td>Invincible (2006)</td>\n",
       "      <td>Gandhi (1982)</td>\n",
       "      <td>Galaxy Quest (1999)</td>\n",
       "      <td>Training Day (2001)</td>\n",
       "      <td>Romy and Michele's High School Reunion (1997)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>12</td>\n",
       "      <td>'burbs, The (1989)</td>\n",
       "      <td>Hercules (1997)</td>\n",
       "      <td>Payback (1999)</td>\n",
       "      <td>Three Men and a Baby (1987)</td>\n",
       "      <td>Enemy of the State (1998)</td>\n",
       "      <td>Top Gun (1986)</td>\n",
       "      <td>White Squall (1996)</td>\n",
       "      <td>Dumbo (1941)</td>\n",
       "      <td>Amistad (1997)</td>\n",
       "      <td>Quiz Show (1994)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>13</td>\n",
       "      <td>Die Hard (1988)</td>\n",
       "      <td>Outbreak (1995)</td>\n",
       "      <td>Zootopia (2016)</td>\n",
       "      <td>I Am Legend (2007)</td>\n",
       "      <td>Kate &amp; Leopold (2001)</td>\n",
       "      <td>Lost in Translation (2003)</td>\n",
       "      <td>Battle Royale (Batoru rowaiaru) (2000)</td>\n",
       "      <td>Mallrats (1995)</td>\n",
       "      <td>Gangs of New York (2002)</td>\n",
       "      <td>Lethal Weapon (1987)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>14</td>\n",
       "      <td>Don Juan DeMarco (1995)</td>\n",
       "      <td>Jungle Book, The (1994)</td>\n",
       "      <td>River Wild, The (1994)</td>\n",
       "      <td>National Treasure (2004)</td>\n",
       "      <td>Super Troopers (2001)</td>\n",
       "      <td>Amistad (1997)</td>\n",
       "      <td>Django Unchained (2012)</td>\n",
       "      <td>Son in Law (1993)</td>\n",
       "      <td>Chocolat (1988)</td>\n",
       "      <td>Doctor Zhivago (1965)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>15</td>\n",
       "      <td>Raiders of the Lost Ark (Indiana Jones and the...</td>\n",
       "      <td>Silence of the Lambs, The (1991)</td>\n",
       "      <td>Hobbit: The Desolation of Smaug, The (2013)</td>\n",
       "      <td>50 First Dates (2004)</td>\n",
       "      <td>Prometheus (2012)</td>\n",
       "      <td>Vertigo (1958)</td>\n",
       "      <td>Fear and Loathing in Las Vegas (1998)</td>\n",
       "      <td>Billy Madison (1995)</td>\n",
       "      <td>Knocked Up (2007)</td>\n",
       "      <td>Eraser (1996)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>16</td>\n",
       "      <td>Life Is Beautiful (La Vita è bella) (1997)</td>\n",
       "      <td>Ed Wood (1994)</td>\n",
       "      <td>Wallace &amp; Gromit: A Close Shave (1995)</td>\n",
       "      <td>Pinocchio (1940)</td>\n",
       "      <td>Fast Times at Ridgemont High (1982)</td>\n",
       "      <td>Corpse Bride (2005)</td>\n",
       "      <td>Basic Instinct (1992)</td>\n",
       "      <td>Billy Elliot (2000)</td>\n",
       "      <td>Before Sunrise (1995)</td>\n",
       "      <td>Gia (1998)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>17</td>\n",
       "      <td>Dr. Strangelove or: How I Learned to Stop Worr...</td>\n",
       "      <td>RoboCop (1987)</td>\n",
       "      <td>The Devil's Advocate (1997)</td>\n",
       "      <td>eXistenZ (1999)</td>\n",
       "      <td>Robin Hood: Men in Tights (1993)</td>\n",
       "      <td>Bourne Supremacy, The (2004)</td>\n",
       "      <td>Blair Witch Project, The (1999)</td>\n",
       "      <td>Sicario (2015)</td>\n",
       "      <td>The Count of Monte Cristo (2002)</td>\n",
       "      <td>Indiana Jones and the Kingdom of the Crystal S...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>18</td>\n",
       "      <td>Run Lola Run (Lola rennt) (1998)</td>\n",
       "      <td>WALL·E (2008)</td>\n",
       "      <td>Fury (2014)</td>\n",
       "      <td>Fish Called Wanda, A (1988)</td>\n",
       "      <td>Peter Pan (1953)</td>\n",
       "      <td>Braveheart (1995)</td>\n",
       "      <td>Scanner Darkly, A (2006)</td>\n",
       "      <td>The Hobbit: The Battle of the Five Armies (2014)</td>\n",
       "      <td>Day After Tomorrow, The (2004)</td>\n",
       "      <td>Cowboy Bebop: The Movie (Cowboy Bebop: Tengoku...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>19</td>\n",
       "      <td>Like Water for Chocolate (Como agua para choco...</td>\n",
       "      <td>Crocodile Dundee (1986)</td>\n",
       "      <td>Speed (1994)</td>\n",
       "      <td>Congo (1995)</td>\n",
       "      <td>Ruthless People (1986)</td>\n",
       "      <td>101 Dalmatians (One Hundred and One Dalmatians...</td>\n",
       "      <td>Pleasantville (1998)</td>\n",
       "      <td>Killing Fields, The (1984)</td>\n",
       "      <td>Shanghai Noon (2000)</td>\n",
       "      <td>*batteries not included (1987)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>20</td>\n",
       "      <td>Dodgeball: A True Underdog Story (2004)</td>\n",
       "      <td>Harry Potter and the Goblet of Fire (2005)</td>\n",
       "      <td>Citizen Kane (1941)</td>\n",
       "      <td>13 Going on 30 (2004)</td>\n",
       "      <td>Star Wars: Episode III - Revenge of the Sith (...</td>\n",
       "      <td>Alien: Resurrection (1997)</td>\n",
       "      <td>Lord of the Rings, The (1978)</td>\n",
       "      <td>Mask, The (1994)</td>\n",
       "      <td>20,000 Leagues Under the Sea (1954)</td>\n",
       "      <td>Over the Hedge (2006)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>21</td>\n",
       "      <td>Gone Girl (2014)</td>\n",
       "      <td>Whiplash (2014)</td>\n",
       "      <td>Grown Ups 2 (2013)</td>\n",
       "      <td>Mercury Rising (1998)</td>\n",
       "      <td>(500) Days of Summer (2009)</td>\n",
       "      <td>Apocalypto (2006)</td>\n",
       "      <td>Red Riding Hood (2011)</td>\n",
       "      <td>Creepshow (1982)</td>\n",
       "      <td>World Is Not Enough, The (1999)</td>\n",
       "      <td>Dear Zachary: A Letter to a Son About His Fath...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>22</td>\n",
       "      <td>Bowling for Columbine (2002)</td>\n",
       "      <td>Memento (2000)</td>\n",
       "      <td>Blow (2001)</td>\n",
       "      <td>Pianist, The (2002)</td>\n",
       "      <td>Star Wars: Episode II - Attack of the Clones (...</td>\n",
       "      <td>Apollo 13 (1995)</td>\n",
       "      <td>Rudy (1993)</td>\n",
       "      <td>Romeo and Juliet (1968)</td>\n",
       "      <td>Beetlejuice (1988)</td>\n",
       "      <td>X-Men: Days of Future Past (2014)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>23</td>\n",
       "      <td>Spirited Away (Sen to Chihiro no kamikakushi) ...</td>\n",
       "      <td>Excalibur (1981)</td>\n",
       "      <td>Once Upon a Time in America (1984)</td>\n",
       "      <td>Austin Powers: The Spy Who Shagged Me (1999)</td>\n",
       "      <td>Jerk, The (1979)</td>\n",
       "      <td>Cat on a Hot Tin Roof (1958)</td>\n",
       "      <td>Wallace &amp; Gromit: The Wrong Trousers (1993)</td>\n",
       "      <td>You Can Count on Me (2000)</td>\n",
       "      <td>Out of Sight (1998)</td>\n",
       "      <td>Harry Potter and the Deathly Hallows: Part 1 (...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>24</td>\n",
       "      <td>Léon: The Professional (a.k.a. The Professiona...</td>\n",
       "      <td>King Kong (2005)</td>\n",
       "      <td>Lethal Weapon 3 (1992)</td>\n",
       "      <td>Star Trek Beyond (2016)</td>\n",
       "      <td>Star Trek: Nemesis (2002)</td>\n",
       "      <td>Road to Perdition (2002)</td>\n",
       "      <td>To Kill a Mockingbird (1962)</td>\n",
       "      <td>A-Team, The (2010)</td>\n",
       "      <td>Home (2015)</td>\n",
       "      <td>Stripes (1981)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>25</td>\n",
       "      <td>Shawshank Redemption, The (1994)</td>\n",
       "      <td>WALL·E (2008)</td>\n",
       "      <td>Walk the Line (2005)</td>\n",
       "      <td>Town, The (2010)</td>\n",
       "      <td>Cars (2006)</td>\n",
       "      <td>Elite Squad: The Enemy Within (Tropa de Elite ...</td>\n",
       "      <td>Mulholland Falls (1996)</td>\n",
       "      <td>Motorcycle Diaries, The (Diarios de motociclet...</td>\n",
       "      <td>Rainmaker, The (1997)</td>\n",
       "      <td>Dallas Buyers Club (2013)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    id                                             item_0  \\\n",
       "0    1                                  Saint, The (1997)   \n",
       "1    2                        Seven (a.k.a. Se7en) (1995)   \n",
       "2    3                 Four Weddings and a Funeral (1994)   \n",
       "3    4                         Killing Fields, The (1984)   \n",
       "4    5  Harry Potter and the Sorcerer's Stone (a.k.a. ...   \n",
       "5    6                                  Home Alone (1990)   \n",
       "6    7                There's Something About Mary (1998)   \n",
       "7    8                                   Firm, The (1993)   \n",
       "8    9     Cinema Paradiso (Nuovo cinema Paradiso) (1989)   \n",
       "9   10                          Young Frankenstein (1974)   \n",
       "10  11                                        Heat (1995)   \n",
       "11  12                                 'burbs, The (1989)   \n",
       "12  13                                    Die Hard (1988)   \n",
       "13  14                            Don Juan DeMarco (1995)   \n",
       "14  15  Raiders of the Lost Ark (Indiana Jones and the...   \n",
       "15  16         Life Is Beautiful (La Vita è bella) (1997)   \n",
       "16  17  Dr. Strangelove or: How I Learned to Stop Worr...   \n",
       "17  18                   Run Lola Run (Lola rennt) (1998)   \n",
       "18  19  Like Water for Chocolate (Como agua para choco...   \n",
       "19  20            Dodgeball: A True Underdog Story (2004)   \n",
       "20  21                                   Gone Girl (2014)   \n",
       "21  22                       Bowling for Columbine (2002)   \n",
       "22  23  Spirited Away (Sen to Chihiro no kamikakushi) ...   \n",
       "23  24  Léon: The Professional (a.k.a. The Professiona...   \n",
       "24  25                   Shawshank Redemption, The (1994)   \n",
       "\n",
       "                                               item_1  \\\n",
       "0                             Fisher King, The (1991)   \n",
       "1                             Django Unchained (2012)   \n",
       "2                           African Queen, The (1951)   \n",
       "3              Dracula (Bram Stoker's Dracula) (1992)   \n",
       "4                           This Is Spinal Tap (1984)   \n",
       "5   Supercop (Police Story 3: Supercop) (Jing cha ...   \n",
       "6                                    Gladiator (2000)   \n",
       "7                              Dangerous Minds (1995)   \n",
       "8   Lord of the Rings: The Fellowship of the Ring,...   \n",
       "9                               Batman & Robin (1997)   \n",
       "10                                Analyze This (1999)   \n",
       "11                                    Hercules (1997)   \n",
       "12                                    Outbreak (1995)   \n",
       "13                            Jungle Book, The (1994)   \n",
       "14                   Silence of the Lambs, The (1991)   \n",
       "15                                     Ed Wood (1994)   \n",
       "16                                     RoboCop (1987)   \n",
       "17                                      WALL·E (2008)   \n",
       "18                            Crocodile Dundee (1986)   \n",
       "19         Harry Potter and the Goblet of Fire (2005)   \n",
       "20                                    Whiplash (2014)   \n",
       "21                                     Memento (2000)   \n",
       "22                                   Excalibur (1981)   \n",
       "23                                   King Kong (2005)   \n",
       "24                                      WALL·E (2008)   \n",
       "\n",
       "                                               item_2  \\\n",
       "0                               Lost Boys, The (1987)   \n",
       "1                                    Kung Fury (2015)   \n",
       "2                             Jeremiah Johnson (1972)   \n",
       "3                              Midnight Cowboy (1969)   \n",
       "4                                    Mask, The (1994)   \n",
       "5                                  Dragonheart (1996)   \n",
       "6                     Ferris Bueller's Day Off (1986)   \n",
       "7                              Few Good Men, A (1992)   \n",
       "8                                 Finding Nemo (2003)   \n",
       "9                                      Tangled (2010)   \n",
       "10  To Wong Foo, Thanks for Everything! Julie Newm...   \n",
       "11                                     Payback (1999)   \n",
       "12                                    Zootopia (2016)   \n",
       "13                             River Wild, The (1994)   \n",
       "14        Hobbit: The Desolation of Smaug, The (2013)   \n",
       "15             Wallace & Gromit: A Close Shave (1995)   \n",
       "16                        The Devil's Advocate (1997)   \n",
       "17                                        Fury (2014)   \n",
       "18                                       Speed (1994)   \n",
       "19                                Citizen Kane (1941)   \n",
       "20                                 Grown Ups 2 (2013)   \n",
       "21                                        Blow (2001)   \n",
       "22                 Once Upon a Time in America (1984)   \n",
       "23                             Lethal Weapon 3 (1992)   \n",
       "24                               Walk the Line (2005)   \n",
       "\n",
       "                                          item_3  \\\n",
       "0                         West Side Story (1961)   \n",
       "1            There's Something About Mary (1998)   \n",
       "2                        Fantastic Voyage (1966)   \n",
       "3                                      Pi (1998)   \n",
       "4                               Airplane! (1980)   \n",
       "5                            Funny People (2009)   \n",
       "6                            Crimson Tide (1995)   \n",
       "7                Who Framed Roger Rabbit? (1988)   \n",
       "8                           Hangover, The (2009)   \n",
       "9                   Louis C.K.: Hilarious (2010)   \n",
       "10                        Mystery, Alaska (1999)   \n",
       "11                   Three Men and a Baby (1987)   \n",
       "12                            I Am Legend (2007)   \n",
       "13                      National Treasure (2004)   \n",
       "14                         50 First Dates (2004)   \n",
       "15                              Pinocchio (1940)   \n",
       "16                               eXistenZ (1999)   \n",
       "17                   Fish Called Wanda, A (1988)   \n",
       "18                                  Congo (1995)   \n",
       "19                         13 Going on 30 (2004)   \n",
       "20                         Mercury Rising (1998)   \n",
       "21                           Pianist, The (2002)   \n",
       "22  Austin Powers: The Spy Who Shagged Me (1999)   \n",
       "23                       Star Trek Beyond (2016)   \n",
       "24                              Town, The (2010)   \n",
       "\n",
       "                                               item_4  \\\n",
       "0   Harry Potter and the Sorcerer's Stone (a.k.a. ...   \n",
       "1                                        Hanna (2011)   \n",
       "2                                        Cobra (1986)   \n",
       "3                             Truman Show, The (1998)   \n",
       "4                                       Friday (1995)   \n",
       "5                                       Kazaam (1996)   \n",
       "6                   Die Hard: With a Vengeance (1995)   \n",
       "7                             Bonnie and Clyde (1967)   \n",
       "8                             Running Man, The (1987)   \n",
       "9                                  Pacific Rim (2013)   \n",
       "10                                    I, Robot (2004)   \n",
       "11                          Enemy of the State (1998)   \n",
       "12                              Kate & Leopold (2001)   \n",
       "13                              Super Troopers (2001)   \n",
       "14                                  Prometheus (2012)   \n",
       "15                Fast Times at Ridgemont High (1982)   \n",
       "16                   Robin Hood: Men in Tights (1993)   \n",
       "17                                   Peter Pan (1953)   \n",
       "18                             Ruthless People (1986)   \n",
       "19  Star Wars: Episode III - Revenge of the Sith (...   \n",
       "20                        (500) Days of Summer (2009)   \n",
       "21  Star Wars: Episode II - Attack of the Clones (...   \n",
       "22                                   Jerk, The (1979)   \n",
       "23                          Star Trek: Nemesis (2002)   \n",
       "24                                        Cars (2006)   \n",
       "\n",
       "                                               item_5  \\\n",
       "0                           Courage Under Fire (1996)   \n",
       "1                                        Crash (2004)   \n",
       "2                                    Notorious (1946)   \n",
       "3                                 Out of Sight (1998)   \n",
       "4                Star Trek IV: The Voyage Home (1986)   \n",
       "5                                     Red Dawn (1984)   \n",
       "6                          Shakespeare in Love (1998)   \n",
       "7                                     Superman (1978)   \n",
       "8                                      Ben-Hur (1959)   \n",
       "9                           Planet of the Apes (2001)   \n",
       "10                                  Invincible (2006)   \n",
       "11                                     Top Gun (1986)   \n",
       "12                         Lost in Translation (2003)   \n",
       "13                                     Amistad (1997)   \n",
       "14                                     Vertigo (1958)   \n",
       "15                                Corpse Bride (2005)   \n",
       "16                       Bourne Supremacy, The (2004)   \n",
       "17                                  Braveheart (1995)   \n",
       "18  101 Dalmatians (One Hundred and One Dalmatians...   \n",
       "19                         Alien: Resurrection (1997)   \n",
       "20                                  Apocalypto (2006)   \n",
       "21                                   Apollo 13 (1995)   \n",
       "22                       Cat on a Hot Tin Roof (1958)   \n",
       "23                           Road to Perdition (2002)   \n",
       "24  Elite Squad: The Enemy Within (Tropa de Elite ...   \n",
       "\n",
       "                                         item_6  \\\n",
       "0                             Thing, The (1982)   \n",
       "1                          The Boss Baby (2017)   \n",
       "2                        Monsoon Wedding (2001)   \n",
       "3                      Last Emperor, The (1987)   \n",
       "4                         Kelly's Heroes (1970)   \n",
       "5                            Patch Adams (1998)   \n",
       "6                     Young Frankenstein (1974)   \n",
       "7                          Carlito's Way (1993)   \n",
       "8               Talented Mr. Ripley, The (1999)   \n",
       "9                           American Pie (1999)   \n",
       "10                                Gandhi (1982)   \n",
       "11                          White Squall (1996)   \n",
       "12       Battle Royale (Batoru rowaiaru) (2000)   \n",
       "13                      Django Unchained (2012)   \n",
       "14        Fear and Loathing in Las Vegas (1998)   \n",
       "15                        Basic Instinct (1992)   \n",
       "16              Blair Witch Project, The (1999)   \n",
       "17                     Scanner Darkly, A (2006)   \n",
       "18                         Pleasantville (1998)   \n",
       "19                Lord of the Rings, The (1978)   \n",
       "20                       Red Riding Hood (2011)   \n",
       "21                                  Rudy (1993)   \n",
       "22  Wallace & Gromit: The Wrong Trousers (1993)   \n",
       "23                 To Kill a Mockingbird (1962)   \n",
       "24                      Mulholland Falls (1996)   \n",
       "\n",
       "                                               item_7  \\\n",
       "0                                      Tin Cup (1996)   \n",
       "1                                  Unbreakable (2000)   \n",
       "2                             Heartbreak Ridge (1986)   \n",
       "3                           Once Were Warriors (1994)   \n",
       "4                                    Inception (2010)   \n",
       "5                              Ruthless People (1986)   \n",
       "6                                       Batman (1989)   \n",
       "7                                    Rocky III (1982)   \n",
       "8                                Sliding Doors (1998)   \n",
       "9                      Guardians of the Galaxy (2014)   \n",
       "10                                Galaxy Quest (1999)   \n",
       "11                                       Dumbo (1941)   \n",
       "12                                    Mallrats (1995)   \n",
       "13                                  Son in Law (1993)   \n",
       "14                               Billy Madison (1995)   \n",
       "15                                Billy Elliot (2000)   \n",
       "16                                     Sicario (2015)   \n",
       "17   The Hobbit: The Battle of the Five Armies (2014)   \n",
       "18                         Killing Fields, The (1984)   \n",
       "19                                   Mask, The (1994)   \n",
       "20                                   Creepshow (1982)   \n",
       "21                            Romeo and Juliet (1968)   \n",
       "22                         You Can Count on Me (2000)   \n",
       "23                                 A-Team, The (2010)   \n",
       "24  Motorcycle Diaries, The (Diarios de motociclet...   \n",
       "\n",
       "                                 item_8  \\\n",
       "0             Mask of Zorro, The (1998)   \n",
       "1                   Finding Dory (2016)   \n",
       "2         Miracle on 34th Street (1947)   \n",
       "3       Everyone Says I Love You (1996)   \n",
       "4           What Dreams May Come (1998)   \n",
       "5                      Footloose (1984)   \n",
       "6                      Game, The (1997)   \n",
       "7                  Trainspotting (1996)   \n",
       "8                      Kagemusha (1980)   \n",
       "9                          X-Men (2000)   \n",
       "10                  Training Day (2001)   \n",
       "11                       Amistad (1997)   \n",
       "12             Gangs of New York (2002)   \n",
       "13                      Chocolat (1988)   \n",
       "14                    Knocked Up (2007)   \n",
       "15                Before Sunrise (1995)   \n",
       "16     The Count of Monte Cristo (2002)   \n",
       "17       Day After Tomorrow, The (2004)   \n",
       "18                 Shanghai Noon (2000)   \n",
       "19  20,000 Leagues Under the Sea (1954)   \n",
       "20      World Is Not Enough, The (1999)   \n",
       "21                   Beetlejuice (1988)   \n",
       "22                  Out of Sight (1998)   \n",
       "23                          Home (2015)   \n",
       "24                Rainmaker, The (1997)   \n",
       "\n",
       "                                               item_9  \n",
       "0              On Her Majesty's Secret Service (1969)  \n",
       "1               Dr. Horrible's Sing-Along Blog (2008)  \n",
       "2                            Raise the Titanic (1980)  \n",
       "3              Nightmare Before Christmas, The (1993)  \n",
       "4                         For Love of the Game (1999)  \n",
       "5                                     Sleepers (1996)  \n",
       "6                           Christmas Story, A (1983)  \n",
       "7               What's Love Got to Do with It? (1993)  \n",
       "8                             Some Like It Hot (1959)  \n",
       "9                                28 Days Later (2002)  \n",
       "10      Romy and Michele's High School Reunion (1997)  \n",
       "11                                   Quiz Show (1994)  \n",
       "12                               Lethal Weapon (1987)  \n",
       "13                              Doctor Zhivago (1965)  \n",
       "14                                      Eraser (1996)  \n",
       "15                                         Gia (1998)  \n",
       "16  Indiana Jones and the Kingdom of the Crystal S...  \n",
       "17  Cowboy Bebop: The Movie (Cowboy Bebop: Tengoku...  \n",
       "18                     *batteries not included (1987)  \n",
       "19                              Over the Hedge (2006)  \n",
       "20  Dear Zachary: A Letter to a Son About His Fath...  \n",
       "21                  X-Men: Days of Future Past (2014)  \n",
       "22  Harry Potter and the Deathly Hallows: Part 1 (...  \n",
       "23                                     Stripes (1981)  \n",
       "24                          Dallas Buyers Club (2013)  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recommended_df = encoder.to_df(test_users.flatten(), recommended)\n",
    "recommended_df.head(25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## That's a wrap\n",
    "\n",
    "And that's it for this example. Be sure to raise any issues you have [on GitHub](https://github.com/markdouthwaite/xanthus), or get in touch [on Twitter](https://twitter.com/MarklDouthwaite)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}