notebooks/cvar_computation.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.distributions as d\n",
    "import torch.nn.functional as F\n",
    "from retail import retail\n",
    "import numpy as np\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this experiment, we define our CVaR at the 5% level: knowing that we're in the 5% worst cases, how much do we expect to waste per item on average? To do so, we weight a linear utility that only caters about waste, not sales or availability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_customers = 2500\n",
    "n_buckets = 4\n",
    "monte_carlo_size = 100\n",
    "store_args= {'assortment_size': 1000, 'bucket_cov': torch.eye(n_buckets)/100, 'seed' : 1066,\n",
    "             'max_stock': 1000, 'forecastVariance' :0., 'horizon': 100, 'lead_time': 1}\n",
    "bucketDist = d.uniform.Uniform(0,1)\n",
    "\n",
    "store_args = {\n",
    "'assortment_size': 1000,\n",
    "    'max_stock': 1000,\n",
    " 'bucket_cov': torch.eye(n_buckets)/100, \n",
    " 'seed' : 1066,\n",
    " 'utility_function': 'linear',\n",
    " # We give a null weight to availability and sales\n",
    " 'utility_weights': {\n",
    "  'alpha': 0., \n",
    "  'beta': 1., \n",
    "  'gamma':0. },\n",
    "'forecastVariance' :0., 'horizon': 100, 'lead_time': 1\n",
    "}\n",
    "# We define our quantile for the CVAR\n",
    "cvar_level = 0.05\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We simply define a computation loop where we operate on our environment and store waste results. We compute the CVaR over trajectories of length 100, over 100 stores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Create the list of the average daily reward for each customer distribution for the chosen policy\n",
    "summed_rewards_policy = []\n",
    "for i in range(monte_carlo_size):\n",
    "    sub_rewards = []\n",
    "    done = False\n",
    "    #Generate the store and its customer repartition throughout the day\n",
    "    torch.manual_seed(i)\n",
    "    sampled = bucketDist.sample((n_buckets,))\n",
    "    sample_bucket_customers = (n_customers*sampled/sampled.sum()).round()\n",
    "    store = retail.StoreEnv(**store_args, bucket_customers = sample_bucket_customers)\n",
    "    while not (done):\n",
    "        #Compute the order according to the policy \n",
    "        customers = sample_bucket_customers.max()\n",
    "        p = store.forecast.squeeze()\n",
    "        std = torch.sqrt(customers*p+(1-p))\n",
    "        order = F.relu(3*std+store.forecast.squeeze()*customers-store.get_full_inventory_position()).round()\n",
    "        # Step the environment and get its observation\n",
    "        obs = store.step(order.numpy())\n",
    "        # Store reward for the specific time step\n",
    "        sub_rewards.append(obs[1])\n",
    "        done = obs[2]\n",
    "    #Append average reward of this customer repartition to the list of rewards\n",
    "    summed_rewards_policy.append(torch.stack(sub_rewards).mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Having stored the results, we simply need to compute the estimator of the 5% quantile (VaR), and the expectation below it (our CVaR)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(-0.4768)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rewards = torch.stack(summed_rewards_policy)\n",
    "# We first obtain the Value-at-risk\n",
    "var = np.quantile(rewards, cvar_level)\n",
    "# We retrieve elements below the var\n",
    "bad_cases = rewards[rewards<var]\n",
    "# Finally, we compute the CVAR:\n",
    "bad_cases.mean()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, we can expect to waste 48 cents of monetary unit (aka € here) per item in the assortment per day, over our defined item distribution for 100-days long trajectories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}