08 - Differential Privacy in Pipelines.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "268e15b0",
   "metadata": {},
   "source": [
    "### Differential Privacy with Spark using Tumult Analytics\n",
    "\n",
    "For this notebook, you'll need to follow the [Tumult installation instructions](https://docs.tmlt.dev/analytics/latest/installation.html) in a separate virtual environment, as the library currently has clashing requirements compared to some of the other libraries in these notebooks.\n",
    "\n",
    "Once you have it installed, you're ready to try out your differential privacy knowledge with Spark, let's go!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "03d739e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "\n",
    "from pyspark.sql import SparkSession\n",
    "from tmlt.analytics.keyset import KeySet\n",
    "from tmlt.analytics.privacy_budget import PureDPBudget\n",
    "from tmlt.analytics.protected_change import AddOneRow\n",
    "from tmlt.analytics.query_builder import QueryBuilder, ColumnType, BinningSpec\n",
    "from tmlt.analytics.session import Session\n",
    "\n",
    "\n",
    "spark = SparkSession.builder.getOrCreate()\n",
    "members_df = spark.read.csv(\"data/members.csv\", header=True, inferSchema=True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "07baa055",
   "metadata": {},
   "outputs": [],
   "source": [
    "session = Session.from_dataframe(\n",
    "    privacy_budget=PureDPBudget(epsilon=1.1),\n",
    "    source_id=\"members\",\n",
    "    dataframe=members_df,\n",
    "    protected_change=AddOneRow(),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fdbefbb2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['id',\n",
       " 'name',\n",
       " 'age',\n",
       " 'gender',\n",
       " 'education_level',\n",
       " 'zip_code',\n",
       " 'books_borrowed',\n",
       " 'favorite_genres',\n",
       " 'date_joined']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "members_df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33907a81",
   "metadata": {},
   "source": [
    "I'm curious if there is a correlation between education_level and books_borrowed. Let's take a look!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed6c5e8d",
   "metadata": {},
   "source": [
    "I first need to build a KeySet with the values I'd like to use... Normally I would use Spark to do this, but I need to get the list of values without looking at the data itself, as this would count towards my budget. Thankfully, we have well documented data, so I was able to get the following list! :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2bdbe680",
   "metadata": {},
   "outputs": [],
   "source": [
    "edu_levels = KeySet.from_dict({\n",
    "    \"education_level\": [\n",
    "        \"up-to-high-school\",\n",
    "        \"high-school-diploma\",\n",
    "        \"bachelors-associate\",\n",
    "        \"masters-degree\",\n",
    "        \"doctorate-professional\",\n",
    "    ]\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d64d73d8",
   "metadata": {},
   "source": [
    "Now I can use the QueryBuilder to group by education and then give an average. Here I am binning the number of books borrowed between 0 and 100."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "0131bb58",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/envs/tumult/lib/python3.8/site-packages/pyspark/sql/pandas/functions.py:394: UserWarning: In Python 3.6+ and Spark 3.0+, it is preferred to specify type hints for pandas UDF instead of specifying pandas UDF type which will be deprecated in the future releases. See SPARK-28264 for more details.\n",
      "  warnings.warn(\n",
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----------------------+----------------------+\n",
      "|education_level       |books_borrowed_average|\n",
      "+----------------------+----------------------+\n",
      "|masters-degree        |19.0265726681128      |\n",
      "|doctorate-professional|19.13195435092725     |\n",
      "|bachelors-associate   |19.177823348469314    |\n",
      "|up-to-high-school     |19.37279031819418     |\n",
      "|high-school-diploma   |19.603978997061514    |\n",
      "+----------------------+----------------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "edu_average_books_query = (\n",
    "    QueryBuilder(\"members\")\n",
    "    .groupby(edu_levels)\n",
    "    .average(\"books_borrowed\", low=0, high=100)\n",
    ")\n",
    "edu_average_books = session.evaluate(\n",
    "    edu_average_books_query,\n",
    "    privacy_budget=PureDPBudget(0.6),\n",
    ")\n",
    "edu_average_books.sort(\"books_borrowed_average\").show(truncate=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a596713",
   "metadata": {},
   "source": [
    "There doesn't seem to be any correlation to find here! I wonder if age might be a better indicator, maybe even connected with an education level?\n",
    "\n",
    "To take a look, I first want to create age groups by binning the age in ranges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b9a24214",
   "metadata": {},
   "outputs": [],
   "source": [
    "age_binspec = BinningSpec([10*i for i in range(0, 11)])\n",
    "\n",
    "age_bin_keys = KeySet.from_dict({\n",
    "    \"age_binned\": age_binspec.bins()\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bff1b4d",
   "metadata": {},
   "source": [
    "Now I can filter and group by age... Here I am singling out those with masters or doctorates and I am using a new bounds for my books borrowed as I think 100 was too high!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "c4782ac7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----------+----------------------+\n",
      "|age_binned|books_borrowed_average|\n",
      "+----------+----------------------+\n",
      "|(10, 20]  |-6.0                  |\n",
      "|(20, 30]  |10.846464646464646    |\n",
      "|(30, 40]  |11.547257876312718    |\n",
      "|(40, 50]  |11.070460704607045    |\n",
      "|(50, 60]  |11.566094100074682    |\n",
      "|(60, 70]  |11.075132275132274    |\n",
      "|(70, 80]  |11.117088607594937    |\n",
      "|(80, 90]  |10.222222222222221    |\n",
      "|(90, 100] |11.0                  |\n",
      "|[0, 10]   |10.0                  |\n",
      "+----------+----------------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "binned_age_with_filter_query = QueryBuilder(\"members\")\\\n",
    "      .filter(\"education_level='masters-degree' or education_level='doctorate-professional'\")\\\n",
    "      .bin_column(\"age\", age_binspec)\\\n",
    "      .groupby(age_bin_keys)\\\n",
    "      .average(\"books_borrowed\", low=0, high=22)\n",
    "\n",
    "session.evaluate(binned_age_with_filter_query, privacy_budget=PureDPBudget(0.4)).show(truncate=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cacc5a8f",
   "metadata": {},
   "source": [
    "Oye! I can see that there is a lot of noise added to some of these columns. What did I do wrong? In this case, I filtered on age and did not take into account that some of the age groups represented would likely be underrepresented in my filter. The likelihood that a 8 year old has a masters degree is quite small...\n",
    "\n",
    "In the future, I might run a query like the following first! Getting an idea for books borrowed by age before filtering... :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7e3aa556",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                \r"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----------+----------------------+\n",
      "|age_binned|books_borrowed_average|\n",
      "+----------+----------------------+\n",
      "|(10, 20]  |11.576746088557112    |\n",
      "|(20, 30]  |11.46492337972726     |\n",
      "|(30, 40]  |11.550365211482928    |\n",
      "|(40, 50]  |11.257424458565685    |\n",
      "|(50, 60]  |11.23477687403825     |\n",
      "|(60, 70]  |11.349001351554287    |\n",
      "|(70, 80]  |11.620332883490779    |\n",
      "|(80, 90]  |10.83838383838384     |\n",
      "|(90, 100] |243.0                 |\n",
      "|[0, 10]   |11.138160325083119    |\n",
      "+----------+----------------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "binned_age_query = QueryBuilder(\"members\")\\\n",
    "    .bin_column(\"age\", age_binspec)\\\n",
    "    .groupby(age_bin_keys)\\\n",
    "    .average(\"books_borrowed\", low=0, high=22)\n",
    "\n",
    "session.evaluate(binned_age_query, privacy_budget=PureDPBudget(0.1)).show(truncate=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8feb256e",
   "metadata": {},
   "source": [
    "Or even just looking at a count...."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6efbb58b",
   "metadata": {},
   "source": [
    "Oh no! I ran out of budget!\n",
    "\n",
    "Good news: [Tumult Labs](https://www.tmlt.dev/) has a bunch of notebooks to try out with this dataset and there is an option to set your budget to inifinity as you play around and get to know the library. That said, when you are using Tumult or any differential privacy library in production, you'll need to first make real decisions on your queries and budget! \n",
    "\n",
    "Take a look at their tutorials and happy privacying!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c51d1dd0",
   "metadata": {},
   "source": [
    "### Challenges\n",
    "\n",
    "- Fix the query so that you get a better result for the books borrowed average.\n",
    "- Use an unlimited privacy budget (`privacy_budget=PureDPBudget(epsilon=float('inf'))`), and investigate the correlations in the dataset further. If you find an interesting one, switch back to a budget and try to show it via matplotlib or seaborn\n",
    "- Go through the [Tumult Analytics Tutorial](https://docs.tmlt.dev/analytics/latest/tutorials/) to try out more features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65344e49",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}