Attachment_A.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Attachment A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ***STUDY ON THE INCIDENCE OF TUMOR DIAGNOSIS IN THE PROVINCE OF MODENA***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "References: <br>\n",
    "[1] I NUMERI DEL CANCRO IN ITALIA 2018; Stefania G., Lucia M., Fabrizio N., Maria M. - Ed. 2018. <br>\n",
    "[2] I NUMERI DEL CANCRO IN ITALIA 2019; Stefania G., Massimo R., Fabrizio N., Maria M. - Ed. 2019. <br>\n",
    "[3] I NUMERI DEL CANCRO IN ITALIA 2020; Giordano B., Massimo R., Anna S. - Ed. 2020. <br>\n",
    "[4] I NUMERI DEL CANCRO IN ITALIA 2021; Giordano B., Stegania G., Anna S., Maria M. - Ed. 2021. <br>\n",
    "[5] https://www.tuttitalia.it/emilia-romagna/provincia-di-modena/statistiche/popolazione-eta-sesso-stato-civile-2020/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **1. IMPORT OF THE DATABASE AND CONVERSION INTO DATAFRAME**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import of the \"pandas\" and \"numpy\" packages\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we import the database\n",
    "# saving it as a dataframe\n",
    "df = pd.read_excel('incidence.xlsx')\n",
    "\n",
    "# note: the 'incidence.xlsx' file is not publicly provided for privacy reasons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of diagnoses from July 2018 to June 2021:  58969 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# verification of the total number of diagnoses \n",
    "# from July 2018 to June 2021\n",
    "print('Total number of diagnoses from July 2018 to June 2021:  {} diagnoses'.format(df.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **2. BREAKDOWN INTO PERIODS**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.a Pre-Covid Period from Feb 2019 to Jen 2020"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of diagnoses from February 2019 to January 2020 (pre-covid period):  21288 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# selection of only cases within the period specified for the pre-covid\n",
    "# and creation of a new specific dataframe for pre-covid diagnoses\n",
    "df_pre = df[((df.Year == 2019) & (df.Month != 1)) | ((df.Year == 2020) & (df.Month == 1))]\n",
    "# verification of the total number of diagnoses \n",
    "# from February 2019 to January 2020\n",
    "print('Total number of diagnoses from February 2019 to January 2020 (pre-covid period):  {} diagnoses'.format(df_pre.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.b Post-Covid Period from Feb 2020 to Jen 2021"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of diagnoses from February 2020 to January 2021 (post-covid period):  17538 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# selection of only cases within the period specified for the post-covid\n",
    "# and creation of a new specific dataframe for post-covid diagnoses\n",
    "df_post = df[((df.Year == 2020) & (df.Month != 1)) | ((df.Year == 2021) & (df.Month == 1))]\n",
    "# verification of the total number of diagnoses \n",
    "# from February 2020 to January 2021\n",
    "print('Total number of diagnoses from February 2020 to January 2021 (post-covid period):  {} diagnoses'.format(df_post.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **3. RESTRICTION OF DIAGNOSES OF INTEREST**\n",
    "In this phase we eliminate the cases that are not of interest for the purposes of the research, therefore:\n",
    "- BNP;\n",
    "- NDIS;\n",
    "- MD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we create a function that does the operations automatically\n",
    "def drop_cases(dataFrame):\n",
    "    # elimination of diagnoses of BNP\n",
    "    dataFrame.drop(dataFrame[dataFrame.ISTOLOGIA == 1].index, inplace=True)\n",
    "    # elimination of diagnoses of NDIS\n",
    "    dataFrame.drop(dataFrame[dataFrame.ISTOLOGIA == 2].index, inplace=True)\n",
    "    # elimination of diagnoses of MD\n",
    "    dataFrame.drop(dataFrame[dataFrame.ISTOLOGIA == 4].index, inplace=True)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer diagnoses occurred in the pre-covid period:  9848 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# apply the previously created function \n",
    "# to the pre-covid dataframe\n",
    "drop_cases(df_pre)\n",
    "# verification of the total number of cancer diagnoses \n",
    "# occurred in the pre-covid period\n",
    "print('Total number of cancer diagnoses occurred in the pre-covid period:  {} diagnoses'.format(df_pre.shape[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer diagnoses occurred in the post-covid period:  8195 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# apply the previously created function \n",
    "# to the post-covid dataframe\n",
    "drop_cases(df_post)\n",
    "# verification of the total number of cancer diagnoses \n",
    "# occurred in the post-covid period\n",
    "print('Total number of cancer diagnoses occurred in the post-covid period:  {} diagnoses'.format(df_post.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**note**: the annual incidence is expressed without taking into account cancers arising in the skin excluding melanomas [1-4].\n",
    "In view of the above, in order to compare the (raw) incidence rates obtained from the database, it is necessary to eliminate these cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "icd_melanomi = ['Melanoma maligno', \n",
    "                'Melanoma a diffusione superficiale',\n",
    "                'Melanoma a cellule epitelioidi',\n",
    "                'Melanoma amelanotico',\n",
    "                'Melanoma nodulare',\n",
    "                'Melanoma a cellule fusate']\n",
    "\n",
    "topo_zone = ['CUTE',\n",
    "             'CUTE ED EPPENDICI CUTANEE',\n",
    "             'REGIONI CUTANEE',\n",
    "             'REGIONE CUTANEE']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer diagnoses occurred in the pre-covid period without skin cancers but including melanomas:  6395 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# elimination of cases belonging to the skin regions \n",
    "# and which are not melanomas in the pre-covid period\n",
    "df_pre = df_pre[~(df_pre['Zone_T'].isin(topo_zone) & ~df_pre['Descrizione_M_x'].isin(icd_melanomi))]\n",
    "# verification of the total number of diagnoses of the pre-covid period\n",
    "print('Total number of cancer diagnoses occurred in the pre-covid period without skin cancers but including melanomas:  {} diagnoses'.format(df_pre.shape[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer diagnoses occurred in the post-covid period without skin cancers but including melanomas:  5439 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# elimination of cases belonging to the skin regions \n",
    "# and which are not melanomas in the pre-covid period\n",
    "df_post = df_post[~(df_post['Zone_T'].isin(topo_zone) & ~df_post['Descrizione_M_x'].isin(icd_melanomi))]\n",
    "# verification of the total number of diagnoses of the pre-covid period\n",
    "print('Total number of cancer diagnoses occurred in the post-covid period without skin cancers but including melanomas:  {} diagnoses'.format(df_post.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### **4. ELIMINATION OF DOUBLE DIAGNOSIS**\n",
    "From the analysis of the database it is evident that (on average) the patients received 2 histological diagnoses for each pathology. For this reason, since the interest in this case is the incidence, we eliminate the double cases for single patient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer cases occurred in the pre-covid period:  5316 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# we eliminate double diagnoses for single patients \n",
    "# in the pre-covid period\n",
    "df_pre.drop_duplicates(subset=['COD_PATIENT'], inplace=True)\n",
    "# verification of the total number of cases of the pre-covid period\n",
    "print('Total number of cancer cases occurred in the pre-covid period:  {} diagnoses'.format(df_pre.shape[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total number of cancer cases occurred in the post-covid period:  4515 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# we eliminate double diagnoses for single patients \n",
    "# in the pre-covid period\n",
    "df_post.drop_duplicates(subset=['COD_PATIENT'], inplace=True)\n",
    "# verification of the total number of cases of the pre-covid period\n",
    "print('Total number of cancer cases occurred in the post-covid period:  {} diagnoses'.format(df_post.shape[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. EXPLORATION ESTIMATE OF THE ANNUAL INCIDENCE\n",
    "The purpose of this section is to arrive at the incidence per 100.000 inhabitants of cancer diagnoses in the province of Modena.\n",
    "\n",
    "note: the robustness of the estimate set out below was investigated in more detail in attachment B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "# [5]\n",
    "POP_MODENA_F = 360433\n",
    "POP_MODENA_M = 346686\n",
    "\n",
    "# [3]\n",
    "INCIDENCE_ITALY_2020_F = 512.0\n",
    "INCIDENCE_ITALY_2020_M = 735.5\n",
    "INCIDENCE_ITALY_2019_F = 509.4\n",
    "INCIDENCE_ITALY_2019_M = 730.0\n",
    "\n",
    "# [3]\n",
    "NUMBER_CANCERS_ITALY_2020_F = 181857\n",
    "NUMBER_CANCERS_ITALY_2020_M = 194754\n",
    "\n",
    "# note: the Italian cancer report of 2021 does not show the incidences due to covid-19"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aggregation of data by number of cancers occurring in women or men in the pre and post covid periods in the Province of Modena"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of cancers in women before covid in the Province of Modena\n",
    "CANCERS_YEAR_PRE_COVID_MODENA_F = df_pre[df_pre['SESSO'] == 1].shape[0]\n",
    "# number of cancers in women after covid in the Province of Modena\n",
    "CANCERS_YEAR_PRE_COVID_MODENA_M = df_pre[df_pre['SESSO'] == 2].shape[0]\n",
    "# number of cancers in men before covid in the Province of Modena\n",
    "CANCERS_YEAR_POST_COVID_MODENA_F = df_post[df_post['SESSO'] == 1].shape[0]\n",
    "# number of cancers in men after covid in the Province of Modena\n",
    "CANCERS_YEAR_POST_COVID_MODENA_M = df_post[df_post['SESSO'] == 2].shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculation of the incidences of cancer in women or men in the pre and post covid periods in the Province of Modena"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "# incidence of cancer in women before covid in the Province of Modena\n",
    "CANCER_INCIDENCE_MODENA_PRE_COVID_F = round(CANCERS_YEAR_PRE_COVID_MODENA_F / POP_MODENA_F * 100000, 2)\n",
    "# incidence of cancer in women after covid in the Province of Modena\n",
    "CANCER_INCIDENCE_MODENA_POST_COVID_F = round(CANCERS_YEAR_POST_COVID_MODENA_F / POP_MODENA_F * 100000, 2)\n",
    "# incidence of cancer in man before covid in the Province of Modena\n",
    "CANCER_INCIDENCE_MODENA_PRE_COVID_M = round(CANCERS_YEAR_PRE_COVID_MODENA_M / POP_MODENA_M * 100000, 2)\n",
    "# incidence of cancer in man after covid in the Province of Modena\n",
    "CANCER_INCIDENCE_MODENA_POST_COVID_M = round(CANCERS_YEAR_POST_COVID_MODENA_M / POP_MODENA_M * 100000, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Differences in incidence occurred between the pre and post covid periods in the Province of Modena with respect to the female and male population"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Difference in incidence of cancer in women before and after covid in the Province of Modena:     94.05 diagnoses\n",
      "Difference in incidence of cancer in man before and after covid in the Province of Modena:        133.26 diagnoses\n"
     ]
    }
   ],
   "source": [
    "# difference in incidence of cancer in women before and after covid in the Province of Modena\n",
    "INCIDENCE_DIFFERENCE_F = CANCER_INCIDENCE_MODENA_PRE_COVID_F - CANCER_INCIDENCE_MODENA_POST_COVID_F\n",
    "print('Difference in incidence of cancer in women before and after covid in the Province of Modena:     {} diagnoses'.format(round(INCIDENCE_DIFFERENCE_F, 2)))\n",
    "# difference in incidence of cancer in man before and after covid in the Province of Modena\n",
    "INCIDENCE_DIFFERENCE_M = CANCER_INCIDENCE_MODENA_PRE_COVID_M - CANCER_INCIDENCE_MODENA_POST_COVID_M\n",
    "print('Difference in incidence of cancer in man before and after covid in the Province of Modena:        {} diagnoses'.format(round(INCIDENCE_DIFFERENCE_M, 2)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Percentage differences in incidence occurred between the pre and post covid periods in the Province of Modena with respect to the female and male population"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Percentage difference in incidence of cancer in women before and after covid in the Province of Modena:     13.75 %\n",
      "Percentage difference in incidence of cancer in man before and after covid in the Province of Modena:          16.2 %\n"
     ]
    }
   ],
   "source": [
    "# percentage difference in incidence of cancer in women before and after covid in the Province of Modena\n",
    "PERCENTAGE_INCIDENCE_DIFFERENCE_F = round((1- (CANCER_INCIDENCE_MODENA_POST_COVID_F/CANCER_INCIDENCE_MODENA_PRE_COVID_F))*100, 2)\n",
    "print('Percentage difference in incidence of cancer in women before and after covid in the Province of Modena:     {} %'.format(round(PERCENTAGE_INCIDENCE_DIFFERENCE_F, 2)))\n",
    "# percentage difference in incidence of cancer in man before and after covid in the Province of Modena\n",
    "PERCENTAGE_INCIDENCE_DIFFERENCE_M = round((1- (CANCER_INCIDENCE_MODENA_POST_COVID_M/CANCER_INCIDENCE_MODENA_PRE_COVID_M))*100, 2)\n",
    "print('Percentage difference in incidence of cancer in man before and after covid in the Province of Modena:          {} %'.format(round(PERCENTAGE_INCIDENCE_DIFFERENCE_M, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. NATIONAL SCREENING"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is interesting to project the reductions in incidence obtained at the national level to calculate, indicatively, the number of missed diagnoses compared to national projections"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of missed diagnoses compared to national projections in women:          25005.34 diagnoses\n",
      "Number of missed diagnoses compared to national projections in man:               31550.15 diagnoses\n"
     ]
    }
   ],
   "source": [
    "print('Number of missed diagnoses compared to national projections in women:          {} diagnoses'.format(round(NUMBER_CANCERS_ITALY_2020_F/100*PERCENTAGE_INCIDENCE_DIFFERENCE_F, 2)))\n",
    "print('Number of missed diagnoses compared to national projections in man:               {} diagnoses'.format(round(NUMBER_CANCERS_ITALY_2020_M/100*PERCENTAGE_INCIDENCE_DIFFERENCE_M, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In conclusion, if the trend of the Province of Modena were to be confirmed also at the national level, it is possible to predict, with due caution, a number of missed diagnoses equal to about 25.005 for women and 31.550 for man."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "fd7c2ee5f9f63c0dd6824e5091162d2b06b6559e5cded7eb0da24a68e8e4743a"
  },
  "kernelspec": {
   "display_name": "Python 3.9.0 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}