ML0101EN-Clus-K-Means-Customer-Seg-py-v1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src = \"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width = 400, align = \"center\"></a>\n",
    "\n",
    "# <center>K-Means Clustering</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Introduction\n",
    "\n",
    "There are many models for **clustering** out there. In this notebook, we will be presenting the model that is considered the one of the simplest model among them. Despite its simplicity, the **K-means** is vastly used for clustering in many data science applications, especially useful if you need to quickly discover insights from **unlabeled data**. In this notebook, you learn how to use k-Means for customer segmentation.\n",
    "\n",
    "Some real-world applications of k-means:\n",
    "- Customer segmentation\n",
    "- Understand what the visitors of a website are trying to accomplish\n",
    "- Pattern recognition\n",
    "- Machine learning\n",
    "- Data compression\n",
    "\n",
    "\n",
    "In this notebook we practice k-means clustering with 2 examples:\n",
    "- k-means on a random generated dataset\n",
    "- Using k-means for customer segmentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Import libraries\n",
    "Lets first import the required libraries.\n",
    "Also run <b> %matplotlib inline </b> since we will be plotting in this section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import random \n",
    "import numpy as np \n",
    "import matplotlib.pyplot as plt \n",
    "from sklearn.cluster import KMeans \n",
    "from sklearn.datasets.samples_generator import make_blobs \n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# k-Means on a randomly generated dataset\n",
    "Lets create our own dataset for this lab!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "First we need to set up a random seed. Use <b>numpy's random.seed()</b> function, where the seed will be set to <b>0</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "np.random.seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Next we will be making <i> random clusters </i> of points by using the <b> make_blobs </b> class. The <b> make_blobs </b> class can take in many inputs, but we will be using these specific ones. <br> <br>\n",
    "<b> <u> Input </u> </b>\n",
    "<ul>\n",
    "    <li> <b>n_samples</b>: The total number of points equally divided among clusters. </li>\n",
    "    <ul> <li> Value will be: 5000 </li> </ul>\n",
    "    <li> <b>centers</b>: The number of centers to generate, or the fixed center locations. </li>\n",
    "    <ul> <li> Value will be: [[4, 4], [-2, -1], [2, -3],[1,1]] </li> </ul>\n",
    "    <li> <b>cluster_std</b>: The standard deviation of the clusters. </li>\n",
    "    <ul> <li> Value will be: 0.9 </li> </ul>\n",
    "</ul>\n",
    "<br>\n",
    "<b> <u> Output </u> </b>\n",
    "<ul>\n",
    "    <li> <b>X</b>: Array of shape [n_samples, n_features]. (Feature Matrix)</li>\n",
    "    <ul> <li> The generated samples. </li> </ul> \n",
    "    <li> <b>y</b>: Array of shape [n_samples]. (Response Vector)</li>\n",
    "    <ul> <li> The integer labels for cluster membership of each sample. </li> </ul>\n",
    "</ul>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "X, y = make_blobs(n_samples=5000, centers=[[4,4], [-2, -1], [2, -3], [1, 1]], cluster_std=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Display the scatter plot of the randomly generated data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "plt.scatter(X[:, 0], X[:, 1], marker='.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Setting up K-Means\n",
    "Now that we have our random data, let's set up our K-Means Clustering."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "The KMeans class has many parameters that can be used, but we will be using these three:\n",
    "<ul>\n",
    "    <li> <b>init</b>: Initialization method of the centroids. </li>\n",
    "    <ul>\n",
    "        <li> Value will be: \"k-means++\" </li>\n",
    "        <li> k-means++: Selects initial cluster centers for k-mean clustering in a smart way to speed up convergence.</li>\n",
    "    </ul>\n",
    "    <li> <b>n\\_clusters</b>: The number of clusters to form as well as the number of centroids to generate. </li>\n",
    "    <ul> <li> Value will be: 4 (since we have 4 centers)</li> </ul>\n",
    "    <li> <b>n\\_init</b>: Number of time the k-means algorithm will be run with different centroid seeds. The final results will be the best output of n\\_init consecutive runs in terms of inertia. </li>\n",
    "    <ul> <li> Value will be: 12 </li> </ul>\n",
    "</ul>\n",
    "\n",
    "Initialize KMeans with these parameters, where the output parameter is called <b>k_means</b>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "k_means = KMeans(init = \"k-means++\", n_clusters = 4, n_init = 12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's fit the KMeans model with the feature matrix we created above, <b> X </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "k_means.fit(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now let's grab the labels for each point in the model using KMeans' <b> .labels\\_ </b> attribute and save it as <b> k_means_labels </b> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "k_means_labels = k_means.labels_\n",
    "k_means_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We will also get the coordinates of the cluster centers using KMeans' <b> .cluster&#95;centers&#95; </b> and save it as <b> k_means_cluster_centers </b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "collapsed": true,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "k_means_cluster_centers = k_means.cluster_centers_\n",
    "k_means_cluster_centers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Creating the Visual Plot\n",
    "So now that we have the random data generated and the KMeans model initialized, let's plot them and see what it looks like!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Please read through the code and comments to understand how to plot the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    },
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Initialize the plot with the specified dimensions.\n",
    "fig = plt.figure(figsize=(6, 4))\n",
    "\n",
    "# Colors uses a color map, which will produce an array of colors based on\n",
    "# the number of labels there are. We use set(k_means_labels) to get the\n",
    "# unique labels.\n",
    "colors = plt.cm.Spectral(np.linspace(0, 1, len(set(k_means_labels))))\n",
    "\n",
    "# Create a plot\n",
    "ax = fig.add_subplot(1, 1, 1)\n",
    "\n",
    "# For loop that plots the data points and centroids.\n",
    "# k will range from 0-3, which will match the possible clusters that each\n",
    "# data point is in.\n",
    "for k, col in zip(range(len([[4,4], [-2, -1], [2, -3], [1, 1]])), colors):\n",
    "\n",
    "    # Create a list of all data points, where the data poitns that are \n",
    "    # in the cluster (ex. cluster 0) are labeled as true, else they are\n",
    "    # labeled as false.\n",
    "    my_members = (k_means_labels == k)\n",
    "    \n",
    "    # Define the centroid, or cluster center.\n",
    "    cluster_center = k_means_cluster_centers[k]\n",
    "    \n",
    "    # Plots the datapoints with color col.\n",
    "    ax.plot(X[my_members, 0], X[my_members, 1], 'w', markerfacecolor=col, marker='.')\n",
    "    \n",
    "    # Plots the centroids with specified color, but with a darker outline\n",
    "    ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,  markeredgecolor='k', markersize=6)\n",
    "\n",
    "# Title of the plot\n",
    "ax.set_title('KMeans')\n",
    "\n",
    "# Remove x-axis ticks\n",
    "ax.set_xticks(())\n",
    "\n",
    "# Remove y-axis ticks\n",
    "ax.set_yticks(())\n",
    "\n",
    "# Show the plot\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice\n",
    "Try to cluster the above dataset into 3 clusters.  \n",
    "Notice: do not generate data again, use the same dataset as above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# write your code here\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click __here__ for the solution.\n",
    "\n",
    "<!-- Your answer is below:\n",
    "\n",
    "k_means3 = KMeans(init = \"k-means++\", n_clusters = 3, n_init = 12)\n",
    "k_means3.fit(X)\n",
    "fig = plt.figure(figsize=(6, 4))\n",
    "colors = plt.cm.Spectral(np.linspace(0, 1, len(set(k_means3.labels_))))\n",
    "ax = fig.add_subplot(1, 1, 1)\n",
    "for k, col in zip(range(len(k_means3.cluster_centers_)), colors):\n",
    "    my_members = (k_means3.labels_ == k)\n",
    "    cluster_center = k_means3.cluster_centers_[k]\n",
    "    ax.plot(X[my_members, 0], X[my_members, 1], 'w', markerfacecolor=col, marker='.')\n",
    "    ax.plot(cluster_center[0], cluster_center[1], 'o', markerfacecolor=col,  markeredgecolor='k', markersize=6)\n",
    "plt.show()\n",
    "\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Customer Segmentation with K-Means\n",
    "Imagine that you have a customer dataset, and you need to apply customer segmentation on this historical data.\n",
    "Customer segmentation is the practice of partitioning a customer base into groups of individuals that have similar characteristics. It is a significant strategy as a business can target these specific groups of customers and effectively allocate marketing resources. For example, one group might contain customers who are high-profit and low-risk, that is, more likely to purchase products, or subscribe for a service. A business task is to retaining those customers. Another group might include customers from non-profit organizations. And so on.\n",
    "\n",
    "Lets download the dataset. To download the data, we will use **`!wget`**. To download the data, we will use `!wget` to download it from IBM Object Storage.  \n",
    "__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "!wget -O Cust_Segmentation.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/Cust_Segmentation.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  \n",
    "Before you can work with the data, you must use the URL to get the Cust_Segmentation.csv."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "cust_df = pd.read_csv(\"Cust_Segmentation.csv\")\n",
    "cust_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "As you can see, __Address__ in this dataset is a categorical variable. k-means algorithm isn't directly applicable to categorical variables because Euclidean distance function isn't really meaningful for discrete variables. So, lets drop this feature and run clustering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "df = cust_df.drop('Address', axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Normalizing over the standard deviation\n",
    "Now let's normalize the dataset. But why do we need normalization in the first place? Normalization is a statistical method that helps mathematical-based algorithms to interpret features with different magnitudes and distributions equally. We use __tandardScaler()__ to normalize our dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "X = df.values[:,1:]\n",
    "X = np.nan_to_num(X)\n",
    "Clus_dataSet = StandardScaler().fit_transform(X)\n",
    "Clus_dataSet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In our example (if we didn't have access to the k-means algorithm), it would be the same as guessing that each customer group would have certain age, income, education, etc, with multiple tests and experiments. However, using the K-means clustering we can do all this process much easier.\n",
    "\n",
    "Lets apply k-means on our dataset, and take look at cluster labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "clusterNum = 3\n",
    "k_means = KMeans(init = \"k-means++\", n_clusters = clusterNum, n_init = 12)\n",
    "k_means.fit(X)\n",
    "labels = k_means.labels_\n",
    "print(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Insights\n",
    "We assign the labels to each row in dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "df[\"Clus_km\"] = labels\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We can easily check the centroid values by averaging the features in each cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "df.groupby('Clus_km').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, lets look at the distribution of customers based on their age and income:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "area = np.pi * ( X[:, 1])**2  \n",
    "plt.scatter(X[:, 0], X[:, 3], s=area, c=labels.astype(np.float), alpha=0.5)\n",
    "plt.xlabel('Age', fontsize=18)\n",
    "plt.ylabel('Income', fontsize=16)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mpl_toolkits.mplot3d import Axes3D \n",
    "fig = plt.figure(1, figsize=(8, 6))\n",
    "plt.clf()\n",
    "ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)\n",
    "\n",
    "plt.cla()\n",
    "# plt.ylabel('Age', fontsize=18)\n",
    "# plt.xlabel('Income', fontsize=16)\n",
    "# plt.zlabel('Education', fontsize=16)\n",
    "ax.set_xlabel('Education')\n",
    "ax.set_ylabel('Age')\n",
    "ax.set_zlabel('Income')\n",
    "\n",
    "ax.scatter(X[:, 1], X[:, 0], X[:, 3], c= labels.astype(np.float))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "k-means will partition your customers into mutually exclusive groups, for example, into 3 clusters. The customers in each cluster are similar to each other demographically.\n",
    "Now we can create a profile for each group, considering the common characteristics of each cluster. \n",
    "For example, the 3 clusters can be:\n",
    "\n",
    "- AFFLUENT, EDUCATED AND OLD AGED\n",
    "- MIDDLE AGED AND MIDDLE INCOME\n",
    "- YOUNG AND LOW INCOME"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Want to learn more?\n",
    "\n",
    "IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: [SPSS Modeler](http://cocl.us/ML0101EN-SPSSModeler).\n",
    "\n",
    "Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at [Watson Studio](https://cocl.us/ML0101EN_DSX)\n",
    "\n",
    "### Thanks for completing this lesson!\n",
    "\n",
    "Notebook created by: <a href = \"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>\n",
    "\n",
    "<hr>\n",
    "Copyright &copy; 2018 [Cognitive Class](https://cocl.us/DX0108EN_CC). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/).​"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "widgets": {
   "state": {},
   "version": "1.1.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}