About

This is UC Davis BAX452 Machine Learning Individual Project.

The objective of the project is to predict whether a person makes over 50K a year given their demographic variations. To achieve this, several classification techniques are explored. In the end, random forest model yields to the best prediction result.

• Source: https://archive.ics.uci.edu/ml/datasets/adult/

• Income dataset is available at UCI machine learning website: http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data

• Data dictionary is available at: https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.names

How to navigate this notebook

Below is the table of content:

• 1  Introduction
  • 1.1  Objective of the porject
  • 1.2  The importance of census statistics
• 2  Fetching Data
  • 2.1  Import Package and Data
  • 2.2  Data Dictionary
• 3  Data Cleaning
  • 3.1  Dealing with Missing Value
• 4  Feature Engineering
  • 4.1  Predclass
  • 4.2  Education
  • 4.3  Marital-status
  • 4.4  Occupation
  • 4.5  Workclass
  • 4.6  age
  • 4.7  Race
  • 4.8  Hours of Work
  • 4.9  Create a crossing feature: Age + hour of work
• 5  EDA
  • 5.1  Pair Plot
  • 5.2  Correlation Heatmap
  • 5.3  age vs. income level
  • 5.4  Working hour vs. income level
  • 5.5  Occupation vs. Income Level
  • 5.6  Bivariate Analysis
  • 5.7  Race vs. Income Level
• 6  Building Machine Learning Models
  • 6.1  Feature Encoding
  • 6.2  Train-test split
  • 6.3  Principal Component Analysis (PCA)
  • 6.4  Classification Models
    • 6.4.1  Perceptron Method
    • 6.4.2  Gaussian Naive Bayes
    • 6.4.3  Linear Support Vector Machine
    • 6.4.4  Radical Support Vector Machine
    • 6.4.5  Logistic Regression
    • 6.4.6  Random Forest
    • 6.4.7  K-Nearest Neighbors
  • 6.5  Cross Validation
    • 6.5.1  GridSearch
• 7  Reflection

Visualization Excerpts

1. Gini Index for US:

A measure of statistical dispersion intended to represent the income or wealth distribution of a nation's residents, and is the most commonly used measure of inequality.

Source: Gini coefficient

2. Violin Plot

This is a violin plot using matplotlib to show how different occupations yield to salary variations, controlling age variables.

3. Bivariate Analysis

Bivariate analysis is one of the simplest forms of quantitative (statistical) analysis.[1] It involves the analysis of two variables (often denoted as X, Y), for the purpose of determining the empirical relationship between them.

Source: Bivariate Analysis

4. Principal Component Analysis

A statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.

Source:

• Bivariate Analysis: wikipedia
• Bivariate Analysis: Python Data Science Handbook

Data dictionary

Extraction was done by Barry Becker from the 1994 Census database. A set of reasonably clean records was extracted using the following conditions: ((AAGE>16) && (AGI>100) && (AFNLWGT>1)&& (HRSWK>0))

Target

1. Predclass: >50K, <=50K.

• Categorical, income Level is either higher or lower than $50K

Categorical Attributes

1. workclass: (categorical) Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.

• Individual work category

2. education: (categorical) Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.

• Individual's highest education degree

3. marital-status: (categorical) Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.

• Individual marital status

4. occupation: (categorical) Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.

• Individual's occupation

5. relationship: (categorical) Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.

• Individual's relation in a family

6. race: (categorical) White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black.

• Race of Individual

7. sex: (categorical) Female, Male.
8. native-country: (categorical) United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.

• Individual's native country

Continuous Attributes

1. age: continuous.

• Age of an individual

2. education-num: number of education year, continuous.

• Individual's year of receiving education

3. fnlwgt: final weight, continuous.

• The weights on the CPS files are controlled to independent estimates of the civilian noninstitutional population of the US. These are prepared monthly for us by Population Division here at the Census Bureau.

4. capital-gain: continuous.
5. capital-loss: continuous.
6. hours-per-week: continuous.

• Individual's working hour per week