Skip to content

Latest commit

 

History

History
74 lines (56 loc) · 5.19 KB

README.md

File metadata and controls

74 lines (56 loc) · 5.19 KB
Raw

Job Classification Model

Motivation

Having completed Data School: Machine Learning with Text course, I wanted to apply the concepts I learned to answer a question I frequently got asked: What is the difference between a data scientist and a data analyst?

Summary

I created a supervised learning model on a subset of job postings and used the keywords to predict whether a job description was for a data scientist or data analyst. I then examined the false negatives and false positives and extracted the frequent keywords/phrases to improve the model and learn more about the roles.

Results

My Multinomial Naive Bayes model was able to predict 76% of job cases correctly, with the default 0.5 threshold and had an AUC of 88%, meaning the model did fairly well overall to account for both the true positive rate (sensitivity) and the false positive rate (100-specificity).

I was happy to see that the top terms that appeared for each role were in line with what I expected.

For a more detailed write up of the results, please see my blog post.

Contents

  • jd_classification.py
    • This is the main file where the model is built and tested as well as where the token exploration happens (WIP: modularize this more, see Next Steps).
  • clean_data.py
    • Will eventually contain all the functions that are used to clean and extract data from the original input files.
  • requirements.txt
    • List of all the packages/libraries used in the jd_classification.py
  • images (folder)
    • Contains all the chart outputs
  • jd_files (folder)
    • Contains all the job descriptions text files

Process

Data Collection:

  • I collected a total of 34 unique jobs primarily from big tech companies in Silicon Valley and a handful of smaller companies.

Data Preparation:

  • I stored all the job descriptions in one folder in the .docx format. Each file's name has the company name followed by the title of the role.
  • Function create_corpus_df(): Loops through all files in the folder containing the job postings in order to create a dataframe with columns for the the full file name, description, simplified title and yes/no classifier for primary role.

Feature Exploration/Engineering:

  • Function years_of_experience(): Extracts the number of years listed in the job requirements using regular expressions and creates an additional column in the JD dataframe. If a job has a range for years of experience, the difference between the lower and upper bound is added to a separate column.
  • Function is_a_match(): Finds matches for a given regex for the selected column. (example: creates a yes/no column for whether principal, senior or sr appears in the JD)
  • Chart Box-plot and Violin: Explore the distribution of years of experience within the sample of JDs.

Data Preprocessing & Model Optimization:

  • I used CountVectorizer(), a python scikit-learn library, to create the document-term matrix. I further optimized my model performance by modifying the default parameters such as the n-gram size, the length of tokens, token frequency within each document and the list of stop words.
  • I tried logistic regression, but found multinomial naive bayes to return better results.

Model Validation:

  • I created a pipeline for the tokenizer and model and used cross-validation to test my model accuracy and ROC.
  • Split the data into train/test
  • Create a confusion matrix and examine the false positives and false negatives

Token Exploration:

  • Function create_token_df(): Creates a dataframe of count of tokens and their weighted frequency for each role.
  • Chart - Tornado: Isolate which terms appear most frequently for a given role.
  • Chart - Venn Diagram: Shows a count of common and unqiue terms for each role, conditional on having those terms appear in at least 30% of jop postings per role.
  • Function extract_surrounding_text(): Extracts n number of characters that surround a given word/phrase in a text file with the goal of learning more about the context in which a given keyword was used in.

Next Steps

  • Add confusion matrix
  • Split out the main file into smaller code files for easier management
  • Use Latent Dirichlet Allocation to find topic themes
  • Identify which section requirements are listed in (nice to haves, bottom, top, requirements, etc.)
  • Collect more job samples using BeautifulSoup
  • Identify whether there is some sort of regional pattern
  • Further tune the model parameters using GridSearchCV
  • Create four separate classifiers (junior and senior for each role)
  • Try a clustering algorithm without a training set to see which jobs get automatically grouped
  • Generalize this model for other job pairings

Future Questions

  • How does my model stand up against future job postings? What new terms will become common? Will the distinction between data analyst and data scientist become more grey?