Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix professors for sections not being scraped #244

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

mehallhm
Copy link
Contributor

NOTE: This is a redo of this PR to decouple this from the Yarn Berry stuff. Unfortunately, there had to be a little package patching to make this match that will have to be pulled out later.

Purpose

Scrapes the professors for a section. Because the Banner API removed this field from the search results when not authenticated through Northeastern SSO, this has to be acquired via a separate request to the section professor details page.

Tickets

Feature List

  • When scraping sections, fires an additional request to the section's professor details page to get the faculty information.

Notes

This is kinda thrown together and the implementation very much can be improved. For now, it gets the job done but the scraper needs some love... I think there is code in there that is tested but never used?

Reviewers

Primary:
@ananyaspatil

Secondary:
@nickpfeiffer05 @Anzhuo-W @cherman23

@coveralls
Copy link
Collaborator

Coverage Status

coverage: 79.896% (-0.3%) from 80.173%
when pulling 6d6e9fc on scraper-prof-update
into a5d14a0 on master.

Copy link
Member

@Lucas-Dunker Lucas-Dunker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, merge and rescrape asap! Just remove that console.log lol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Teaching faculty for classes not being scrapped
3 participants