Classfindr

Goal: a service that scrapes Western Washington University's classfinder and generates a mirror database with more diverse and accessible tools.

Current status: No tools are available for an end user, but the program is functional for scraping data and uploading to AWS. Embedded database integration with H2 is partially functional, but has no tools for accessing the data yet. Local writing also contains bugs of varying degrees of severity.

Building:

Prerequisites

• AWS SDK - see link for setup and configuration
• Maven

Setting Up:

1. Clone or download the repository
2. Navigate to the repository folder
3. Run mvn clean install in terminal - this should produce an uberjar (about 9 MB at the time of this commit).
4. Run with java -jar [name of Classfindr jar]

Built With

• Maven
• Jsoup
• H2 DB

Current ToDo

• Complete documentation and in-code comments
• Write functions to read from DynamoDB/local database
• Overhaul document parser; it's convoluted and fails to account for some of the formatting idiosyncrasies of WWU's Classfinder
• Implement BatchWriteItem
• Add constraints for requests to WWU servers in a YAML or JSON
• write script to run a build of this every so often to keep the database updated

Completed ToDos:

• Clean up overall application structure and file organization
• Implement a metrics function with timers to analyze speed of different implementations
• check AWS traffic tier limitations and check if requests need to be metered
• Add writing to a local database instead of DynamoDB
• Implement duplicate CRN checker
• Auto-create missing duplicate-checking files if not present
• Comment clarification

"Sometime" - low-priority long term possible features

• user website or app makes requests either to WWU's classfinder or to the AWS mirror (or to another service altogether), depending on the nature of the query and urgency of the information

UI

Here's a shot of what the UI looks like at this stage (or something close to it)