IT Jobs Meta v0.2.1

Bugfix release (v0.2.1)

Includes:

• Fix various minor mypy errors
• Add mypy type check to the GitHub Actions pipeline

IT Jobs Meta v0.2.1

Bugfix release (v0.2.1)

Includes:

• Fix capitalization in help messages
• Restart app services on deployment

IT Jobs Meta v0.2.0

Feature release (v0.2.0)

This release puts the project into archive mode

Due to changing NoFluffJobs APIs and increased security keeping this project in sync with the data source has become time consuming and challenging. The project will be put into archive mode (see the changelog below to understand what it means from technical terms). However, the dashboard is still available in the archive mode and the project can still be revived.

Client-side dashboard changes

• The navabr label is now customizable and will display "ARCHIVE" in the main instance. See the changelog for CLI options to understand how this option is utilized.

Server-side CLI changes

See README or use --help options to get CLI reference.

Data pipeline command

• Archive mode -a or --from-archive is introduced; now data can be obtained from a custom URL that serves postings in a format that follows the data lake storage convention.
• Dumping raw data (with metadata) to data lake is now optional. Previously mandatory -r/--redis/-b/--s3-bucket arguments can now be omitted to skip the data lake storage step.

Dashboard command

• Custom (optional) navbar label -l/--label is introduced.

Bug fixes and internal changes

• Concept drift is mitigated for new locations format (this update is backwards compatible compatible) – ETL pipeline accepts entries without 'city' fields.
• Deprecated setuptools related files/entries are removed (e.g. setup.py).
• Requirements versions are now pinned down.
• Requirements file is linked into setup.cfg (now there is single point of truth for packages and versions).
• Major deployment process revamp:
  • Terraform setup is restructured and enhanced:
    • Terraform setup now supports separate development mode (or any custom setup) via Terraform workspaces.
    • Most of Terraform resources are tagged.
    • Custom VPC is used instead of the default one, networking is revamped.
    • Ubuntu AMI selection is less cryptic.
    • Terraform outputs now include .pem access key, templated S3 bucket config for the app and Ansible inventory for the EC2 server.
  • Deployment to EC2 instance is now done using Ansible:
    • Old shell-based deployment script is removed.
    • Ansible runs from Terraform generated inventory
    • Deployment is fully automated (swap, Docker, database, Python, pip, nginx config, SSL cert with Let's Encrypt, application installation, systemd units for app launch).
    • MongoDB config is generated from Ansible (with random credentials).
    • Apps (pipeline & dashboard) startup parameters are specified via the it_jobs_meta_vars.yml variables file.
  • Read the README.md file in the deployment dir for more instructions on the deployment process.

IT Jobs Meta v0.1.2

Bugfix release (v0.1.2)

Includes:

• Bump Python module version. From now python should be versioned in sync with git version tags and releases.

IT Jobs Meta v0.1.1

Bugfix release (v0.1.1)

Includes:

• Fix issues with string formatting for categories (now they are formatted nicely in title case)
• Update help messages in README

IT Jobs Meta v0.1.0

Feature release (v0.1.0)

Client-side dashboard changes

• Concept drift is mitigated for lower number of Junior-seniority vacancies
  • Maps feature top N cities with largest number of postings (contrary to previous thresholding)
  • Violin plots discard drawing curves with insufficient/small number of samples for better visuals

Server-side CLI changes

• CLI switch to specify log-level/verbosity with -v or --log-level; see it-jobs-meta --help for the available options

Bugfixes and internal changes

• During the ETL process string values in technologies and categories are unified to lower before further processing
• Static assets are now correctly installed and accessed by the application when installed by pip (missing CSS files are no longer a problem)

IT Jobs Meta v0.0.2

Bugfix release (v0.0.2)

Includes:

• Fix issues with capitalization (e.g. Sql vs SQL, etc.)
• Limit cache threshold

IT Jobs Meta v0.0.1

First release! (v0.0.1)

Includes:

• Data pipeline sub application with:
  • Data Lakes: AWS S3 (deployment) and Redis (development)
  • Data warehouse/databases: SQL (MariaDB), NoSQL (MongoDB)
• Dashboard app based on MongoDB data warehouse (WSGI supported for deployment)
• Infrastructure deployment with Terraform on AWS with S3 and EC2
• Jupter documentation for the data exploration process

Resort to README for help and available options.