What other websites (providers) should we scrape?

[rpmullig]

I attempted to adjust the 'providers' in the settings.yaml, but I found a few that raised errors. The following would be great additions to impact the tool:

• 'hire.google'
• 'Angel.co'
• 'greenhouse.io'
• 'jobs.jobvite'
• 'workable'

[PaulMcInnis]

Just adding to this list here, so it is captured in the right place: https://news.ycombinator.com/jobs

[PaulMcInnis]

This seems like a decent source as well: https://remote.co/remote-jobs/

[muhib1]

I tried job funnel on replit but it's failing.

I am also looking for scraper where we can scrape only remote jobs. Since indeed, monster and glassdoor have implemented antiscraping tools.

I am trying to scrape it from weworkremotely or remoteok. Is there anyway i can use job funnel code and plug in these sites? I am not at all an expert and can use some help. Thank you!

[rpmullig]

Before moving into development, I worked in Finance and found many venture capital firms that had websites with listings of jobs for their investments. I could find a few of the main ones and post here.

[apontejosea]

Is there any thoughts on inverting the dependency and make the scrapers pluggable? I'm thinking there is opportunity to define a clear interface/abstract class for people to implement their own scrapers. Later on, we could use the entrypoint mechanism to enable people to implement plugins in a separate package. In that way, the main JobFunnel framework can become more stable and anyone could create and maintain their own plugins as necessary as Python packages. Once in pypi, a requirements file could eventually look something like:

jobfunnel
jobfunnel-linkedin
jobfunnel-monster
...

[PaulMcInnis]

@Josian I'd be interested to see this idea with a more fleshed-out example architecture, It might be a good idea for maintainability and as the bloat grows around special cases.

I could see that fitting into the current ABC design where we are essentially building Base objects and then specializing per-locale, per-provider.

If you're interested in taking this further, a design for the class stubs would be a good place to start the discussion.

[PaulMcInnis]

#135 indeed India