Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve reliability of ennexOS (and maybe SEC) webscraper #39

Open
solderq35 opened this issue Jan 29, 2024 · 0 comments
Open

Improve reliability of ennexOS (and maybe SEC) webscraper #39

solderq35 opened this issue Jan 29, 2024 · 0 comments

Comments

@solderq35
Copy link
Contributor

Relatively low priority as ennexOS only errors every few weeks at most, and SEC not at all for several months. Can't hurt though.

By moving most of the webscraper code into a try catch block + while loop as shown in PacificPower branch below, we can improve the reliability of EnnexOS and maybe SEC.

A key thing to note is that "await browser.newPage" is in the try catch block / while loop, which is ultimate failsafe. However, remember that clicking the cookies and logging in does not have to be re-done, so refer to PacificPower code for how to keep track of that with a flag.

https://github.com/OSU-Sustainability-Office/automated-jobs/blob/pacific-power/PacificPower/readPP.js#L72

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant