Scraper for HTML to CSV import

[MirandaEcho]

No description provided.

[benlk]

https://github.com/INN/current-ltw-scraper

[benlk]

Notes on the scraper output:

• where a post meta or term name has dashes, CSV keys have underscores
• project-contact-name is a messy field that will require either manual review or time to write a better importer, for reasons in Scraper: project contact name/email current-ltw-scraper#4
• project-contact-email is left blank, because of the difficulty parsing the project contact name
• post_date uses May 26 in a given year for the reason given in Scraper: project year current-ltw-scraper#12
• project-embed is raw HTML: Scraper: project video or embed current-ltw-scraper#10

Things not done:

• project accessory links are not being scraped at this time; I'm leaving them in the post_content. They aren't being displayed to readers. Scraper: project supplementary links current-ltw-scraper#7
• Fancy download trigger JS isn't implemented; it was a nice-to-have. Right now, the workflow is to copy it from the console. Export: trigger a download offer for the CSV once it's completed current-ltw-scraper#13
• project-contact-email is left blank, because of the difficulty parsing the project contact name

[benlk]

The latest output from the scraper:
output.csv.txt

[MirandaEcho]

Two hours added to estimate for clean up of contact info field.

[benlk]

But we're holding on that for now, right?

[MirandaEcho]

Nope, I found the time for you, so this can go ahead.

[benlk]

Latest scrape data is in INN/current-ltw-scraper#4 (comment)