HTML2PDF: System chromedriver for PDF export

[haxtibal]

PDF export requires chrome/chromedriver. Currently StrictDoc always uses webdriver_manager to download a suitable chromedriver and install in a strictdoc cache subdirectory.

There may be reasons to prefer a system installation over an adhoc download (e.g. security policy). Notably Debian provides packages that work out-of-the-box for StrictDoc. GitHub Ubuntu CI images have the upstream version pre installed.

This adds an CLI option --chromedriver to select an explicit chromedriver. If not given, strictdoc uses webdriver_manager as usual.

To use the Debian package, one would call

apt install chromium chromium-driver
strictdoc export --fromats=html2pdf --chromedriver=/usr/bin/chromedriver .

To use chromedriver from GitHub Ubuntu image, one would call

 strictdoc export --fromats=html2pdf --chromedriver=$CHROMEWEBDRIVER .

[haxtibal]

@stanislaw Sorry if these CI debug trials cause E-Mail spam at your side. I try to figure out why $CHROMEWEBDRIVER (defined by GitHub CI image) is not passed into lit in the tox py37 env, while it works in py312 env.

EDIT: I think it's solved now, thanks to a former stackoverflow answer of yours. I thought this snippet from LLVM guide

the syntax of the RUN lines is similar to a shell’s syntax for pipelines including I/O redirection and variable substitution

meant one could use $MY_VAR in RUN lines, but seemingly not. Still want to investigate why py312 reported OK. Most likely --chromedriver= is interpreted as "" in py37 and as None in py312.

[haxtibal]

e2e test edit_grammar_add_field_move_up_and_down_save fails on macOS, but works on Linux and Windows. It should be unrelated to this PR, but can't prove it.

[stanislaw]

e2e test edit_grammar_add_field_move_up_and_down_save fails on macOS, but works on Linux and Windows. It should be unrelated to this PR, but can't prove it.

Yeah, that seems to be a rather rare flaky test. I recorded it in the code climate ticket. For now, just restart.

@stanislaw Sorry if these CI debug trials cause E-Mail spam at your side. I try to figure out why $CHROMEWEBDRIVER (defined by GitHub CI image) is not passed into lit in the tox py37 env, while it works in py312 env.

No worries at all! I am happy to see all the updates, including CI. I am glad that you figured it out for LIT.

Overall, your approach looks good but I will see if I have any specific comments.

[stanislaw]

@haxtibal the tests like this one edit_grammar_move_field_down are known issues but they used to fail very rarely. I did one round of checking why the could fail and didn't find an obvious reason back then.

I often get an impression that SeleniumBase/Selenium receive updates to their logic and this makes some of these flaky tests to re-surface and start failing more often. Right now it looks like these errors have nothing to do with your branch. I recorded all of these in the code climate and should investigate them, otherwise we will get blocked all the time in the future.

[stanislaw]

One reason that used to plague a lot of the end2end tests used to be a generic HTTP 500 Web Driver Exception. A solution to this error in that case was to implement a retry mechanism, catching this 500 error. After that, you can very often see in the build logs that this machine actually works all the time, catching these 500 errors.

I think this may be the only strategy that actually works: catch and retry, two times.

[stanislaw]

This is the one I was talking about. It is right there in the failing job:

selenium_connection_error_try_handler: Exception <class 'selenium.common.exceptions.WebDriverException'> thrown when attempting to run <function E2ECase.open at 0x10f3210d0>, attempt 1 of 3.
selenium_connection_error_try_handler: This is a ERR_CONNECTION_REFUSED exception. Trying again...

[haxtibal]

I think this may be the only strategy that actually works: catch and retry, two times.

SeleniumBase has retry logic built in and we see it in action. The failing exception happens in this code path, which is already the retry of a retry, IIUC.

Any ideas how to reproduce this for debugging, or get good debug information? Ideally we could attach into a running gh runner session, but not sure if that's possible at all. Or maybe making a browser screenshot right before the failing test statement, or dumping the DOM state...

[stanislaw]

I think this may be the only strategy that actually works: catch and retry, two times.

SeleniumBase has retry logic built in and we see it in action. The failing exception happens in this code path, which is already the retry of a retry, IIUC.

Any ideas how to reproduce this for debugging, or get good debug information? Ideally we could attach into a running gh runner session, but not sure if that's possible at all. Or maybe making a browser screenshot right before the failing test statement, or dumping the DOM state...

I cannot check this right now. What I normally do is to explore the context around the failing line very carefully and try to add intermediate checks or add the retry logic. No silver bullets in my pocket.

Looking at the number of the tests randomly failing, I assume SeleniumBase/Selenium received an update that changed logic/timing and this revealed the existing weaknesses in the tests that are currently failing.

[haxtibal]

I cannot check this right now. What I normally do is to explore the context around the failing line very carefully and try to add intermediate checks or add the retry logic. No silver bullets in my pocket.

I'll revert the last debug commit to set the PR in a reviewable/functional state.

And then will do experiments regarding the failing test in a separate branch or personal repository. Debug your GitHub Actions by using ssh sounds interesting.