test: explicit failure if Valgrind is explicitly required but not installed

[grom72]

Fail all Valgrind tests when --force-enable option is used for RUNTESTS.[sh|py] and Valgrind is not installed.

Resolves #5784

---

This change is 

[osalyk]

Reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @janekmi)

[codecov]

Codecov Report

Merging #5791 (92a229c) into master (a0be753) will decrease coverage by 0.02%.
The diff coverage is n/a.

❗ Current head 92a229c differs from pull request most recent head 1527cfc. Consider uploading reports for the commit 1527cfc to get more accurate results

@@            Coverage Diff             @@
##           master    #5791      +/-   ##
==========================================
- Coverage   70.99%   70.98%   -0.02%     
==========================================
  Files         131      131              
  Lines       19175    19175              
  Branches     3192     3193       +1     
==========================================
- Hits        13614    13612       -2     
- Misses       5561     5563       +2     

[janekmi]

Reviewed 3 of 4 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @grom72)

---

src/test/RUNTESTS.sh line 321 at r2 (raw file):

					FORCE_CHECK_TYPE=$checktype \
					CHECK_POOL=$check_pool \
					$special_params"

Suggestion:

				# FORCE_CHECK_TYPE is required for easy force-enable condition failure detection.
				# Please see require_valgrind() for details.
				export RUNTEST_EXTRA="CHECK_TYPE=$checktype \
					FORCE_CHECK_TYPE=$checktype \
					CHECK_POOL=$check_pool \
					$special_params"

---

src/test/unittest/unittest.sh line 1445 at r2 (raw file):

	restore_exit_on_error
	if [ $ret -ne 0 ]; then
		if [ "$FORCE_CHECK_TYPE" != "none" ]; then

Suggestion:

		# FORCE_CHECK_TYPE is basically a copy of CHECK_TYPE as it is provided by force-enable.
		# This copy allowed us to workaround the overcomplicated legacy Valgrind processing logic
		# and here just check whether any Valgrind tool has been force-enabled or not.
		# Lack of Valgrind when it is force-enabled causes test failure.
		# Lack of Valgrind, when it was just required by the given test, causes a skip.
		if [ "$FORCE_CHECK_TYPE" != "none" ]; then

---

src/test/unittest/valgrind.py line 196 at r2 (raw file):

        try:
            out = sp.check_output('which valgrind', shell=True,
                                  universal_newlines=True, stderr=sp.STDOUT)

Debug-only or left here intentionally?

[grom72]

Reviewable status: 2 of 4 files reviewed, 3 unresolved discussions (waiting on @janekmi and @osalyk)

---

src/test/RUNTESTS.sh line 321 at r2 (raw file):

					FORCE_CHECK_TYPE=$checktype \
					CHECK_POOL=$check_pool \
					$special_params"

Done.

---

src/test/unittest/unittest.sh line 1445 at r2 (raw file):

	restore_exit_on_error
	if [ $ret -ne 0 ]; then
		if [ "$FORCE_CHECK_TYPE" != "none" ]; then

Done.

---

src/test/unittest/valgrind.py line 196 at r2 (raw file):

Previously, janekmi (Jan Michalski) wrote…

Debug-only or left here intentionally?

Intentionally, to avoid the which valgrind command's srderr be added to the test log.

[janekmi]

Reviewed 2 of 2 files at r3, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @grom72)