Update jck tap file content to support enable jck_custom rerun link

[sophia-guo]

If diagnostic=all, keep all outputs
if diagnostic=failure, filter the testcase and summarize result, same format as openjdk. This will

1. make it easy to know which test cases failed
2. enable rerun jck_custom link with failed test cases.

[smlambert]

This seems like a nice feature to add, moving this into the 2024 1Q plan.

[sophia-guo]

The interesting thing is for failed or error testcases sometimes [the test outputs are stored to tapfile]( https://ci.eclipse.org/temurin-compliance/view/JCK17/job/Test_openjdk17_hs_extended.jck_aarch64_mac_testList_1/1/ , https://ci.eclipse.org/temurin-compliance/view/JCK17/job/Test_openjdk17_hs_special.jck_x86-64_linux_testList_0/1/console) sometimes test outputs are not stored to tapfile. Suppose all those outputs should be kept in tapfile.

[sophia-guo]

The reason that part of stderr message is missing is the parsing logic is if tests failed the output between testcase starts and string testcase_FAILED will be kept. For some reason the output messages are later than the testcase_FAILED message, which is quite common for jck tests( doesn't matter it's passed or failed test cases)

08:14:02  Setup time: 0 seconds
08:14:02  Cleanup time: 0 seconds
08:14:02  Tests found in the suite: 59,288
08:14:02  Test results: passed: 3,657; error: 1
08:14:02  
08:14:02  Results written to /home/jenkins/workspace/Test_openjdk17_hs_extended.jck_arm_linux/aqa-tests/TKG/output_17065237431305/jck-compiler-lang-MOD_0/work
08:14:02  Error: Some tests did not pass
08:14:02  Report written to /home/jenkins/workspace/Test_openjdk17_hs_extended.jck_arm_linux/aqa-tests/TKG/output_17065237431305/jck-compiler-lang-MOD_0/report
08:14:03  -----------------------------------
08:14:03  jck-compiler-lang-MOD_0_FAILED
08:14:03  -----------------------------------
08:14:03  jckversion determined to be jck17
08:14:04  lang/MOD/mod004/mod00401m0/mod00401m0.html   Passed. compilation failed as expected
08:14:04  lang/MOD/mod004/mod00401m001/mod00401m001.html   Passed. compilation failed as expected
08:14:04  lang/MOD/mod004/mod00401m01/mod00401m01.html   Passed. compilation failed as expected
08:14:04  lang/MOD/mod004/mod00401m011/mod00401m011.html   Passed. compilation failed as expected
08:14:04  lang/MOD/mod004/mod00401m02/mod00401m02.html   Passed. compilation failed as expected
08:14:04  lang/MOD/mod004/mod00401m021/mod00401m021.html   Passed. compilation failed as expected
08:14

[sophia-guo]

There are two solutions to correctly keep the failed testcases detail info.

One is to update the logic to consider both testcase failed string and the string Testcase Finish (for exampe jck-compiler-lang-MOD_0_FAILED and jck-compiler-lang-MOD_0 Finish). String jck-compiler-lang-MOD_0 Finish will always be at the end of testcase output.

The other option is for jck test if there are failed testcases summary.txt will always have all the testcase result summary.

[smlambert]

Could also check if an option or setting in the underlying javatest framework would result in different behaviour. (User Guide: https://docs.oracle.com/javacomponents/javatest-4-6/cli-user-guide/CLI.pdf).

[sophia-guo]

I think it might be related with test itself as testcases running in a same build might have that output late issue some of them don't.