Open issues for ACT implementation inconsistencies

[philljenkins]

• Review inconsistencies and
• open issues against ACT and/or Checker to resolve the 5 remaining inconsistent rules.

Current reported status from Checker v 3.1.77

The test results were last updated 7 November 2024.

References

• Checker's Implementation report of ACT rules https://www.w3.org/WAI/standards-guidelines/act/implementations/equal-access/ (updated 7 November 2024)
• Checker's raw results sent to ACT https://unpkg.com/accessibility-checker-engine@3.1.77/act-report-v2.txt
• Checker's Rule mapping spreadsheet file on box

[philljenkins]

This was not part of original work estimated for 6.3 release

[philljenkins]

This issue will be much easier and faster when #2004 is completed.

[philljenkins]

Summary of inconsistent ACT rules below (updated with 7 November results):

• Failed examples should be failed by at least one of the Checker rules fail reason codes.
• Checker should be mapping (report against) the same success criteria as what is expected by the ACT rule.

1. Partial: Element marked as decorative is not exposed 46ca7f:

   • Mapping: The Checker implementation reports different success criteria as failed from what is expected by this rule. Checker reports 1.1.1 while ACT expects None.
   • aria_attribute_valid: Pass, Fail_invalid_role_attr, Fail_invalid_implicit_role_attr and maps ARIA same as ACT.
   • "failed correctly by " img_alt_null: potential_aria_override on Failed Example 2, but maps to 1.1.1
   • Mapping: This rule reports 1.1.1 while ACT expects None. This rule & reason code should map to ARIA, not 1.1.1.
   • opened issue fixrule(img_alt_null) Maps to 1.1.1 while ACT expects None #2089

2. Partial: Image has non-empty accessible name 23a2a8:

   • Mapping: The Checker implementation correctly reports which success criteria ACT expects are failed by this rule: 1.1.1.
   • "inapplicable" Failed Example 2 is incorrectly reported by Checker rule img_alt_valid as "inapplicable", should be "failed".
   • opened issue fixrule(img_alt_valid) Reports "inapplicable" while ACT 23a2a8 expects "Failed" for Fail Example 2 #2090

3. Partial: Scrollable content can be reached with sequential focus navigation 0ssw9k:

   • Mapping: ACT is incorrectly expecting mapping to a AAA 2.4.3 Keyboard (No exception). Checker doesn't support AAA's
   • Opened issue against ACT: Scrollable content [0ssw9k]: Equal Access checker listed as inconsistent because it doesn't report additional AAA criteria #2198

4. Partial: Meta element has no refresh delay (no exception) bisz58 with 3 test cases:

   • "cannot tell" Failed Example 1
   • "passed" Failed Example 2
   • "passed" Failed Example 3
   • Mapping: Checker reports against 2.2.1 Timing Adjustable, while ACT expects both 2.2.4 Interruptions AAA and 3.2.5 Change on Request AAA.
   • RuleID: meta_refresh_delay with ReasonID: Potential_1 reports against bisz58:cantTell | bc659a:cantTell
   • RuleID: meta_refresh_delay with ReasonID: Pass_0 also reports against bisz58:pass | bc659a:pass
   • RuleID: meta_redirect_optionalwith ReasonID: fail_longrefresh flags (doesn't report to ACT) this Fail Example 2 test case as Violation, but it's mapped to 2.2.1 Timing Adjustable while ACT would expect it to be mapped to 2.2.4 Interruptions AAA and 3.2.5: Change on Request AAA. Again, this ...optional rule does not currently report against this ACT bisz58. The ...optional rule currently only report against ACT bc659a.
   • Secondary requirement: ACT considers 2.2.1 Timing Adjustable (Level A) a "secondary requirement" because 2.2.1 is less strict than this ACT bisz58 rule. 2.2.1 allows redirects longer than 20 hours. The meta element in Failed Example 2 redirects after 20 hours.
   • opened issue fixrule(meta_refresh_delay | meta_redirect_optional) Reports "cannot tell" and "passed" while ACT bisz58 expects "Failed" for 3 Fail Example test cases #2091

5. Partial: Orientation of the page is not restricted using CSS transforms b33eff:

   • "inapplicable" Failed Example 4
   • RuleID: element_orientation_unlocked with ReasonID: pass, fail_locked.
   • The Checker implementation correctly reports 1.3.4 Orientation as failed by this ACT rule.
   • opened issue fixrule(element_orientation_unlocked) Reports "inapplicable" while ACT b33eff expects "Failed" #2092

[shunguoy]

focus on API differences.

[philljenkins]

Using new Rules mapping spreadsheet auto-built Artifact for each PR Action

[philljenkins]

Checker info from Rules mapping spreadsheet

img_alt_null	potential_aria_override	When the intent is to mark an image as decorative with an empty 'alt' attribute, the 'aria' or 'title' attributes should not be used	The image 'alt' attribute is empty, but the 'aria' label is not empty and overrides the 'alt' attribute	Violation Potential	1	1.1.1	46ca7f:fail

[MHoov]

@philljenkins what is the status?

[philljenkins]

Waiting on the latest Rules mapping spreadsheet auto-built Artifact for the latest PR Action

[philljenkins]

Now updated 7 November for v 3.1.77 release (build):

• Rules mapping spreadsheet file on box
• Rule set https://unpkg.com/accessibility-checker-engine@3.1.77/help/rules.html
• ACT results https://unpkg.com/accessibility-checker-engine@3.1.77/act-report-v2.txt
• Checker's ACT Implementation report https://www.w3.org/WAI/standards-guidelines/act/implementations/equal-access/

[philljenkins]

• Inconsistencies investigated using latest build / release of Checker v 3.1.77
• 1 issue opened with ACT
• 4 issues opened with Checker:
• issue fixrule(img_alt_null) Maps to 1.1.1 while ACT expects None #2089
• issue fixrule(img_alt_valid) Reports "inapplicable" while ACT 23a2a8 expects "Failed" for Fail Example 2 #2090
• issue fixrule(meta_refresh_delay | meta_redirect_optional) Reports "cannot tell" and "passed" while ACT bisz58 expects "Failed" for 3 Fail Example test cases #2091
• issue fixrule(element_orientation_unlocked) Reports "inapplicable" while ACT b33eff expects "Failed" #2092