Zvfh/Zvfhmin ACT

[jjscheel]

Technical Group

Unprivileged Spec IC

ratification-pkg

Zvfh/Zvfhmin

Technical Liaison

Andrew Waterman

Task Category

Arch Tests

Task Sub Category

• gcc
• binutils
• gdb
• intrinsics
• Java
• KVM
• ld
• llvm
• Linux kernel
• QEMU
• Spike

Ratification Target

2Q2023

Statement of Work (SOW)

Component names:
Zvfh/Zvfhmin

Requirements:
Extend the base vector ACT tests and test generation tooling to include Zvfhmin and Zvfh extensions for any VLEN less than 4k.

If necessary, the base vector testcase framework will be refactored as a part of this work so that the future vector "functions" can be supported more simply. This common vector framework should:

• support base vector registers
• support immediates
• support vector configuration (SEW, LMUL, AVL)
• support invalid configurations (e.g. LMUL*VLEN < EGW)
• support different VLEN values

Deliverables:

1. Assembly language tests
2. Coverage models

Acceptance Criteria:

1. Assembly language tests that meet the TestFormatSpec
2. Coverage models using riscv-ctg YAML formatted schema Acceptance criteria
3. Tests pass using the riscof framework

Projected timeframe:
6 months

SOW Signoffs:

• Task group liaison sign-off
• Development partner sign-off
• ACT SIG sign-off date

Waiver

• Freeze
• Ratification

Pull Request Details

No response

[jjscheel]

Need to find someone to help me complete the SOW content.

[allenjbaum]

Any/all vector specs should have very similar SOWs; it is basically a cross product of the scalar form, and applying it to vectors. (with exceptions for Load/Store and ops that have no scalar equivalents (e.g. slides and permutations) The challenge is deciding how to apply that to the 80-90 different configurations that the vector spec allows, and enumeration of all the corner cases (which I think is independent of the actual operation - except Zvk, which introduces the concept of register groups for some ops). It would be fabulous if the coverage for vectors could be automatically generated from the coverage of the equivalent scalar operation - that should be the goal. Not all operation have scalar equivalents, of course (slides and permutes), and vector load/stores ar RIOS has a simple test generator, but I think it is currently too simple (e.g. the same op is used across all lanes, needs configurations to be manually applied). Those should not be difficult things to fix, but it needs to be done. It also needs to implement test size limits (as do scalar FP tests) What I don't know is how well the vector test generator captures corner cases (e.g. mask values, vstart values, etc) and how well the coverage has been defined for those. I don't have a good idea of what the corner cases are beyond the corner cases for the individual operations. I also don't know how it deals with non-determinism, and the vector spec has a fair amount of that (e.g.which values are legal when writing vstart, tail-agnostic ), which is independent of the operation. This is an SOW that could be split into multiple parts: -designing a general framework for testing vector corner cases (as distinguished from operation corner cases) - automatically taking scalar coverage to generate vector tests - scattering operand values across lanes instead of replicating them - automatically generating tests for every legal configuration (which I've been told is in the 80-90 range) - limiting test sizes

…

On Tue, Apr 11, 2023 at 6:54 AM Jeff Scheel ***@***.***> wrote: Need to find someone to help me complete the SOW content. — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJQAYXXJXP4CS2IJFQDXAVO7ZANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were assigned.Message ID: ***@***.***>

[jjscheel]

@allenjbaum, while I agree on the vision, I want this issue in DevPartners to be simply Zvfh and Zvfhmin support. Then, we can utilize the experience to solve this bigger picture as it's own effort, in a separate set of SOWs. Is that ok?

[allenjbaum]

IT works both ways: if they can do Zvfh and Zvfhmin support, without the existing ops, the existing op can use that as a baseline, or they can use whatever tests exist and draft off them. But somebody, at some point, will need to come up with tests that we think have acceptable coverage. I don't really care who goes first... I noticed that Christoff has submitted coverage files for Zvk, which is interesting - I would like to dig into those and see how he's done it.

…

On Wed, Apr 12, 2023 at 8:10 AM Jeff Scheel ***@***.***> wrote: @allenjbaum <https://github.com/allenjbaum>, while I agree on the vision, I want this issue in DevPartners to be simply Zvfh and Zvfhmin support. Then, we can utilize the experience to solve this bigger picture as it's own effort, in a separate set of SOWs. Is that ok? — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJS5YC5OLHZWMJQED2LXA3AYBANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[cmuellner]

I noticed that Christoff has submitted coverage files for Zvk, which is interesting - I would like to dig into those and see how he's done it.

No magic there. In the absence of fundamental vector support, these files are pretty useless.

What's needed is fundamental vector support, which includes at least:

• support of vector registers (ok, that's the part that my coverage files somehow address)
• support of immediates
• support for vector configuration (SEW, LMUL, AVL)
• support for invalid configurations (e.g. LMUL*VLEN < EGW)
• support for different VLEN

I don't think this should be provided as part of any Zv* support.
Instead a dedicated task/SoW for ACT Vector enablement would be more appropriate and transparent.

[allenjbaum]

Exactly the point I was trying to get across.

…

On Wed, Apr 12, 2023 at 9:25 PM Christoph Müllner ***@***.***> wrote: I noticed that Christoff has submitted coverage files for Zvk, which is interesting - I would like to dig into those and see how he's done it. No magic there. In the absence of fundamental vector support, these files are pretty useless. What's needed is fundamental vector support, which includes at least: - support of vector registers (ok, that's the part that my coverage files somehow address) - support of immediates - support for vector configuration (SEW, LMUL, AVL) - support for invalid configurations (e.g. LMUL*VLEN < EGW) - support for different VLEN I don't think this should be provided as part of any Zv* support. Instead a dedicated task/SoW for ACT Vector enablement would be more appropriate and transparent. — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJTVZBQNOLEFQ4TA5G3XA552NANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[cmuellner]

We can already see the following when looking at the Zvk* ACT PR:

• The PR won't be merged without coverage information
• Coverage information depends on vector support as the foundation
• Holding a PR open will lead to bit-rot and cause duplicated work (the Zvk* PR introduces vector support but is targeted for Zvk*)
• It is obvious that every new Zv* extension PR will conflict with existing Zv* PRs (they need to duplicate work for limited vector support for their purposes)
• It is unlikely that people will rewrite their PRs in several months (from scratch) when vector support is finally available (and likely incompatible with the code in the PRs)

So as long as we don't find a way to merge the Zv* ACT PRs, it does not make sense to write them (major parts will be deleted later anyway because the tests should be generated, not written).

It is probably more time efficient to focus on vector support first (or at least a subset of the complete vector support) and add Zvfh/Zvfhmin on top (which should be nothing more than adding a few cgf files). Almost all instructions of Zfh are listed in the chapter "Vector Floating-Point Instructions" of the vector spec (for SP FP and DP FP).

So I believe the following would be a reasonable order of tasks to move forward:

• support of vector registers (ok, that's the part that my coverage files somehow address)
• support of immediates
• support for vector configuration (SEW, LMUL, AVL) - vset(i)vl(i).v
• support for invalid configurations (e.g. LMUL*VLEN < EGW)
• support for different VLEN
• support for exception handling incl vstart support
• support for masking
• at this point the infrastructure should be complete and stable enough to add support for most vector instructions
• support for floating point
• support for Zvfh*
• support for arithmetic
• support for load/store instructions
• support for other parts (alignment, fix-point arith, reduction, permutation, Zvl*, Zve*, etc)

CC: @pawks, as he might have some opinion here as well.

[pawks]

Preface - My understanding of the vector spec is rudimentary, so some of my comments below may be incorrect.

As far as vector goes, there are 4 categories of support required for getting ACT up.

1. Support for architectural states(specifically vtype csr)
2. Support for the vector register file and splitting up the value in the registers into an array of elements (needed for coverage calculation)
3. Writing/Generating Tests
4. Privilege support(testing illegal combinations)

These changes span across 2 to 4 tools:

• riscv-config: 1
• riscv-isac: 1, 2 & 4(some additional work would be required to effectively express the coverpoints)
• riscv-ctg(optional): 3
• riscof: 4(Maybe)

(major parts will be deleted later anyway because the tests should be generated, not written)

This is not entirely true. The policy for ACT is that we don't worry about where the tests come from. The test generator is a means to an end and not the norm. ACT is more concerned about coverage measurement and expression. As long as the coverage is sufficient and measurable, the tests can be handwritten. This is a handwritten test which is trying to test the architectural hazards among various instructions as they may appear in the real world applications.

support of vector registers (ok, that's the part that my coverage files somehow address)

The cgf files in the PR only add the relevant nodes. But that is not sufficient for measuring coverage. Appropriate changes will also be required in riscv-isac(regfile support, support for parsing information from the logs etc).

(they need to duplicate work for limited vector support for their purposes)

Looks like each subextension implements a constrained variant of the function for vectorisation of register values. In which case, I definitely think that writing a general function once should be the priority i.e the first vector subextension to have coverage support should implement them generally.

I agree with the order of the proposed changes mostly. I would definitely de-prioritize support for invalid configurations and exceptional handling towards the end. As always, privilege support is really tricky in itself. It would be easier to do that once the unpriv instruction support is added.

[cmuellner]

Thanks for your input! I think we all agree that partial vector support would be the right thing to.
However, we are confronted with the fact that Zv* extension TGs want ACT support for their ratification asap.

One point, where I'd like to add some more details:

(major parts will be deleted later anyway because the tests should be generated, not written)

This is not entirely true. The policy for ACT is that we don't worry about where the tests come from. The test generator is a means to an end and not the norm. ACT is more concerned about coverage measurement and expression. As long as the coverage is sufficient and measurable, the tests can be handwritten. This is a handwritten test which is trying to test the architectural hazards among various instructions as they may appear in the real world applications.

For floating-point tests, there are these ibm_b* functions, which attempt something similar: avoid purely randomized data, but specify properties that the test cases must fulfill (e.g. NaNs, normal numbers, etc). So we have abstract test case descriptions that define constraints for the generated tests. I think this could (and should) be applied to sha256sum0-rwp1.S as well. So if we define a property that teaches the test case generator to generate appropriate tests, then this property can be reused by other extensions/instructions.

So I believe that we should not have a single handwritten test case file. This would also reduce test case maintenance. For example, you don't need to adjust test cases manually if you change the test language (e.g. remove trapreg_sv and tramptbl_sv) because you can just regenerate all of them.

[allenjbaum]

I am generally in agreement here - maybe more than just "generally". I don't understand the comment about "trapreg_Sv" and "tramptbl_sv", though Those are internal to the trap handler, and they should not need to be changed for future tests. They are effectively spill/fill areas for trap handlers. re thry bring rxpodrf beyond the trap handler?

…

On Thu, Apr 13, 2023 at 5:58 AM Christoph Müllner ***@***.***> wrote: Thanks for your input! I think we all agree that partial vector support would be the right thing to. However, we are confronted with the fact that Zv* extension TGs want ACT support for their ratification asap. One point, where I'd like to add some more details: (major parts will be deleted later anyway because the tests should be generated, not written) This is not entirely true. The policy for ACT is that we don't worry about where the tests come from. The test generator is a means to an end and not the norm. ACT is more concerned about coverage measurement and expression. As long as the coverage is sufficient and measurable, the tests can be handwritten. This <https://github.com/riscv-non-isa/riscv-arch-test/blob/main/riscv-test-suite/rv32i_m/K/src/sha256sum0-rwp1.S> is a handwritten test which is trying to test the architectural hazards among various instructions as they may appear in the real world applications. For floating-point tests, there are these ibm_b* functions, which attempt something similar: avoid purely randomized data, but specify properties that the test cases must fulfill (e.g. NaNs, normal numbers, etc). So we have abstract test case descriptions that define constraints for the generated tests. I think this could (and should) be applied to sha256sum0-rwp1.S as well. So if we define a property that teaches the test case generator to generate appropriate tests, then this property can be reused by other extensions/instructions. So I believe that we should not have a single handwritten test case file. This would also reduce test case maintenance. For example, you don't need to adjust test cases manually if you change the test language (e.g. remove trapreg_sv and tramptbl_sv) because you can just regenerate all of them. — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJUPUI6MRW66YXWYNA3XA72AXANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[cmuellner]

Those are internal to the trap handler, and they should not need to be changed for future tests. They are effectively spill/fill areas for trap handlers.

Why should a test case author ever have to care about such implementation details of the test suite? That is not even slightly related to most unpriv extensions. So my argument is, that a full-automation approach allows hiding such details and let the author focus on the actual test case.

[allenjbaum]

I think that’s the point- they shouldn’t care, and I am confused why you think the do have to care. All they should care about is that the test expects to trap (and the priv level(s) that it it expects to trap into.) That is enough to have the trap handlers get installed and those areas defined without the test developer needing to know anything about those internals.

…

On Sunday, April 16, 2023, Christoph Müllner ***@***.***> wrote: Those are internal to the trap handler, and they should not need to be changed for future tests. They are effectively spill/fill areas for trap handlers. Why should a test case author ever have to care about such implementation details of the test suite? That is not even slightly related to most unpriv extensions. So my argument is, that a full-automation approach allows hiding such details and let the author focus on the actual test case. — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJQ7HMINJAHU2F4H2P3XBRGS5ANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jjscheel]

@allenjbaum, can you review my proposed SOW in the first entry and let me know what changes you'd like to see?

Given that we're hoping that RIOS will pick this up after they complete the base Vector work, I don't have a problem building in the assumption that upon completion of this work, we have a base vector framework in ACT. How or when that work occurs (base vector SOW or this one) does not matter to me as long as it exists before we declare this one done.

@cmuellner, your thoughts are also welcome here as I used key points from your previous post.

THANKS!

[allenjbaum]

The only thing I would add is that coverage for Zvfh should include the coverage for Zfh operations (which would subsume Zfhmin). The interesting part of that is how it applies to the scaffolding part, .e.g, do you want coverage to be 100% for each lane individually, or is it adequate to do it collectively (under the assumption that each lane is identical so if you test one, you test them all). That is something that every vector test has to consider (i.e. it doesn't apply to all vector ops, specifically those that have inter-element operations like permutations, but that doesn't apply here)

[jjscheel]

@allenjbaum, I understand your logic and generally agree going forward.

In this instance, however, isn't Prasanna already working on Zfh in issue #18? If true, this should make the Zvfh work simpler as the basic operation implementation should be done. Right?

[allenjbaum]

Zvfh is a crossproduct of Zfh and V, so both are necessary in order to save the work. V is the scaffolding part. How operations fit into that scaffolding is the critical element, and once there you can plug in the functions and its operands reasonably easily. Another prerequisite, though, might be the ctg scaffolding to limit the size of tests, because I suspect that sizes will explode. The scaffolding might help there, e.g. instead of a testing that tests just one pair of values it loads for the data region, it would test a vectors of different values, and for extra coverage could reload with the address advanced (and test data replicated so it effectively rotates different values though all lanes. This will keep test sizes fairly constant, of those sorts or tests. What concerns me most is the vector configuration; There are 90(?) legal configurations, which might mean 90 sets of tests must be generated. Signatures will be larger also, so has to be figured into that as well. That ctg dcaffolong

…

On Wed, May 10, 2023 at 5:35 AM Jeff Scheel ***@***.***> wrote: @allenjbaum <https://github.com/allenjbaum>, I understand your logic and generally agree going forward. In this instance, however, isn't Prasanna already working on Zfh in issue #18 <#18>? If true, this should make the Zvfh work simpler as the basic operation implementation should be done. Right? — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJRGQYA4MUK4Q6YRXXLXFODSLANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jjscheel]

@allenjbaum, I understand your comments. However, I'm struggling to know what, if anything needs to change in the SOW. Can you explicitly state the text changes you'd like to see in the SOW description (first entry of this issue)?

Thanks!

[allenjbaum]

My only concern is this is not explicit about the dependency on base vector test support, and I’d rather not put the burden of refactoring here as opposed to requiring the base test to do that - or making that a separate work item not tied to this.

…

On Tuesday, May 23, 2023, Jeff Scheel ***@***.***> wrote: @allenjbaum <https://github.com/allenjbaum>, I understand your comments. However, I'm struggling to know what, if anything needs to change in the SOW. Can you explicitly state the text changes you'd like to see in the SOW description (first entry of this issue)? Thanks! — Reply to this email directly, view it on GitHub <#29 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJXY53PLTCGXLPXUH4TXHSX5LANCNFSM6AAAAAAW2J3PKM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jjscheel]

Ok, I understand. I propose that we wait to decide whether we create a new SOW/item in the list until we have some more work done in the Arch Test SIG with the base Vector ACT. If you folks decide to start restructuring the initial vector ACT into framework and operations, then an additional SOW may not be needed.