Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HW rounding fix for all scales #1732

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

polymetal0
Copy link

@abaire @antangelo I continued the work from #735 and #1362 by adding the scale factor to the shader function. From the tests I've done, the behavior is the same for all scales.

antangelo and others added 2 commits July 27, 2024 13:43
Co-authored-by: Erik Abair <erik.abair@gmail.com>
Since screen_pos (and therefore also pixel) are multiplied by the scale factor but the correct subpixel offset is always the same, it was fair to assume that the final subpixel value had to be divided by the scale factor for it to be corrected.

Tested on pgraph tests, Fable: TLC and Halo: CE (weapon zoom in).
@polymetal0 polymetal0 closed this Aug 3, 2024
@polymetal0 polymetal0 reopened this Aug 3, 2024
@polymetal0 polymetal0 marked this pull request as draft August 3, 2024 01:17
@antangelo
Copy link
Contributor

I ran the pgraph tests against this and it doesn't look like the output is correct at or above 3x. Testing on BFBB has shadows clipping at or above 2x. If I understand the approach correctly (and I need to spend some more time thinking about it either way), this is essentially flooring the vertex coordinate, applying the scale and then attempting to correct rounding using the unscaled subpixel. At 10x a subpixel offset of 0.5 will expand to 5 pixels error after scaling, which I think is the artifacting I'm seeing.

In any case, I think the more pressing issue to be resolved before any of these approaches can be landed is passing the ProjAdjacentGeometry_0.5625 test case and fixing the regression in Tiger Woods that the test case is based on. The issue of it not working above 1x is less critical, and we can probably fix it later (if there even is a good fix-- I don't think the problem is well defined).

@polymetal0
Copy link
Author

@antangelo Thanks for the quick testing. Oh, shame...yeah, I noticed some weird wavy things at high scales.

I implemented this while working on #1707 because depth testing messes something up and the W-param tests also suffer from rounding issues. I thought maybe the z-fighting/flickering could be solved like this, but nope.

Still, this fixes the top and left margins issue in games like Fable and Halo CE.

@polymetal0
Copy link
Author

In any case, I think the more pressing issue to be resolved before any of these approaches can be landed is passing the ProjAdjacentGeometry_0.5625 test case and fixing the regression in Tiger Woods that the test case is based on. The issue of it not working above 1x is less critical, and we can probably fix it later (if there even is a good fix-- I don't think the problem is well defined).

In case it might be of help, setting the lessThan operator to lessThanEqual in the vsh code breaks the other 0.5625 tests, but I get this output for this particular case:

ProjAdjacentGeometry_0 5625

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants