Add Occlusion-Parallax mapping to terrain shaders

[yaRnMcDonuts]

As it stands, all of our primary terrain shaders (TerrainLighting.frag, PBRTerrain.frag, AdvancedPBRTerrain.frag) already store a normal map and could have a parllax map easily packed into the alpha channel of this map to keep texture reads low. I actually wrote the advanced PBR terrain shaders so it refers to the normal map as a PackedNormalParallax map for this exact reason.

However, when I tried implementing occlusion-parallax into the PBR shaders, I encountered 2 issues that caused me to put the feature on hold. But hopefully the addition of the tile-deferred rendering will make the optimization issues less severe and can make this feature a real possibility in the core terrain shaders now.

1. I noticed that occlusion-parallax required more than 1 texture read from the same parallax texture, and this seemed to consequentially cause the framerate to tank if I added too many texture-slots to the terrain

2. I did not know what to do if only 1 of the terrain's texture-slots had a parallax map. I tried blending to a parallax value of 0.0 as well as 1.0 for slots with no parallax texture (very similar to how I consider a texture-slot without a unique AmbientOcclusion map to have an AO value of 1.0 for blending purposes). Not sure if this was a mistake on my end, but I'm guessing there is a correct way to do this that I just didn't find.

So I ended up removing the feature from the core PBR terrains (although some parallax code is left commented out in the .fraf file) since it tanked the frame-rate and only worked correctly if every texture-slot had a valid parallax map. But if the terrain's framerate is no longer as big of an issue with the new renderer, then it could be time to finally add this feature to the core terrain shaders.

Any thoughts on this? I know @riccardobl and @oxplay2 (not sure of your @ on here and it didnt auto-fill, my apologies if I tagged you wrong) already have their own implementations of PBR terrains that have occlusion parallax but not a packed metallic-roughness-ao-emissiveIntensity map (I think, correct me if I'm wrong), so I'm interested to hear more on this from you guys.

[oxplay2]

about 1.
I might be wrong, but i think we can solve this by blending final result, instead each texture.
Ofc this will be much slower due to double/triple/etc light calculations
But we could try blend just before light calculations.

about 2.
I think 1. comment might solve this too.

There are more ways, tho dont know any better.

Could you explain me what is your main issue when you blend 0.0 or 1.0 with parallax slots?

As i guess, you mean that for one texture is "too-low spaced" while for another its "too-high spaced"

I guess there is no real way to solve this. Even if all slots are parallax and some of them have very low value parallax compared to another with very high parallax it is visible. Tho thats why terrain is mostly smooth-blended with edit tools to hide this difference.

myself i have it done very raw way
main issue for me are frames per second really. Tho it work fast enough for me, still it is not optimal.

Ofc the more textures are blended together the slower it is, so i started splitting terrain into "grid system" where each terrain can have little different slots. so i try keep 4 slots per grid terrain.

        TextureArray PhongTextures
        TextureArray NormalmapTextures -LINEAR
        TextureArray ParallaxmapTextures -LINEAR

        // Texture map #0
        Float Slot_0_scale
        Float ParallaxHeight_0
        Float ParallaxHeightFix_0
        Float ParallaxAOColorFactor_0 : 0.3
        Boolean UseSlot_0
        Boolean UseNormalMap_0
        Boolean UseParallaxMap_0

#define DEFINE_COORD(index) vec2 coord##index = newTexCoord * m_Slot##index##_scale;

#define CALCULATE_BLEND_PERCENT(ab)\
    currentBlendValue += ab;\
    blendPercent = (0.000001+ab)/(currentBlendValue+0.000001);

#define BLEND(index, ab, texIndex)\
    tempAlbedo.rgb = texture2DArray(m_PhongTextures, vec3(coord##index, texIndex) ).rgb;\
    albedo.rgb = mix( albedo.rgb, tempAlbedo.rgb, blendPercent);\
    normal.rgb = mix(normal.xyz, wNormal.xyz, blendPercent);\
    Metallic = mix(Metallic, m_Metallic##index, blendPercent);\
    Roughness = mix(Roughness, m_Roughness##index, blendPercent);

#define BLEND_NORMAL(index, ab, texIndex)\
    tempAlbedo.rgb = texture2DArray(m_PhongTextures, vec3(coord##index, texIndex) ).rgb;\
    albedo.rgb = mix( albedo.rgb, tempAlbedo.rgb, blendPercent);\
    Metallic = mix(Metallic, m_Metallic##index, blendPercent);\
    Roughness = mix(Roughness, m_Roughness##index, blendPercent);\
    newNormal = texture2DArray(m_NormalmapTextures, vec3(coord##index, texIndex) ).rgb;\
    normal = mix(normal, newNormal, blendPercent);

#define PARALLAX_BLEND(index, ab, texIndex)\
    calculateParallax(coord##index, m_ParallaxHeightFix##index, m_ParallaxHeight##index, m_ParallaxAOColorFactor##index, ab, texIndex);

void calculateParallax(inout vec2 parallaxTexCoord, in float initialParallaxHeight, in float parallaxHeight, in float parallaxAOIntensity, in float intensity, in int texIndex) {
    #ifdef PARALLAX_LOD_DISTANCE  
        if(camDist < m_ParallaxLODDistance && blendPercent > 0.2){
            vec3 vViewDir =  viewDir * tbnMat;
            Parallax_initFor(vViewDir,parallaxHeight);
            Parallax_TextureArray_displaceCoords(parallaxTexCoord, m_ParallaxmapTextures, initialParallaxHeight, texIndex);
        }
    #else
        if(blendPercent > 0.2){
            vec3 vViewDir =  viewDir * tbnMat;
            Parallax_initFor(vViewDir,parallaxHeight);
            Parallax_TextureArray_displaceCoords(parallaxTexCoord, m_ParallaxmapTextures, initialParallaxHeight, texIndex);
        }
    #endif
    parallaxAverageDepth += lastParallaxDepth * blendPercent;
    parallaxAverageAOIntensity += parallaxAOIntensity * blendPercent;
}

usage:

    #ifdef SLOT_0   
        DEFINE_COORD(_0)
        CALCULATE_BLEND_PERCENT(alphaBlend.r)
        #if defined(PARALLAXMAPPING) && defined (PARALLAXMAP_0)
            PARALLAX_BLEND(_0, alphaBlend.r, 0)
        #endif
        #ifdef NORMALMAP_0
            BLEND_NORMAL(_0,  alphaBlend.r, 0)
        #else
            BLEND(_0,  alphaBlend.r, 0)
        #endif
    #endif

as you can see my code is very rough and have crazy values like 0.000001 or 0.2 just to adjust shader.

[yaRnMcDonuts]

In regards to number 1, I was referring to the code in these methods:

        Parallax_initFor(vViewDir,parallaxHeight);
        Parallax_TextureArray_displaceCoords(parallaxTexCoord, m_ParallaxmapTextures, initialParallaxHeight, texIndex);

I'm taking a look through my local copy of OcclusionParallax.glslib (I still have the glslib in my project even though I failed to implement it), and this is the part of the code where it appears to do multiple texture reads from the same parallax map more than once by calling float _Parallax_TextureArray_sampleDepth in a while loop:

float _Parallax_sampleDepth(in sampler2DArray depthMap,in vec2 uv,in int index){
    vec4 d= texture2DArray(depthMap, vec3(uv, index) );
    return _Parallax_selectDepth(d);
}

while(currentLayerDepth < currentDepthMapValue){
        currentTexCoords -= ParallaxData.deltaTexCoords;
        currentDepthMapValue = (_Parallax_sampleDepth(depthMap, currentTexCoords));
        currentLayerDepth += ParallaxData.layerDepth;
    }

So I was assuming that's where the big hit to the framerate comes from.

That also eliminates most of the benefits that come from packing parallax into the alpha channel of the normal map. The value read from the alpah channel of a packedNormalMap can't be reused, since each parallax texture read has a slightly altered texCoord. (But it still is useful to pack them together anyways for saving texture space and reducing texture count)

Could you explain me what is your main issue when you blend 0.0 or 1.0 with parallax slots?

It has been a while since I worked on it, but if I remember correctly, setting a value between 0.0 and 1.0 caused the textures to all be stretched out horizontally, similar to the way a terrain would look on a vertical slope without using triplanar mode.

Although maybe that happened beacuse the parallax equation is not supposed to have an output value in the 0.0-1.0 range (even though the input is in 0.0-1.0 for the parallax map), and maybe that's where I was going wrong.

I do recall looking into copying your LOD-distance optimization. That looks like a pretty good way of making sure occlusion-parallax can be used some way or another even if it causes a low framerate and can be raised for better devices.
But I also was limited by the 64 Defines limit at that point (which a previous PR finally fixed) I and couldn't add that feature without removing other defines that were ultimately more necessary for my game, so that was an unfortunate circumstance.

But now we finally aren't limited by 64 defines thanks to the recent PR fixing that, so I can try to work on this sometime soon and see how it goes.

With no more define limit, I should also be able to add a UseParallax boolean for each texture-slot (i.e. UseParallax_0, UseParallax_1, etc etc. for 0-12) to make sure that enabling one parallax texture for a terrain with 6 texture slot layers wont suddenly read the blank alpha channel of every other packedNormalParallax map with a blank alpha channel.

[oxplay2]

In regards to number 1, I was referring to the code in these methods:

You are right, texture read is most consuming for GPU there, and its called multiple times, multipled by slot numbers(thats what i meant). So if we have 12 blended parallax textures it will be like 12 * displaceCoords_num that could be a lot.

Im not sure of any real way to optimize it. I only limited myself to 4 blends per geom.

It has been a while since I worked on it, but if I remember correctly, setting a value between 0.0 and 1.0 caused the textures to all be stretched out horizontally, similar to the way a terrain would look on a vertical slope without using triplanar mode.

Hmm i didnt noticed this issue, but probably i were using every slot as parallax. I just dont remember now.
But blending parallax textures as shown before in one of my videos, there were normal blend between them. (i just dont remember now if i tried use any of them non-parallax)

I do recall looking into copying your LOD-distance optimization. That looks like a pretty good way of making sure occlusion-parallax can be used some way or another even if it causes a low framerate and can be raised for better devices.

It was just experiment, real LOD im using is splitting Terrain into "grids" and adjust params/data so for distant one can use no parallax or etc. (it is possible with JME terrain, tho i needed write custom to avoid seams)

But now we finally aren't limited by 64 defines thanks to the recent PR fixing that, so I can try to work on this sometime soon and see how it goes.

With no more define limit, I should also be able to add a UseParallax boolean for each texture-slot (i.e. UseParallax_0, UseParallax_1, etc etc. for 0-12) to make sure that enabling one parallax texture for a terrain with 6 texture slot layers wont suddenly read the blank alpha channel of every other packedNormalParallax map with a blank alpha channel.

Yes,
could also just make another j3md with Texture Array that just require GLSL 330+
why? because if someone have 100/150, i think their hardware will not be good enough to run OC parallax anyway.
But maybe there is good reason to stick with 100/150 and still use oc parallax.

[yaRnMcDonuts]

I think it could also help me to understand everything better if I also implement occlusion-parallax in PBRLighting first (unless anyone beats me to it of course), so that way PBRLighting can be used as a simple reference and baseline for how expensive occlusion-parallax should be for a classic geometry, and this will be helpful while trying to optimize occlusion parallax for terrain shaders.

I am surprised to see that Occlusion-Parallax was never implemented into jme's PBRLighting.frag when I just checked on the master branch. I could've swore it was officially added in the past but I must have been wrong.

I also notice that the standard PBRLighting shader has classic parallax as well as steep parallax. Steep parallax also appears to do more than 1 texture read pre parallax map, and based on my experience using it, steep parallax sometimes caused weird artifacts so I stopped using steep-parallax mode.

And there is a classic Parallax mode that is the default for PBRLighting. It sounds like this is not as good as occlusion-parallax, but it could still be useful for terrains to toggle some texture slots between occlusion-parallax and classic-parallax depending on the texture and distance to camera, and low spec devices could just have occlusion-parallax turned off but still gain some benefit from classic parallax.

I also wonder if steep-parallax should be removed completely in place of occlusion-parallax? Or if they are different enough that we should still leave both.

I have little experience in parallax mapping and never really paid attention to how expensive it is in classic vs steep, and barely tried occlusion-parallax. I'll hopefully be figuring some of this out soon, but if anyone knows much about the different parallax mapping approaches id be interested to hear your input.

For reference, this is the classic parallax code, which appears to only do 1 single texture read to get the parallax value

https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/resources/Common/ShaderLib/Parallax.glsllib#L62

and here is the current steep-parallax that appears to do texture reads in a while-loop similar to occlusion-parallax. (i suspect steep parallax is not much more efficient (possibly even less efficient) than occlusion parallax, so if occlusion parallax is added to the terrain shaders, then i suspect there's no good reason to add steep parallax to terrains as well?):
https://github.com/jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/resources/Common/ShaderLib/Parallax.glsllib#L2

[oxplay2]

Im not sure about frame efficiency of classic or steep parallax, tho i know visual quality difference.

All i can say i were using classic and steep one before, but they do not look well. At least for "deep height" tex heightmaps.
While Occlusion Parallax have no problem with deep height textures.

I think good optimization would be change layer amount depend on texture heightmap lowest to highest scale.

const float minLayers = 8.;
const float maxLayers = 64.;

Also im not sure if it was there already, but in wireframe mode need little fix:

        if(isnan(numLayers)){
            numLayers = minLayers;
        }

Ofc if someone want learn all algorithm, Riccardo left link in file comment:
https://learnopengl.com/Advanced-Lighting/Parallax-Mapping

[yaRnMcDonuts]

Hmm i didnt noticed this issue, but probably i were using every slot as parallax. I just dont remember now.

I think I was wrong about this being related to having som empty parallax textures.

It could also be related to triplanar mapping. Does your terrain shader with occlusion-parallax also have triplanar mapping? I think I may have got it working without triplanar mapping, and then I must have struggled too much with triplanar and given up because I still have an unfinished tri-planar parallax mapping method I see in my local copy of OcclusionParallax.glslib

I think that using triplanar mapping with occlusion parallax will be even more resource intensive and I wonder if it would even be viable to use in that case. I suspect that's why I gave up on it, since I already saw heavy framerate drops before I even got to implementing triplanar.

While a normal map will only require 3 texture reads in triplanar mode, a parallax map that has an average depth requiring 5 loops per pixel would end up with 15 texture reads, then multiply that by the number of textureslots on the terrain and things get really out of hand fast. I also can't tell from the code how exactly high the texture reads for occlusion-parallax may go, since that value appears to be partially determined by the height value pulled from the parallaxMap. But it could be very high since the layerDepth value used to detertmine the total number of texture reads on a single parallax map has a min value of 8 and max of 64.

I will still continue with implementing occlusion-parallax into PBRLighting.frag, but I might need to do some extensive optimizing to get it to work in the terrain shader, and even then it will likely be something that only gets enabled for 1 or 2 texture-slots at most and especially only for nearby terrains. I also have plans to make tri-planar mode a per-slot param (by adding boolean params such as Use_Ttriplanar_0, Use_Tri_Planar_1, etc etc) and this way occlusion parallax could be enabled on texture slots that are not using triplanar, and then of course youd have to avoid painting that texture on vertical surfaces then... but this should be suitable for things like rocky dirt or roads that only get painted on flat surfaces, and then (ideally) only things like a mountain texture would require triplanar mode to be enabled.

[yaRnMcDonuts]

I am starting to worry that tri-planar mapping is actually not possible with this type of parallax mapping... if you or @riccardobl have triplanar in your terrain shaders with occlusion parallax, I would be very interested to see how its done, because I think it might be alot more difficult than I thought, if it is possible at all.

Since parallax mapping offsets the textCoord for the following texture read, I think that could make the texCoords invalid with the way triplanar mode has to do 3 tex reads with the world (x/y) (x/z) and (y/z) coords. I worry that the parallax offset applied to those individual texture coords is what is breaking the final result from blending the 3 triplanar tex-reads together for the final output. Triplanar is very finicky so I suspect it does not like having its 3 texCoord inputs adjusted in non-consistent ways based on the heightmap, but im still unsure if its impossible or if there's maybe another small mistake I'm overlooking tha'ts breaking things...

edit: further research shows it may not be possible, but this is mostly just people answering questions in other threads without much explanation backing up my suspicion as to why it won't work. The best explanation tha I found to support my concern that it is not possible is this: "parallax mapping in general doesn't do well outside of planar surfaces so while I suspect it could "work" I would expect a lot of artefacting as soon as you strayed away from axis aligned geometry."

[oxplay2]

indeed, im not using triplannar, so seems that is is root of this problem.

hard topic.

[yaRnMcDonuts]

I've done a bit more research on occlusion parallax,.

I also implemented it into my local fork of PbrLighting.frag to test it out there. But I'm getting some strange artefacts. I'm testing with a model that I thought had a packed normal parallax map but it turns out its alpha channel is empty... I think...

@oxplay2 do you think you could post your whole OcclusionParallax glslib for me to look at? just to make sure I didn't accidentally break something mine when I tried implementing triplanar incorrectly a few years ago.

I also read in the unreal documentation that they advise to use occlusion parallax very sparsely, since it is indeed very resource intensive. So while I will implement it for non tri-planar, I'm not sure it will be worthwhile even trying to get to work in combination with a tri-planar texture slot. It seems to be one of those cool Unreal features that gets shown off a lot in demos or for very high spec devices to impress people, but doesn't seem to scale very well in large indie games (typical Unreal marketing in a nutshell, imo atleast lol)

So I suspect your terrains will benefit greatly from the per-layer occlusion parallax setting (unless of course you are planning to still try to use occlusion parllax for every layer, in which case that will inevitably be a struggle to optimize, although certainly isn't impossible if managed right)

[oxplay2]

unreal is right, i mean not every texture should use it, for example i planned grassy terrain be without it, while where are more blank areas to use it.

Thats why setting triplannar for non parallax ones, would be cool feature.

What about "ifdef parallax_#index -> dont do triplannar, indef parallax_#index -> do triplannar" ?

Also i were using little normal hack (i do not remember why but Ricc helped me back then):

    #if defined(NORMALMAP_0) || defined(NORMALMAP_1) || defined(NORMALMAP_2) || defined(NORMALMAP_3) || defined(NORMALMAP_4) || defined(NORMALMAP_5) || defined(NORMALMAP_6) || defined(NORMALMAP_7) || defined(NORMALMAP_8) || defined(NORMALMAP_9) || defined(NORMALMAP_10) || defined(NORMALMAP_11)
       normal += norm * 0.9;
       normal = normalize(normal * vec3(2.0) - vec3(1.0));
       normal.z /= 100;
       normal = normalize(normal);
    #else
       normal = normalize(norm * vec3(2.0) - vec3(1.0));
       normal = wNormal;
    #endif

Here was OcclusionParallax.glsllib :

All i added are really only TextureArray functions as i remember.

(please note this if(isnan(numLayers)){ is no needed, it was just quick fix for wireframes)

/**
*   Occlusion Parallax Mapping
*   The implementation is based on this: https://learnopengl.com/Advanced-Lighting/Parallax-Mapping
*
*       - Riccardo Balbo
*/

#ifndef _OCCLUSION_PARALLAX_
    #define _OCCLUSION_PARALLAX_
     
    #ifndef Texture_sample
        #define Texture_sample texture
    #endif

    // #define HEIGHT_MAP R_COMPONENT
    // #define DEPTH_MAP R_COMPONENT
    
    #define R_COMPONENT 0
    #define G_COMPONENT 1
    #define B_COMPONENT 2    
    #define A_COMPONENT 3

    #if !defined(HEIGHT_MAP) && !defined(DEPTH_MAP)
        #define HEIGHT_MAP R_COMPONENT
    #endif
    
    #ifdef DEPTH_MAP
        #define HEIGHT_MAP DEPTH_MAP
    #endif

     struct _ParallaxData{
        vec2 deltaTexCoords;
        float layerDepth;
    } ParallaxData;

    float lastParallaxDepth = 0;

    void Parallax_initFor(in vec3 viewDir,in float heightScale){
        const float minLayers = 8.;
        const float maxLayers = 64.;

        float numLayers = mix(maxLayers, minLayers, abs(dot(vec3(0.0, 0.0, 1.0), viewDir))); 

        //fix freeze, np dla wireframe on.
        if(isnan(numLayers)){
            numLayers = minLayers;
        }

        vec2 P = viewDir.xy / viewDir.z * heightScale;

        ParallaxData.layerDepth = 1.0 / numLayers;
        ParallaxData.deltaTexCoords = P / numLayers;
    }

    float _Parallax_selectDepth(in vec4 d){
        float depth;
        
        #if HEIGHT_MAP==R_COMPONENT
            depth=d.r;
        #elif HEIGHT_MAP==G_COMPONENT
            depth=d.g;
        #elif HEIGHT_MAP==B_COMPONENT
            depth=d.b;
        #else 
            depth=d.a;            
        #endif

        #ifndef DEPTH_MAP
            depth=1.-depth;
        #endif

        return depth;
    }

    float _Parallax_sampleDepth(in sampler2D depthMap,in vec2 uv){
        vec4 d=Texture_sample(depthMap,uv);
        return _Parallax_selectDepth(d);
    }

    void Parallax_displaceCoords(inout vec2 texCoords,in sampler2D depthMap){
        vec2 currentTexCoords = texCoords;
        float currentDepthMapValue = (_Parallax_sampleDepth(depthMap, currentTexCoords));
        float currentLayerDepth = 0.0;

        while(currentLayerDepth < currentDepthMapValue){
            currentTexCoords -= ParallaxData.deltaTexCoords;
            currentDepthMapValue = (_Parallax_sampleDepth(depthMap, currentTexCoords));
            currentLayerDepth += ParallaxData.layerDepth;
        }

        vec2 prevTexCoords = currentTexCoords + ParallaxData.deltaTexCoords;
        float afterDepth = currentDepthMapValue - currentLayerDepth;
        float beforeDepth = (_Parallax_sampleDepth(depthMap, prevTexCoords)) - currentLayerDepth + ParallaxData.layerDepth;

        float weight = afterDepth / (afterDepth - beforeDepth);
        lastParallaxDepth = currentDepthMapValue;
        texCoords = prevTexCoords * weight + currentTexCoords * (1.0 - weight);
    }

    float _Parallax_TextureArray_sampleDepth(in sampler2DArray depthMap,in vec2 uv, in float initialParallaxHeight, in int index){
        vec4 d = texture2DArray(depthMap, vec3(uv, index) );
        return initialParallaxHeight + _Parallax_selectDepth(d);
    }

    void Parallax_TextureArray_displaceCoords(inout vec2 texCoords,in sampler2DArray depthMap, in float initialParallaxHeight, in int index){
        vec2 currentTexCoords = texCoords;
        float currentDepthMapValue = (_Parallax_TextureArray_sampleDepth(depthMap, currentTexCoords, initialParallaxHeight, index));
        float currentLayerDepth = 0.0;
        lastParallaxDepth = currentDepthMapValue;

        while(currentLayerDepth < currentDepthMapValue){
            currentTexCoords -= ParallaxData.deltaTexCoords;
            currentDepthMapValue = (_Parallax_TextureArray_sampleDepth(depthMap, currentTexCoords, initialParallaxHeight, index));
            currentLayerDepth += ParallaxData.layerDepth;
        }

        vec2 prevTexCoords = currentTexCoords + ParallaxData.deltaTexCoords;
        float afterDepth = currentDepthMapValue - currentLayerDepth;
        float beforeDepth = (_Parallax_TextureArray_sampleDepth(depthMap, prevTexCoords, initialParallaxHeight, index)) - currentLayerDepth + ParallaxData.layerDepth;

        float weight = afterDepth / (afterDepth - beforeDepth);
        texCoords = prevTexCoords * weight + currentTexCoords * (1.0 - weight);
    }
#endif

[yaRnMcDonuts]

What about "ifdef parallax_#index -> dont do triplannar, indef parallax_#index -> do triplannar" ?

I'm thinking I will make both a TriPlanar and Parallax define, and if both are set true for the same texture, it will probably just disable parallax (until when/if we're able to implement both on the same texture). And (until then) I will document the behavior and let users know that both shouldn't be enabled for a single texture slot. I think it is best to have triplanar take precedence over parallax if both do get enabled, since ignoring the parallax map will not create as any artifcats, while ignoring triplanar for a vertically projected texture will cause many artifacts, so its best to just ignore parallax in case both accidentally get set I think. But the important part will be documenting and letting users know to not enable both together in the first place.

unreal is right, i mean not every texture should use it, for example i planned grassy terrain be without it, while where are more blank areas to use it.

yes I'm thinking the optimal situation would be to have 1-2 triplanar textures on a terrain, and then that would easily make room for another 1-10 extra non-tri planar textures on flat areas, and should still have room to enable parallax on 1 or 2 of those flat textures. (so before enabling any parallax, that would get you a full 12 slot terrain with ~48 texture reads coming from 2 triplanar texture slots (18 texture reads) as well as 10 non triplanar texture slots (30 texture reads) all having normalParallaxMap, baseColormap, and metallicRoughnessAoEiMap.

And that will be faster than my current implementation where I can have ~ 6 texture layers per terrain (with triplanar enabled for everything even where not needed) and no parallax , which ends up being~ 54 texture reads, from 6 (texture slots) X 3 (all triplanar) x 3 (normalMap, baseColorMap, metallicRoughnessAoEiMap).

[pspeed42]

I haven't looked at the code but is your intent to have the layers/triplanar directions interact with one another? Like, if the depth is really deep on one layer then a lower one might show through? Or is each texture set essentially independently calculated and the final pixel color mixed based on some factor?

[yaRnMcDonuts]

Or is each texture set essentially independently calculated and the final pixel color mixed based on some factor?

Each texture slot does its texture reads and calculates an indpendent albedo/normal/roughness/ao/metallic value per layer (and will use an altered texCoord for just that layer's texture reads if parallax or triplanar are enabled for that layer), and then those values immediately get mixed into the final albedo/normal/roughness/etc values based on the alphaBlend value pulled from alphaMap, meaning the final layers always overwrite previous layer as it is now. And then the lighting calculation only happens once after the pre-lighting values have been blended for every texture set,

Like, if the depth is really deep on one layer then a lower one might show through?

So I think (if Im understanding correctly) this should be possible to do for layers/texture slots that have a heightMap, even if they don't have parallax enabled. I think I recall watching a video or reading an article showing how to do something that sounds like what you're describing, but can't find it right now. Although I think it should work to just take the depth value from each layer's heightMap and combine that with the alphaBlend value somehow when doing the blending, and that wouldn't even require an extra texture read if parallaxMap is disabled and heightMap is packed into the normalMap. But I haven't looked into this much yet to know if its that simple.

[oxplay2]

I'm thinking I will make both a TriPlanar and Parallax define, and if both are set true for the same texture, it will probably just disable parallax (until when/if we're able to implement both on the same texture). And (until then) I will document the behavior and let users know that both shouldn't be enabled for a single texture slot. I think it is best to have triplanar take precedence over parallax if both do get enabled, since ignoring the parallax map will not create as any artifcats, while ignoring triplanar for a vertically projected texture will cause many artifacts, so its best to just ignore parallax in case both accidentally get set I think. But the important part will be documenting and letting users know to not enable both together in the first place.

Sounds like a good plan.

btw. for a parallax(each slot) it would also be nice to add "heightOffset" param and "heightScale" param (with default values ofc)

When i were using many cc0 textures, their heightmap(parallax) had different brightness level and needed adjust.

[oxplay2]

@yaRnMcDonuts

I spent some time trying to optimize terrain.

I noticed another way to optimize Parallax Occlusion.
Mean its what described before, but i implemented this.

its basically:

    Parallax_initValues()
    float factor = 1.0 - clamp(camDist/PARALLAX_LOD_DISTANCE, 0.0, 1.0);
    ParallaxConfig.intensity *= factor;
    ParallaxConfig.minLayers = clamp(factor*ParallaxConfig.minLayers, 4., 8.);
    ParallaxConfig.maxLayers = clamp(factor*ParallaxConfig.maxLayers, 16., 64.);

Where ParallaxConfig.minLayers and ParallaxConfig.maxLayers were used to optimized based on View Angle already. But they were not optimized based on camera distance. Ofc every game camera distance might be different thats why i provided this as Matdef Parameter.

And it increased my FPS arround 150 -> 190 fps. There are not much visual difference, depends on distance param ofc.

Another approach (but i still need to test it well) that might "fix" tri-plannar is just additionally lower intensity or layers amount based on Terrain normal. Here example:

    //intensity based on Y normal
    ParallaxConfig.intensity *= clamp(normal.y, 0.0, 1.0); // intensity based on Y value

Could also do same about min/max layers, but i would need verify here since im not using tri-plannar.

Oh btw. here updated glsllib (you can use ParallaxConfig and ParallaxOutputs struct values - i pulled them from functions)

also added Parallax_initValues() to call before any param changes.

/**
*   Occlusion Parallax Mapping
*   The implementation is based on this: https://learnopengl.com/Advanced-Lighting/Parallax-Mapping
*
*       - Riccardo Balbo
*/

#ifndef _OCCLUSION_PARALLAX_
    #define _OCCLUSION_PARALLAX_
     
    #ifndef Texture_sample
        #define Texture_sample texture
    #endif

    // #define HEIGHT_MAP R_COMPONENT
    // #define DEPTH_MAP R_COMPONENT
    
    #define R_COMPONENT 0
    #define G_COMPONENT 1
    #define B_COMPONENT 2    
    #define A_COMPONENT 3

    #if !defined(HEIGHT_MAP) && !defined(DEPTH_MAP)
        #define HEIGHT_MAP R_COMPONENT
    #endif
    
    #ifdef DEPTH_MAP
        #define HEIGHT_MAP DEPTH_MAP
    #endif

     struct _ParallaxData{
        vec2 deltaTexCoords;
        float layerDepth;
    } ParallaxData;

     struct _ParallaxConfig{
        float intensity;
        float initialParallaxDepth;
        float minLayers;
        float maxLayers;
    } ParallaxConfig;

     struct _ParallaxOutputs{
        float lastParallaxDepth;
        float afterDepth;
        float beforeDepth;
        float weight;
        float numLayers;
    } ParallaxOutputs;

    void Parallax_initValues(){
        ParallaxConfig.intensity = 1.0;
        ParallaxConfig.initialParallaxDepth = 0.0;
        ParallaxConfig.minLayers = 8.0;
        ParallaxConfig.maxLayers = 64.0;
        ParallaxOutputs.lastParallaxDepth = 0;
        ParallaxOutputs.afterDepth = 0.0;
        ParallaxOutputs.beforeDepth = 0.0;
        ParallaxOutputs.weight = 0.0;
        ParallaxOutputs.numLayers = 8.;
    }

    void Parallax_initFor(in vec3 viewDir,in float heightScale){
        ParallaxOutputs.numLayers = mix(ParallaxConfig.maxLayers, ParallaxConfig.minLayers, abs(dot(vec3(0.0, 0.0, 1.0), viewDir)));

        //fix freeze, np dla wireframe on.
        if(isnan(ParallaxOutputs.numLayers)){
            ParallaxOutputs.numLayers = ParallaxConfig.minLayers;
        }

        vec2 P = viewDir.xy / viewDir.z * heightScale;

        ParallaxData.layerDepth = 1.0 / ParallaxOutputs.numLayers;
        ParallaxData.deltaTexCoords = P / ParallaxOutputs.numLayers;
    }

    float _Parallax_selectDepth(in vec4 d){
        float depth;
        
        #if HEIGHT_MAP==R_COMPONENT
            depth=d.r;
        #elif HEIGHT_MAP==G_COMPONENT
            depth=d.g;
        #elif HEIGHT_MAP==B_COMPONENT
            depth=d.b;
        #else 
            depth=d.a;            
        #endif

        #ifndef DEPTH_MAP
            depth=1.-depth;
        #endif

        return depth;
    }

    float _Parallax_sampleDepth(in sampler2D depthMap,in vec2 uv){
        vec4 d=Texture_sample(depthMap,uv);
        return (ParallaxConfig.initialParallaxDepth + _Parallax_selectDepth(d)) * ParallaxConfig.intensity;
    }

    void Parallax_displaceCoords(inout vec2 texCoords,in sampler2D depthMap){
        vec2 currentTexCoords = texCoords;
        float currentDepthMapValue = (_Parallax_sampleDepth(depthMap, currentTexCoords));
        float currentLayerDepth = 0.0;

        while(currentLayerDepth < currentDepthMapValue){
            currentTexCoords -= ParallaxData.deltaTexCoords;
            currentDepthMapValue = (_Parallax_sampleDepth(depthMap, currentTexCoords));
            currentLayerDepth += ParallaxData.layerDepth;
        }

        vec2 prevTexCoords = currentTexCoords + ParallaxData.deltaTexCoords;
        ParallaxOutputs.afterDepth = currentDepthMapValue - currentLayerDepth;
        ParallaxOutputs.beforeDepth = (_Parallax_sampleDepth(depthMap, prevTexCoords)) - currentLayerDepth + ParallaxData.layerDepth;

        ParallaxOutputs.weight = ParallaxOutputs.afterDepth / (ParallaxOutputs.afterDepth - ParallaxOutputs.beforeDepth);
        ParallaxOutputs.lastParallaxDepth = currentDepthMapValue;
        texCoords = prevTexCoords * ParallaxOutputs.weight + currentTexCoords * (1.0 - ParallaxOutputs.weight);
    }

    float _Parallax_TextureArray_sampleDepth(in sampler2DArray depthMap,in vec2 uv, in int index){
        vec4 d = texture2DArray(depthMap, vec3(uv, index) );
        return (ParallaxConfig.initialParallaxDepth + _Parallax_selectDepth(d)) * ParallaxConfig.intensity;
    }

    void Parallax_TextureArray_displaceCoords(inout vec2 texCoords,in sampler2DArray depthMap, in int index){
        vec2 currentTexCoords = texCoords;
        float currentDepthMapValue = (_Parallax_TextureArray_sampleDepth(depthMap, currentTexCoords, index));
        float currentLayerDepth = 0.0;
        ParallaxOutputs.lastParallaxDepth = currentDepthMapValue;

        while(currentLayerDepth < currentDepthMapValue){
            currentTexCoords -= ParallaxData.deltaTexCoords;
            currentDepthMapValue = (_Parallax_TextureArray_sampleDepth(depthMap, currentTexCoords, index));
            currentLayerDepth += ParallaxData.layerDepth;
        }

        vec2 prevTexCoords = currentTexCoords + ParallaxData.deltaTexCoords;
        ParallaxOutputs.afterDepth = currentDepthMapValue - currentLayerDepth;
        ParallaxOutputs.beforeDepth = (_Parallax_TextureArray_sampleDepth(depthMap, prevTexCoords, index)) - currentLayerDepth + ParallaxData.layerDepth;

        ParallaxOutputs.weight = ParallaxOutputs.afterDepth / (ParallaxOutputs.afterDepth - ParallaxOutputs.beforeDepth);
        texCoords = prevTexCoords * ParallaxOutputs.weight + currentTexCoords * (1.0 - ParallaxOutputs.weight);
    }
#endif

[yaRnMcDonuts]

So I had mentioned implementing the Oclusion-Parallax into PBRLighting.j3md as well, but am not sure if it will really be useful or not.

Im also far from an exprt on parallax-mapping, and usually just setup my materials with higher-intensity normal maps. So prior to trying to implement it into the terrain shaders, I have never spent anytime researching how parallax works as much as I have normal maps.

But now I'm noticing that parallax can never be used with texture atlases, and only works for wrappable textures. I had always just thought that I was noticing a bug with steep-parallax, but the same issue appears to happen with occlusion parallax. The way that parallax offsets the texture coordinates will almost always break the texture atlas and cause improper projection of the textures onto the model.

Is this correct, or can anyone with more experience with how parallax works clarify on this? Thanks

[pspeed42]

Yes, I think that's correct. Texture atlases have to have wide borders between the texture parts to work with any kind of parallax.

[oxplay2]

That is correct based on my knowledge.

Parallax offset UV, so if UV is outside your texture in atlas it will create artifacts ofc.

One of fixes to still use atlas might be "repeat" texture a little on borders.