Unexpected output from Hello Triangle vulkan-tutorial

[JesseRMeyer]

Instead of filing an issue, I wanted to clarify what the expected output of the now well know vulkan-tutorial (https://vulkan-tutorial.com) should be on MoltenVK nearly* out of the box. I suspect to have tripped over either a MoltenVK or Intel GFX driver bug. Fortunately, the official MoltenVK cube program seems to work as expected, which presumably exercises the foundational aspects of vulkan->metal wrapper. So finding an even more minimal program with unexpected output caught me by surprise.

Edit: I've tested with the following validation features enabled, with no difference (no warnings emitted, etc): BEST_PRACTICES, GPU_ASSISTED_RESERVE_BINDING_SLOT, DEBUG_PRINTF, SYNCHRONIZATION_VALIDATION. (Not that DEBUG_PRINTF would work here anyway, unless I made a configuration mistake due to a misunderstanding).

What I expected to see:

What I saw:

Nearly* : I had to make two simple adjustments to the code listed here to appease the validation layers as per the MoltenVK documentation: code

extensions.push_back(VK_KHR_GET_PHYSICAL_DEVICE_PROPERTIES_2_EXTENSION_NAME); //NOTE(Jesse): Added
...
const std::vector<const char*> deviceExtensions = {
    VK_KHR_SWAPCHAIN_EXTENSION_NAME,
    "VK_KHR_portability_subset" //NOTE(Jesse): Added, was unable to find the define directly.
};

The vertex and fragment shader are as follows in GLSL.

#version 450

layout(location = 0) in vec3 fragColor;

layout(location = 0) out vec4 outColor;

void main() {
    outColor = vec4(fragColor, 1.0);
}

#version 450

layout(location = 0) out vec3 fragColor;

vec2 positions[3] = vec2[](
    vec2(0.0, -0.5),
    vec2(0.5, 0.5),
    vec2(-0.5, 0.5)
);

vec3 colors[3] = vec3[](
    vec3(1.0, 0.0, 0.0),
    vec3(0.0, 1.0, 0.0),
    vec3(0.0, 0.0, 1.0)
);

void main() {
    gl_Position = vec4(positions[gl_VertexIndex], 0.0, 1.0);
    fragColor = colors[gl_VertexIndex];
}

Curious, I captured the metal frame render via Xcode.

Notice how the vertices are being correctly accessed and stored as a direct function of gl_VertexIndex, but the color is not. Oddly, the last vertex appears to get the first color, although this could be coincidence, and all prior vertices get some default 0.0 (not even a possible choice in the array), implying the zero initialization of main0_out out = {}; was never overwritten.

Here is the final transformation for both shaders to Metal. I don't see anything glaringly wrong. We can see that flipping the y dimension for the output vertex is correctly being applied. This would seem to suggest that array access via gl_VertexIndex works, and that accesses to the store variable also work. But for some inexplicable reason, this is violated for out.fragColor. I think we can all agree that the output is at least unexpected, but I'm not sure how to continue the investigation from here lacking shader printf / RenderDoc support.

#pragma clang diagnostic ignored "-Wmissing-prototypes"
#pragma clang diagnostic ignored "-Wmissing-braces"

#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

template<typename T, size_t Num>
struct spvUnsafeArray
{
    T elements[Num ? Num : 1];
    
    thread T& operator [] (size_t pos) thread
    {
        return elements[pos];
    }
    constexpr const thread T& operator [] (size_t pos) const thread
    {
        return elements[pos];
    }
    
    device T& operator [] (size_t pos) device
    {
        return elements[pos];
    }
    constexpr const device T& operator [] (size_t pos) const device
    {
        return elements[pos];
    }
    
    constexpr const constant T& operator [] (size_t pos) const constant
    {
        return elements[pos];
    }
    
    threadgroup T& operator [] (size_t pos) threadgroup
    {
        return elements[pos];
    }
    constexpr const threadgroup T& operator [] (size_t pos) const threadgroup
    {
        return elements[pos];
    }
};

constant spvUnsafeArray<float2, 3> _20 = spvUnsafeArray<float2, 3>({ float2(0.0, -0.5), float2(0.5), float2(-0.5, 0.5) });
constant spvUnsafeArray<float3, 3> _29 = spvUnsafeArray<float3, 3>({ float3(1.0, 0.0, 0.0), float3(0.0, 1.0, 0.0), float3(0.0, 0.0, 1.0) });

struct main0_out
{
    float3 fragColor [[user(locn0)]];
    float4 gl_Position [[position]];
};

vertex main0_out main0(uint gl_VertexIndex [[vertex_id]])
{
    main0_out out = {};
    out.gl_Position = float4(_20[int(gl_VertexIndex)], 0.0, 1.0);
    out.fragColor = _29[int(gl_VertexIndex)];
    out.gl_Position.y = -(out.gl_Position.y);    // Invert Y-axis for Metal
    return out;
}

#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct main0_out
{
    float4 outColor [[color(0)]];
};

struct main0_in
{
    float3 fragColor [[user(locn0)]];
};

fragment main0_out main0(main0_in in [[stage_in]])
{
    main0_out out = {};
    out.outColor = float4(in.fragColor, 1.0);
    return out;
}

C++ code built via: clang++ -std=c++17 vulkan_cpp.cpp -I/usr/local/Cellar/glfw/3.3.6/include/ -I/Users/<redacted>/VulkanSDK/1.2.198.1/macOS/include/ -o v_tutorial -L/usr/local/Cellar/glfw/3.3.6/lib/ -L/Users/<redacted>/VulkanSDK/1.2.198.1/macOS/lib/ -lglfw -lvulkan

Both shaders were compiled following the form glslc <shader file name> -g -O0 -o <shader stage name>.spv

Vulkan API dump: gist here

OSX Monterey 12.1
Intel HD Graphics 5000

[JesseRMeyer]

By changing the fragment shader to .. :

#version 450

layout(location = 0) out vec3 fragColor;

vec2 positions[3] = vec2[](
    vec2(0.0, -0.5),
    vec2(0.5, 0.5),
    vec2(-0.5, 0.5)
);

void main() {
    gl_Position = vec4(positions[gl_VertexIndex], 0.0, 1.0);

    vec3 color = vec3(1.0, 1.0, 1.0);
    if (gl_VertexIndex == 0) {
        color = vec3(1.0, 0.0, 0.0);
    }
    else if (gl_VertexIndex == 1) {
        color = vec3(0.0, 1.0, 0.0);
    }
    else {
        color = vec3(0.0, 0.0, 1.0);
    }

    fragColor = color;
}

.. the magical rainbow triangle appears!

While I don't know if this difference is ultimately caused by MoltenVK or Intel, I'm going to upgrade this discussion to an issue proper here now.