Embedding Unity Build into Uno Platform (Windows and Wasm)

[BertrandDeSaintBezier]

Still new to C#/XAML developement. Unity (the game engine) provides a way of embedding a build into a Windows application via Window Handles (HWND), but I'm still trying to wrap my head around the concept of Handles as a whole.

The code below embeds the Unity build into the entire window, but I would like it to only occupy a portion of it, such as containing it in a grid or canvas/panel - a "child window" of sorts.

Any ideas or pointers (haha) 🤔 ?

public MainPage()
{
    this.InitializeComponent();

    try
    {
        var process = new Process();
        process.StartInfo.WorkingDirectory = "path to working dir...";
        process.StartInfo.FileName = "UnityApp.exe";

        var hWnd = WinRT.Interop.WindowNative.GetWindowHandle(App._window);
        process.StartInfo.Arguments = $"-parentHWND {hWnd.ToInt32()} {Environment.CommandLine}";
        process.StartInfo.UseShellExecute = true;
        process.StartInfo.CreateNoWindow = true;

        process.Start();
        process.WaitForInputIdle();
    }
    catch (Exception ex)
    {
        Console.Write(ex.Message);
    }
}

[jeromelaban]

Thanks for asking! On Windows, through WinAppSDK, Uno is not present at all. You may have more information asking this question in the WinUI repo.

[BertrandDeSaintBezier]

Hey, thanks for the answer ! After doing some more research, a lot of fingers seem to be pointing at SwapChainPanels (Unity is DirectX11), which are implemented in Uno Platform. I've started digging into that to see how they work but examples are sparse.

+I'm not sure I quite understand what you mean by "Uno is not present at all" ?

[jeromelaban]

Uno Platform implements the APIs for WinUI and UWP on all platforms but Windows. This means that when building and running the .Windows head project, Uno's code is never run and the official code from WinAppSDK is used.

[BertrandDeSaintBezier]

Don't know what the likelihood of someone else trying this is, but here's how I got it working in any case.

WinUI Impementation

The functionality is split into across two classes, the first is a UserControl that is in charge of actually instantiating the Unity process as its content, and the second is a helper/utils class that just has references to all the relevant Win32 methods.

There are definitely some improvements that can be made here, but I'd say it's a good start.

using System.Diagnostics;
using System.Reflection;
using Windows.Foundation;
using WinRT.Interop;

public sealed partial class WinUI_UnityWindow : UserControl
{
    #region Win32 Settings
    private const int SWP_NOACTIVATE = 0x0010;
    private const int SW_HIDE = 0;
    private const int SW_SHOW = 5;
    private const int GWLP_USERDATA = -21;
    private const int UNITY_SPLASHDONE = 0x0002;
    private const int HWND_BOTTOM = 0;
    #endregion

    private IntPtr _applicationHwnd = IntPtr.Zero;
    private IntPtr _unityProcessHwnd = IntPtr.Zero;

    private Process? _unityProcess = null;

    public event EventHandler? FinishedLoading;

    public double UnityHeight;
    public double UnityWidth;
    
    public WinUI_UnityWindow()
    {
        this.InitializeComponent();
        StartupUnity();
    }

    private async void StartupUnity()
    {
        _applicationHwnd = WindowNative.GetWindowHandle(App.AppWindow);
        int processId = StartNewUnityProcess();

        _unityProcessHwnd = await FindWindowHandleByProcessId(processId);

        Win32Wrapper.ShowWindow(_unityProcessHwnd, SW_HIDE);
        await WaitForUnitySplash();
        Win32Wrapper.ShowWindow(_unityProcessHwnd, SW_SHOW);

        FinishedLoading?.Invoke(this, EventArgs.Empty);

        EmbedWindow();
    }

    private void EmbedWindow()
    {
        ScaleWindowToElement(_unityProcessHwnd, this);
        Win32Wrapper.SetParent(_unityProcessHwnd, _applicationHwnd);
    }

    private int StartNewUnityProcess()
    {
        _unityProcess = new Process();

        _unityProcess.StartInfo.WorkingDirectory = /*Path to your Unity Directory*/;
        _unityProcess.StartInfo.FileName = "[NAME OF YOUR UNITY BUILD].exe";
        _unityProcess.StartInfo.Arguments = "-popupwindow"; // hides the window's handle
        _unityProcess.StartInfo.UseShellExecute = true;
        _unityProcess.StartInfo.CreateNoWindow = true;

        _unityProcess.Start();
        return _unityProcess.Id;
    }

    private async Task WaitForUnitySplash()
    {
        bool notInitialized;
        do
        {
            // https://learn.microsoft.com/en-gb/windows/win32/api/winuser/nf-winuser-getwindowlongptra?redirectedfrom=MSDN
            var userData = Win32Wrapper.GetWindowLongPtrA(_unityProcessHwnd, GWLP_USERDATA); 
            // https://docs.unity3d.com/Manual/PlayerCommandLineArguments.html
            notInitialized = Convert.ToBoolean(userData & UNITY_SPLASHDONE); 
            await Task.Delay(10);
        } while (notInitialized is false);
    }

    private void ScaleWindowToElement(IntPtr window, FrameworkElement element)
    {
        var viewport = GetAbsoluteTransform(element);
        var setWindowPosition = Win32Wrapper.SetWindowPos(window, 
                                                          HWND_BOTTOM, 
                                                          (int)viewport.Left, 
                                                          (int)viewport.Top, 
                                                          (int)UnityWidth, 
                                                          (int)UnityHeight, 0 | SWP_NOACTIVATE);
        if (setWindowPosition is false)
        {
            throw new Exception("Failed to set window position");
        }
    }

    private Rect GetAbsoluteTransform(FrameworkElement element)
    {
        var absolutePos = element.TransformToVisual(null).TransformPoint(new Point(0, 0));
        return new Rect(absolutePos.X, absolutePos.Y, 0, 0);
    }

    private async Task<IntPtr> FindWindowHandleByProcessId(int processId)
    {
        IntPtr HWND = IntPtr.Zero;
        while (HWND == IntPtr.Zero)
        {
            Win32Wrapper.EnumWindows(new Win32Wrapper.EnumWindowsProc((tophandle, topparamhandle) =>
            {
                _ = Win32Wrapper.GetWindowThreadProcessId(tophandle, out int cur_pid);
                if (cur_pid == processId)
                {
                    if (Win32Wrapper.IsWindowVisible(tophandle))
                    {
                        HWND = tophandle;
                        return false;
                    }
                }

                return true;
            }), IntPtr.Zero);

            await Task.Yield();
        }

        return HWND;
    }

    public void ToggleVisibility(bool visible) => Win32Wrapper.ShowWindow(_unityProcessHwnd, (visible ? SW_SHOW : SW_HIDE));

    public void EmbedUnityViewport()
    {
        if (_unityProcessHwnd == IntPtr.Zero)
        {
            StartupUnity();
        }
        else
        {
            EmbedWindow();
        }
    }
}

Utilities/Win32 Wrapper Class :

using System.Runtime.InteropServices;

public static class Win32Wrapper
{
    [DllImport("user32.dll")]
    public static extern bool EnumWindows(EnumWindowsProc enumProc, IntPtr lParam);
    public delegate bool EnumWindowsProc(IntPtr hWnd, IntPtr lParam);
    // https://learn.microsoft.com/en-us/previous-versions/windows/desktop/legacy/ms633498(v=vs.85)

    [DllImport("user32.dll", SetLastError = true)]
    public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out int lpdwProcessId);
    // https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-getwindowthreadprocessid

    [DllImport("user32.dll")]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool IsWindowVisible(IntPtr hWnd);
    // https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-iswindowvisible

    [DllImport("user32.dll", EntryPoint = "SetWindowPos", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool SetWindowPos(IntPtr hwnd, int hWndInsertAfter, int x, int y, int cx, int cy, int wFlags);
    // https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowpos

    [DllImport("user32.dll", SetLastError = true)]
    public static extern IntPtr SetParent(IntPtr child, IntPtr parent);
    // https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setparent

    [DllImport("user32.dll", SetLastError = true)]
    public static extern IntPtr GetWindowLongPtrA(IntPtr hWnd, int nIndex);
    // https://learn.microsoft.com/en-gb/windows/win32/api/winuser/nf-winuser-getwindowlongptra?redirectedfrom=MSDN

    [DllImport("user32.dll", SetLastError = true)]
    public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);
    // https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-showwindow
 }

Bonus : WASM Impementation

As an added bonus, here's how to do the same thing for WASM :

using System.Runtime.InteropServices;

[Uno.UI.Runtime.WebAssembly.HtmlElement("canvas")]
public sealed partial class Wasm_UnityWindow : UserControl
{
    private readonly KeyValuePair<string, string> _canvasAttributes = new KeyValuePair<string, string>("id", "unity-canvas");

    public Wasm_UnityWindow()
    {
        this.InitializeComponent();
    }

    public void EmbedUnityViewport()
    {
        if (RuntimeInformation.IsOSPlatform(OSPlatform.Create("BROWSER")))
        {
            this.SetHtmlAttribute(_canvasAttributes.Key, _canvasAttributes.Value);
            this.ExecuteJavascript("instantiateUnity();");
        }
    }
}

You just need to call the EmbedUnityViewport() method from somewhere else, such as the Page that the UserControl is being used in.

The corresponding instantiateUnity(); function is defined in a LoadUnity.js file that is placed inside the WasmScripts folder of the Wasm head. This is essentially just a watered down version of what a Unity WebGL build will generate anyway, with one or two changes.

function instantiateUnity() {
    var canvas = document.querySelector("#unity-canvas");

    var buildUrl = "[PATH TO UNITY WEBGL BUILD]";
    var loaderUrl = buildUrl + "/Web.loader.js";
    var config = {
        dataUrl: buildUrl + "/Web.data",
        frameworkUrl: buildUrl + "/Web.framework.js",
        codeUrl: buildUrl + "/Web.wasm",
        streamingAssetsUrl: "StreamingAssets",
        companyName: "MyCompany",
        productName: "MyProduct",
        productVersion: "0.1",
    };

    canvas.style.width = "100%";
    canvas.style.height = "100%"

    var script = document.createElement("script");
    script.src = loaderUrl;
    script.onload = () => {
        createUnityInstance(canvas, config, (progress) => {
            console.log(progress + "%");
        }).then((unityInstance) => {
            console.log("done");
            FinishedLoading();
            window.unityInstance = unityInstance;
        }).catch((message) => {
            alert(message);
        });
    };
    document.body.appendChild(script);
}

In hopes that this is one day useful to someone 😄

-H

[JeBuSBrian]

Thanks for posting this, @BertrandDeSaintBezier. I was looking around for good ways to embed Unity into a desktop app and came across this. I loaded up a basic Uno project and added your code and it works, sorta.

The Unity build runs in the Uno app. Problem is, it always takes up the full window of the Uno app. Is there a good way to make it position and size itself according to the layout in the XAML? My basic understanding is that the app (these days) only has a single hwnd handle, so that's part of why it's taking up the full thing, I imagine. (Used to be easier back in the WinForms days to get separate handles per control.)

I also note that as soon as I move the app via the top window handle, the Unity window stops rendering and it goes back to the XAML.

Any guidance on how to setup the project / XAML / etc to get this going?

[BertrandDeSaintBezier]

Hi @JeBuSBrian

What you can do is use this code inside of a UserControl, and specifiy to the Unity Window to only take up a certain amount of space. You'll notice that there are two Properties : UnityWidth and UnityHeight, which I'm using in the ScaleWindowToElement method. In your UserControl, you could specify an element of a certain size and pass this to the Unity Window.

Concerning your second point, I don't have the issue of the viewer disappearing when you move the app. However, when you Resize the window, you will need manually update the embed with something like EmbedUnityViewport. Something else I should mention which I missed in the original answer is that you do need to fiddle around with some Unity settings. I believe I needed to change a couple of things in Resolution and Presentation.

I hope this helps.

As a side note, the code I'm running now has come a long way since, and is still very much perfectible as I've been learning along the way. I think once it's reached a state where all the kinks are ironed out and it's pretty much perfectly usable as-is, I'll write a more extensive/comprehensive blog post somewhere and link it here.

So please keep the questions and feedback coming, it'll help me figure out what needs clarifying and focus.

[chuckthompsonprofisee]

Anyway you can post the uno project with this code in it as I am trying to do the same thing. well instead of embedding unity, I am embedding putty.exe for a ssh experience. I followed your example, and it works for a second as I see the putty window appear for a second, but as soon as the window that houses the usercontrol gets focus/activated, the embeded exe UI (putty in my example) disappears from the parent handle.