Swift bindings generator with simple parser and emitter #2525

kotlarmilos · 2024-03-05T10:10:48Z

Description

This PR introduces a toolset for generating C# bindings from Swift interface files. This is the MVP aiming to support the generation of C# bindings for CryptoKit dev templates.

The tooling consists of the following components:

SwiftBindings: A command-line interface with Swift ABI parser, marshalling types, and string-based C# emitter.
SwiftRuntimeLibrary: A library providing projections of common Swift types.

The parser and emitter are modular and implement ISwiftParser and ICSharpEmitter interfaces. The Swift ABI parser can be replaced with a swiftinterface parser, while the string-based emitter can be replaced with an object model-based emitter.

Workflow:

Parse Swift ABI files to aggregate public ABI.
Generate marshalling information if needed.
Emit C# source code using a string-based emitter.

The unit tests verify the XML layout of the type database and various P/Invoke cases. Additionally, CryptoKit dev templates with unsafe (buffer) pointers are implemented.

The SwiftRuntimeLibrary provides runtime marshaling support and projections of common Swift types.

src/SwiftRuntimeLibrary/src/ISwiftObject.cs

src/SwiftRuntimeLibrary/src/SwiftRuntimeLibrary.csproj

src/SwiftRuntimeLibrary/tests/SwiftRuntimeLibrary.Tests.csproj

src/SwiftRuntimeLibrary/src/SwiftCore.xml

src/SwiftRuntimeLibrary/src/ISwiftObject.cs

src/SwiftRuntimeLibrary/src/SwiftCore.xml

…nterface documentation

docs/README.md

src/SwiftRuntimeLibrary/src/SwiftCore.xml

The bindings generator includes a parser, marshaller, and emitter. The parser generates module declarations by consuming an ABI JSON file. The marshaller facilitates type marshalling between C# and Swift, while the emitter is a string-based IndentedTextWriter that outputs C# source code.

kotlarmilos · 2024-03-07T10:22:29Z

This PR has been updated with a bindings generator that includes a simple parser and emitter (refer to the updated description and docs). Please note that this is an MVP for CryptoKit dev templates.

docs/README.md

src/SwiftRuntimeLibrary/src/TypeDatabase.cs

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

This commit introduces support for the UnsafeRawPointer type. It also adds a test case for the existing CryptoKit bindings.

… into swift-bindings/runtime-library

matouskozak

Good job! Looking good and much easier to read through than the original big PR. I put some comments but they are more questions about the future so no need to address them in this PR.

src/Swift.Runtime/src/TypeDatabase.cs

src/Swift.Bindings/src/Model/MethodDecl.cs

src/Swift.Bindings/src/Parser/BindingsGenerator.SwiftABIParser.cs

src/Swift.Runtime/src/ISwiftObject.cs

This commit introduces support for the UnsafeBufferPointer types. It also adds a test case for the CryptoKit bindings.

kotlarmilos · 2024-03-20T17:32:20Z

This PR includes projection tooling skeleton and implements CryptoKit dev templates with unsafe (buffer) pointers. We will move forward with surfacing https://developer.apple.com/documentation/cryptokit/chachapoly enum and https://developer.apple.com/documentation/cryptokit/symmetrickey struct in a subsequent PR.

Please review this PR and provide feedback.

AaronRobinsonMSFT · 2024-03-20T18:19:49Z

.gitignore

+# Swift output files
+*.swiftinterface*
+*.swiftdoc*
+*.swiftmodule*
+*.abi*
+*.swiftsourceinfo*
+*.dylib
+
+# Testing artifacts
+testing/
+src/samples/**/*Bindings.cs


These should all go under artifacts/. In the current experiment form, this seems okay, but prior to merging into dotnet/runtime, all outputs should be under artifacts/.

Oops. Sorry, I thought we were working in a dotnet/runtime fork. Never mind.

docs/functional-outline.png

AaronRobinsonMSFT · 2024-03-20T18:39:10Z

src/Swift.Bindings/src/Model/MethodDecl.cs

+    /// <summary>
+    /// Represents a method declaration.
+    /// </summary>
+    public sealed record MethodDecl


Using records in public APIs hasn't been confirmed as yet. This is likely okay for the current state. Just be aware that if this is going to be publicly consumable, the API review team may push back.

AaronRobinsonMSFT

LGTM

jkotas · 2024-03-21T01:02:48Z

docs/README.md

@@ -44,6 +44,8 @@ The table below lists the Swift types and their corresponding C# types.
 | `Swift.UInt16`                  | `ushort` |
 | `Swift.Int8`                    | `sbyte`  |
 | `Swift.UInt8`                   | `byte`   |
+| `Swift.UnsafePointer`           | `void*`  |


I know we touched on this before - it does not feel right to drop the type information for the strongly typed pointers. Strong typing is an extra level of protection against programming bugs that we are giving away here.

Ok, as we discussed, a thin layer would be useful here.

Types for unsafe pointers have been added. Mutable pointers are projected as structs with generics in C# to address methods overload issues and enable future extensibility.

jkotas · 2024-03-21T01:11:35Z

src/Swift.Runtime/src/Library/UnsafeBufferPointer.cs

+    public unsafe readonly struct UnsafeBufferPointer
+    {
+        private readonly void* _Position;
+        public readonly nint Count;


Swift does not seem to have fields as first-class concept. Their docs only talk about properties: https://docs.swift.org/swift-book/documentation/the-swift-programming-language/properties/

I am wondering whether mapping stored properties to fields in C# is conceptually correct.

Is it a breaking change in Swift to change a stored property to a computed property in non-frozen type? I would expect that it is not a breaking change - can you confirm that?

If computed properties and stored properties are interchangeable in Swift, we should map both to properties in C#. Ie make this public nint Count { get; }; or private nint _count; public nint Count => _count;.

Good catch. This is not a breaking change in Swift. I confirmed it by consuming a library with struct that evolved from

public struct MyType { public var myProperty: Int = 0 public init() {} }

to

public struct MyType { private var _myProperty: Int = 0 public var myProperty: Int { get { _myProperty } set { _myProperty = newValue } } public init() {} }

Buffer pointers are updated with private readonly nint _count; public nint Count => _count;.

IIRC, if a property is stored, you still get getters and setter generated by the compiler. Since BTfS doesn't deal with struct lowering, it maps properties (stored or otherwise) as C# properties.

This commit implements mutable pointers as generics in C# and update CryptoKit test cases. The parser is updated to process Swift generic types and projects them into C# generic types.

jkotas · 2024-03-21T15:36:41Z

src/Swift.Runtime/src/Library/UnsafeBufferPointer.cs

+            _count = count;
+        }
+
+        public void* BaseAddress => _position;


Suggested change

public void* BaseAddress => _position;

public T* BaseAddress => _position;

jkotas · 2024-03-21T15:36:52Z

src/Swift.Runtime/src/Library/UnsafeBufferPointer.cs

+            _count = count;
+        }
+
+        public void* BaseAddress => _position;


Suggested change

public void* BaseAddress => _position;

public T* BaseAddress => _position;

All void* in the strongly typed wrappers should be T*.

jkotas · 2024-03-21T15:42:30Z

src/Swift.Runtime/src/Library/UnsafePointer.cs

+    public unsafe readonly struct UnsafePointer<T>
+    {
+        private readonly void* _rawValue;
+        public UnsafePointer(void* _rawValue)


The argument name should not have underscore. Also, should the argument name match the property name that it corresponds to UnsafePointer(void* poin_rawValue)?

jkotas · 2024-03-21T15:44:18Z

src/Swift.Runtime/src/Library/UnsafePointer.cs

+    // <summary>
+    // Represents Swift UnsafePointer in C#.
+    // </summary>
+    public unsafe readonly struct UnsafePointer<T>


I think it would make sense to have implicit conversions to/from T* for these types.

(And to/from void* for the non-generic variant.)

jkotas · 2024-03-21T15:45:22Z

src/Swift.Runtime/src/Library/UnsafeBufferPointer.cs

+    {
+        private readonly void* _position;
+        private readonly void* _end;
+        public UnsafeRawBufferPointer(void* start, nint count)


Suggested change

public UnsafeRawBufferPointer(void* start, nint count)

public UnsafeRawBufferPointer(void* baseAddress, nint count)

Match property name?

Add SwiftRuntimeLibrary project

23a0642

The SwiftRuntimeLibrary provides runtime marshaling support and projections of common Swift types.

kotlarmilos added the area-SwiftBindings Swift bindings for .NET label Mar 5, 2024

kotlarmilos requested review from rolfbjarne, agocke, jkoritzinsky, vitek-karas, stephen-hawley, AaronRobinsonMSFT and matouskozak March 5, 2024 10:10

kotlarmilos self-assigned this Mar 5, 2024

Add arm64 CI job

abab981

kotlarmilos mentioned this pull request Mar 5, 2024

Implement projection tooling for simple P/Invoke cases #2517

Closed

rolfbjarne reviewed Mar 5, 2024

View reviewed changes

Use BCL managed types and enable nullable reference types

bd9b708

AaronRobinsonMSFT reviewed Mar 5, 2024

View reviewed changes

src/SwiftRuntimeLibrary/src/ISwiftObject.cs Outdated Show resolved Hide resolved

AaronRobinsonMSFT reviewed Mar 5, 2024

View reviewed changes

src/SwiftRuntimeLibrary/src/SwiftCore.xml Outdated Show resolved Hide resolved

kotlarmilos added 2 commits March 6, 2024 10:53

Add Swift type database structure to README.md and add ISwiftObject i…

62bcca6

…nterface documentation

Add public access modifier to SwiftObject

e32d8ee

jkotas reviewed Mar 6, 2024

View reviewed changes

docs/README.md Outdated Show resolved Hide resolved

AaronRobinsonMSFT reviewed Mar 6, 2024

View reviewed changes

src/SwiftRuntimeLibrary/src/SwiftCore.xml Outdated Show resolved Hide resolved

AaronRobinsonMSFT reviewed Mar 6, 2024

View reviewed changes

src/SwiftRuntimeLibrary/src/SwiftCore.xml Outdated Show resolved Hide resolved

kotlarmilos added 3 commits March 6, 2024 20:10

Remove type-specific information from the type database

d213412

Update docs and add HelloWorld sample

aae465e

kotlarmilos changed the title ~~Add SwiftRuntimeLibrary project~~ Swift bindings generator with simple parser and emitter Mar 7, 2024

kotlarmilos mentioned this pull request Mar 7, 2024

Add SyntaxDynamo project #2526

Closed

rolfbjarne reviewed Mar 7, 2024

View reviewed changes

kotlarmilos and others added 2 commits March 7, 2024 12:09

Update docs/README.md

306f45f

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

Update src/SwiftRuntimeLibrary/src/TypeDatabase.cs

a33ce4f

Co-authored-by: Rolf Bjarne Kvinge <rolf@xamarin.com>

kotlarmilos added 4 commits March 8, 2024 12:42

Use deinit calls for Swift projections in C# by default

2002932

Add support for UnsafeRawPointer

aa55c63

This commit introduces support for the UnsafeRawPointer type. It also adds a test case for the existing CryptoKit bindings.

Merge branch 'feature/swift-bindings' of github.com:dotnet/runtimelab…

be8a996

… into swift-bindings/runtime-library

Update dotnet version

f01d13e

matouskozak approved these changes Mar 20, 2024

View reviewed changes

kotlarmilos added 3 commits March 20, 2024 12:12

Add support for UnsafeBufferPointer types

9264f83

This commit introduces support for the UnsafeBufferPointer types. It also adds a test case for the CryptoKit bindings.

Update Swift ABI parser to capture dependencies and C# mapped types

396d395

Update docs with UnsafeBufferPointer section

51ec116

kotlarmilos requested review from rolfbjarne and AaronRobinsonMSFT March 20, 2024 17:20

AaronRobinsonMSFT reviewed Mar 20, 2024

View reviewed changes

docs/functional-outline.png Outdated Show resolved Hide resolved

AaronRobinsonMSFT reviewed Mar 20, 2024

View reviewed changes

AaronRobinsonMSFT approved these changes Mar 20, 2024

View reviewed changes

jkotas reviewed Mar 21, 2024

View reviewed changes

rolfbjarne approved these changes Mar 21, 2024

View reviewed changes

kotlarmilos added 2 commits March 21, 2024 13:07

Implement mutable pointers as generics in C#

03a5b09

This commit implements mutable pointers as generics in C# and update CryptoKit test cases. The parser is updated to process Swift generic types and projects them into C# generic types.

Add P/Invoke bool test case

ba27924

jkotas reviewed Mar 21, 2024

View reviewed changes

Update pointers with implicit operators

8480dbe

stephen-hawley approved these changes Mar 22, 2024

View reviewed changes

kotlarmilos merged commit df0349a into feature/swift-bindings Mar 25, 2024
6 checks passed

kotlarmilos deleted the swift-bindings/runtime-library branch March 25, 2024 09:23

kotlarmilos mentioned this pull request May 10, 2024

Projection tooling for Swift interop in .NET 10 dotnet/runtime#95633

Open

34 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Swift bindings generator with simple parser and emitter #2525

Swift bindings generator with simple parser and emitter #2525

kotlarmilos commented Mar 5, 2024 •

edited

Loading

kotlarmilos commented Mar 7, 2024

matouskozak left a comment

kotlarmilos commented Mar 20, 2024

AaronRobinsonMSFT Mar 20, 2024

AaronRobinsonMSFT Mar 20, 2024

AaronRobinsonMSFT Mar 20, 2024 •

edited

Loading

AaronRobinsonMSFT left a comment

jkotas Mar 21, 2024

kotlarmilos Mar 21, 2024

kotlarmilos Mar 21, 2024

jkotas Mar 21, 2024 •

edited

Loading

kotlarmilos Mar 21, 2024

kotlarmilos Mar 21, 2024

stephen-hawley Mar 22, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

jkotas Mar 21, 2024

	public void* BaseAddress => _position;
	public T* BaseAddress => _position;

	public UnsafeRawBufferPointer(void* start, nint count)
	public UnsafeRawBufferPointer(void* baseAddress, nint count)

Swift bindings generator with simple parser and emitter #2525

Swift bindings generator with simple parser and emitter #2525

Conversation

kotlarmilos commented Mar 5, 2024 • edited Loading

Description

kotlarmilos commented Mar 7, 2024

matouskozak left a comment

Choose a reason for hiding this comment

kotlarmilos commented Mar 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AaronRobinsonMSFT Mar 20, 2024 • edited Loading

Choose a reason for hiding this comment

AaronRobinsonMSFT left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkotas Mar 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kotlarmilos commented Mar 5, 2024 •

edited

Loading

AaronRobinsonMSFT Mar 20, 2024 •

edited

Loading

jkotas Mar 21, 2024 •

edited

Loading