Using .NET 7 AOT and Interoperability between .NET and Go

[TOC]

Background

Actually, I've been planning to write this article for a while, but I've been quite lazy, so I've kept delaying it.

Recently, updates are happening too quickly and the competition is intense, so taking advantage of the release of .NET 7, I stayed up late to finish this article, hoping to catch up a little bit.

This article mainly introduces how to generate system (Windows) dynamic link libraries in .NET and Go language, and how to reference the functions in these libraries from the code.

In the .NET section, I will explain how to use AOT, reduce binary file size, and use the latest [LibraryImport] to import library functions.

In the Go language section, I will explain how to use GCC to compile Go code and how to import library functions using syscall.

The article will demonstrate the mutual calls between the dynamic link libraries generated by .NET and Go, as well as compare the differences between the two.

The content of this article and the source code can be found at https://github.com/whuanle/csharp_aot_golang. If you find this article helpful, you can give a star on GitHub.

C# Section

Environment Requirements

SDK: .NET 7 SDK, Desktop development with C++ workload.

IDE: Visual Studio 2022

The Desktop development with C++ workload is a toolset that contains C++ development tools, which need to be installed in the Visual Studio Installer, as shown in the red box in the following image.

Create a Console Project

First, create a .NET 7 console project named CsharpAot.

After opening the project, the basic code is shown in the image below:

We will use the following code for testing:

public class Program
{
    static void Main()
    {
        Console.WriteLine("C# Aot!");
        Console.ReadKey();
    }
}

Experience AOT Compilation

For this step, you can refer to the official website for more information:

https://learn.microsoft.com/zh-cn/dotnet/core/deploying/native-aot/

To allow the project to use AOT mode at publish time, you need to add the <PublishAot>true</PublishAot> option in the project file.

Then publish the project using Visual Studio.

Set the project publishing configuration as shown in the image below.

The options for AOT and ProduceSingleFile cannot be used simultaneously because AOT itself produces a single file.

After the configuration is completed, click Publish, then open the Release directory and you will see the files as shown in the image below.

The .exe file is a standalone executable and does not require the .NET Runtime environment, meaning this program can run on other machines that do not have the .NET environment installed.

Next, delete the following three files:

    CsharpAot.exp
    CsharpAot.lib
    CsharpAot.pdb

You can run it just with the .exe file. The others are debug symbols and not necessary.

After keeping the remaining CsharpAot.exe file, start this program:

C# Call Library Functions

The following code example is extracted from the author's open-source project, which encapsulates some interfaces to obtain system resources and quickly integrate Prometheus monitoring.

However, it hasn't been updated for a long time, and the author currently lacks motivation for updates. Readers can click here to learn more about this project:

https://github.com/whuanle/CZGL.SystemInfo/tree/net6.0/src/CZGL.SystemInfo/Memory

Please enable "Allow unsafe code" for the subsequent code.

This subsection demonstrates calling Windows kernel API (Win32 API) using kernel32.dll to call the GlobalMemoryStatusEx function, which retrieves information about the current physical and virtual memory usage of the system.

For Win32 functions used, refer to: https://learn.microsoft.com/zh-cn/windows/win32/api/sysinfoapi/nf-sysinfoapi-globalmemorystatusex

Regarding the way to call dynamic link libraries in .NET, prior to .NET 7, it was done like this:

    [DllImport("Kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    internal static extern Boolean GlobalMemoryStatusEx(ref MemoryStatusExE lpBuffer);

In .NET 7, a new method [LibraryImport] was introduced.

The documentation describes:

Indicates that a source generator should create a function for marshalling arguments instead of relying on the runtime to generate an equivalent marshalling function at run time.

Indicates that the source generator should create a function for marshalling parameters instead of relying on the runtime at runtime to generate an equivalent marshalling function.

In simple terms, when writing code with AOT, if the code references other dynamic link libraries, we need to use [LibraryImport] to bring in these functions.

The author has not tested [DllImport] under AOT. Readers who are interested can try it out.

Create two struct files, MEMORYSTATUS.cs and MemoryStatusExE.cs.

MEMORYSTATUS.cs:

public struct MEMORYSTATUS
{
    internal UInt32 dwLength;
    internal UInt32 dwMemoryLoad;
    internal UInt32 dwTotalPhys;
    internal UInt32 dwAvailPhys;
    internal UInt32 dwTotalPageFile;
    internal UInt32 dwAvailPageFile;
    internal UInt32 dwTotalVirtual;
    internal UInt32 dwAvailVirtual;
}

MemoryStatusExE.cs:

public struct MemoryStatusExE
{
    /// <summary>
    /// The size of the structure in bytes, this member must be set before calling GlobalMemoryStatusEx, which can be processed in advance using the Init method.
    /// </summary>
    /// <remarks>Should use the Init provided by this object instead of using the constructor!</remarks>
    internal UInt32 dwLength;

    /// <summary>
    /// A number between 0 and 100 that specifies the approximate percentage of physical memory in use (0 indicates no memory is in use, 100 indicates memory is full).
    /// </summary>
    internal UInt32 dwMemoryLoad;

    /// <summary>
    /// The actual amount of physical memory in bytes.
    /// </summary>
    internal UInt64 ullTotalPhys;

    /// <summary>
    /// The current available amount of physical memory in bytes. This is the amount of physical memory that can be immediately reused without writing its contents to disk first. It is the sum of the sizes of the standby list, free list, and zero list.
    /// </summary>
    internal UInt64 ullAvailPhys;

    /// <summary>
    /// The current memory limit for the system or current process in bytes, whichever is smaller. For system-wide committed memory limit, call GetPerformanceInfo.
    /// </summary>
    internal UInt64 ullTotalPageFile;

    /// <summary>
    /// The maximum amount of memory that the current process can commit in bytes. This value is less than or equal to the system-wide available commit value. To compute the overall system commitment, call GetPerformanceInfo and subtract the value CommitTotal from CommitLimit.
    /// </summary>
    internal UInt64 ullAvailPageFile;

    /// <summary>
    /// The size, in bytes, of the user-mode portion of the virtual address space for the calling process. This value depends on the process type, processor type, and operating system configuration. For example, for most 32-bit processes on x86 processors, this value is approximately 2 GB, and for 32-bit processes running on a system with a 4 GB switch enabled, running with large address awareness, it is about 3 GB.
    /// </summary>
    internal UInt64 ullTotalVirtual;

    /// <summary>
    /// The amount of memory that is not reserved and uncommitted in bytes in the user-mode portion of the virtual address space of the calling process.
    /// </summary>
    internal UInt64 ullAvailVirtual;

    /// <summary>
    /// Reserved. This value is always 0.
    /// </summary>
    internal UInt64 ullAvailExtendedVirtual;

    internal void Refresh()
    {
        dwLength = checked((UInt32)Marshal.SizeOf(typeof(MemoryStatusExE)));
    }
}

Define the entry point for referencing the library function:

public static partial class Native
{
    /// <summary>
    /// Retrieves information about the current physical and virtual memory usage of the system.
    /// </summary>
    /// <param name="lpBuffer"></param>
    /// <returns></returns>
    [LibraryImport("Kernel32.dll", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    internal static partial Boolean GlobalMemoryStatusEx(ref MemoryStatusExE lpBuffer);
}

Then call the function in Kernel32.dll:

public class Program
{
    static void Main()
    {
        var result = GetValue();
        Console.WriteLine($"当前实际可用内存量：{result.ullAvailPhys / 1000 / 1000}MB");
        Console.ReadKey();
    }
    
    /// <exception cref="Win32Exception"></exception>
    public static MemoryStatusExE GetValue()
    {
        var memoryStatusEx = new MemoryStatusExE();
        // Reinitialize the size of the structure
        memoryStatusEx.Refresh();
        // Refresh the values
        if (!Native.GlobalMemoryStatusEx(ref memoryStatusEx)) throw new Win32Exception("无法获得内存信息");
        return memoryStatusEx;
    }
}

Use AOT to publish the project and execute the CsharpAot.exe file.

Reduce Size

In the previous two examples, we saw that the CsharpAot.exe file is about 3MB, which is still too large. So how can we further reduce the size of the AOT file?

Readers can learn about how to trim programs from here: https://learn.microsoft.com/zh-cn/dotnet/core/deploying/trimming/trim-self-contained

It should be noted that trimming is not that simple; it involves complex configurations, and some options cannot be used simultaneously. The effects of each option can be quite confusing for developers.

After extensive testing, the author selected the following configurations that achieve good trimming results for readers to test.

First, add a library:

<ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" />
</ItemGroup>

Next, add the following options in the project file:

    <!--AOT related-->
    <PublishAot>true</PublishAot>    
    <TrimMode>full</TrimMode>
    <RunAOTCompilation>True</RunAOTCompilation>
    <PublishTrimmed>true</PublishTrimmed>
    <TrimmerRemoveSymbols>true</TrimmerRemoveSymbols>
    <PublishReadyToRunEmitSymbols>false</PublishReadyToRunEmitSymbols>
    <DebuggerSupport>false</DebuggerSupport>
    <EnableUnsafeUTF7Encoding>true</EnableUnsafeUTF7Encoding>
    <InvariantGlobalization>true</InvariantGlobalization>
    <HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
    <MetadataUpdaterSupport>true</MetadataUpdaterSupport>
    <UseSystemResourceKeys>true</UseSystemResourceKeys>
    <IlcDisableReflection>true</IlcDisableReflection>

Finally, publish the project.

Surprisingly! The generated executable file is only 1MB and can run normally.

Author's note: Although the AOT file looks small now, if libraries such as HttpClient, System.Text.Json, etc. are used, even if only one or two functions are utilized, the final AOT file size will be significantly larger due to the inclusion of these libraries and their dependencies.

Therefore, if your project uses other NuGet packages, don't expect the generated AOT file to be significantly smaller!

C# Export Functions

This step can be learned in more detail from Shi Zong’s blog: https://www.cnblogs.com/InCerry/p/CSharp-Dll-Export.html

PS: Shi Zong is indeed very impressive.

In C language, the format to export a function can look like this:

// MyCFuncs.h
#ifdef __cplusplus
extern "C" {  // only need to export C interface if
              // used by C++ source code
#endif

__declspec( dllimport ) void MyCFunc();
__declspec( dllimport ) void AnotherCFunc();

#ifdef __cplusplus
}
#endif

Once the code is compiled, we can reference the generated library file and call the MyCFunc and AnotherCFunc methods.

If the functions are not exported, other programs will not be able to call the functions inside the library file.

Because .NET 7's AOT has made many improvements, .NET programs can also export functions.

Create a new project named CsharpExport, and we will write our dynamic link library in this project next.

。

Add a file named CsharpExport.cs with the following content:

using System.Runtime.InteropServices;

namespace CsharpExport
{
    public class Export
    {
        [UnmanagedCallersOnly(EntryPoint = "Add")]
        public static int Add(int a, int b)
        {
            return a + b;
        }
    }
}

Then, add the PublishAot option in the .csproj file.

Publish the project using the following command to generate the link library:

 dotnet publish -p:NativeLib=Shared -r win-x64 -c Release

The size appears to be relatively large. To further reduce the size, we can add the following configuration in CsharpExport.csproj to generate a smaller executable file.

		<!-- AOT related -->
		<PublishAot>true</PublishAot>
		<TrimMode>full</TrimMode>
		<RunAOTCompilation>True</RunAOTCompilation>
		<PublishTrimmed>true</PublishTrimmed>
		<TrimmerRemoveSymbols>true</TrimmerRemoveSymbols>
		<PublishReadyToRunEmitSymbols>false</PublishReadyToRunEmitSymbols>
		<DebuggerSupport>false</DebuggerSupport>
		<EnableUnsafeUTF7Encoding>true</EnableUnsafeUTF7Encoding>
		<InvariantGlobalization>true</InvariantGlobalization>
		<HttpActivityPropagationSupport>false</HttpActivityPropagationSupport>
		<MetadataUpdaterSupport>true</MetadataUpdaterSupport>
		<UseSystemResourceKeys>true</UseSystemResourceKeys>
		<IlcDisableReflection>true</IlcDisableReflection>

C# Calls the AOT Generated by C#

In this section, the CsharpAot project will call the dynamic link library generated by CsharpExport.

Copy CsharpExport.dll to the CsharpAot project and set it to Copy Always.

In the CsharpAot's Native directory, add the following:

    [LibraryImport("CsharpExport.dll", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.I4)]
    internal static partial Int32 Add(Int32 a, Int32 b);

Then use it in the code:

    static void Main()
    {
        var result = Native.Add(1, 2);
        Console.WriteLine($"1 + 2 = {result}");
        Console.ReadKey();
    }

Start debugging in Visual Studio:

It runs normally.

Next, publish the CsharpAot project as AOT and execute it again:

It can be seen that .NET AOT calling .NET AOT code is functioning without any issues.

Golang Section

To generate Windows dynamic link libraries using Go, GCC needs to be installed, and the code must be compiled with GCC to generate the corresponding platform files.

Install GCC

You need to install GCC 10.3; using a newer GCC version may result in compilation failure for Go code.

Visit the TDM-GCC official website to install GCC with this tool. The official site is:

https://jmeubank.github.io/tdm-gcc/download/

After downloading, follow the prompts to install it.

Then, add the environment variable:

D:\TDM-GCC-64\bin

Run gcc -v to check if the installation was successful and that the version is correct.

Go Export Function

The knowledge point in this section is cgo, and readers can learn more about it here:

https://www.programmerall.com/article/11511112290/

Create a new Go project:

Create a main.go file with the following content:

package main

import (
	"fmt"
)
import "C"

//export Start
func Start(arg string) {
	fmt.Println(arg)
}

// No purpose
func main() {
}

In Golang, to export a function from this file, you need to add import "C" on a separate line.

//export {Function Name} indicates the function to be exported, note that there is no space between // and export.

Compile main.go into a dynamic link library:

go build -ldflags "-s -w" -o main.dll -buildmode=c-shared main.go

It must be noted that the size of the file compiled by Go is indeed smaller than that of .NET AOT.

Previously, the author demonstrated that .NET AOT could call .NET AOT; however, can Go call Go?

The answer is: No.

This is because the dynamic link library compiled by Go inherently carries runtime; calling main.dll from Go will result in an exception.

Such cases can be checked through an issue in the official Go repository: https://github.com/golang/go/issues/22192

At this time, .NET gets +1.

Although Go cannot call itself, Go can call .NET. This will be introduced later in the article.

Even though Go cannot call itself, we will continue to complete the code for further demonstration.

Here is an example of calling a function through a dynamic link library in Go:

func main() {
	maindll := syscall.NewLazyDLL("main.dll")
	start := maindll.NewProc("Start")

	var v string = "Test Code"
	var ptr uintptr = uintptr(unsafe.Pointer(&v))
	start.Call(ptr)
}

After executing the code, an error will occur:

.NET C# and Golang Mutual Calls

C# Calls Golang

Copy the main.dll file into the CsharpAot project and set it to Copy Always.

Then, add the following code in Native:

    [LibraryImport("main.dll", SetLastError = true)]
    internal static partial void Start(IntPtr arg);

Call the function in main.dll:

    static void Main()
    {
        string arg = "Let Go run";
        // Convert the unmanaged memory string to pointer
        IntPtr concatPointer = Marshal.StringToHGlobalAnsi(arg);
        Native.Start(concatPointer);
        Console.ReadKey();
    }

In .NET, string is a reference type, while in Go, strings are value types; what result will this code produce?

The output result is a long number.

The author is not quite sure about the internal principles of Golang, and it is uncertain whether this number means that .NET string passed as a pointer address, which Go printed as a string.

Since the internal handling of char and string differs among C, Go, .NET, and other languages, this difference in passing leads to unexpected results.

Next, we will modify the Start function in the main.go file:

//export Start
func Start(a,b int) int{
	return a+b
}

Then, execute the command to regenerate the dynamic link library:

go build -ldflags "-s -w" -o main.dll -buildmode=c-shared main.go

Copy the main.dll file into the CsharpAot project, and modify the Start function reference to:

    [LibraryImport("main.dll", SetLastError = true)]
    [return: MarshalAs(UnmanagedType.I4)]
    internal static partial Int32 Start(int a, int b);

Execute the code calling the Start function:

    static void Main()
    {
        var result = Native.Start(1, 2);
        Console.WriteLine($"1 + 2 = {result}");
        Console.ReadKey();
    }

Golang Calls C#

Copy the CsharpExport.dll file into the Go project.

Modify the code in main to:

func main() {
	maindll := syscall.NewLazyDLL("CsharpExport.dll")
	start := maindll.NewProc("Add")

	var a uintptr = uintptr(1)
	var b uintptr = uintptr(2)
	result, _, _ := start.Call(a, b)

	fmt.Println(result)
}

Change the parameters to 1 and 9 and execute again:

Others

In this article, the author demonstrated .NET AOT; although the simple examples appear to be functioning normally and the size is sufficiently small, if actual business logic codes are included, the final generated AOT files can be large.

For instance, if the project uses the HttpClient library, a large number of dependency files will be included, resulting in a large AOT file.

In .NET libraries, it is often designed with numerous overloads, multiple variants of the same code, and excessively long function call chains, which can bloat the generated AOT file.

Currently, ASP.NET Core does not support AOT, which is another issue.

In the C# section, the author demonstrated how to call system interfaces using C#; readers can refer to pinvoke: http://pinvoke.net/

This library wraps system interfaces, relieving developers from needing to implement everything themselves; it allows for easy calls to system interfaces. For instance, the author recently wrote a MAUI project controlling desktop windows through the Win32 API, using pinvoke to simplify extensive code.

This article was rushed and written overnight by the author. Due to limited ability, there may be errors; please feel free to offer corrections.