The Chronicles of Futile Efforts: A Tutorial and Resource Guide for Building the Armel Version of .NET Core

2019年12月15日 3738点热度 1人点赞 3条评论

内容目录

[TOC]

First of all, I want to state that I have failed~

I am sharing my progress and experience in the hope of helping others complete the compilation work~

Background: Recently, I took over an embedded device of a certain model from Huawei, which requires setting up a .NET Core environment.

The device has an Armel architecture with a Linux kernel version of 3.10; .NET Core ARM only supports Armhf.

Therefore, the binary files compiled cannot run on this device.

I then attempted to download the source code from Git and manually compile the Armel version of the .NET Core SDK/Runtime.

Thanks to Zhang for providing a lot of reference materials.

1. Before Work Begins

The .NET Core SDK/Runtime is not just one Git repository, but multiple:

CoreCLR: Minimal runtime
CoreFX: Library and framework functionalities
CLI: Core of the .NET SDK
Core-SetUp: Divided into CoreCLR and CoreFX

Core-SetUp ultimately generates https://dotnet.microsoft.com/download#core

Before starting the compilation work, it is necessary to confirm the libraries to be compiled.

Source code address:

https://github.com/dotnet/coreclr/

https://github.com/dotnet/corecfx/

https://github.com/dotnet/cli/

You will need a Linux x86/x64 server with more than 1G of memory, using Ubuntu/Debian as the operating system.

Convenient tools that can be installed:

apt install lrzsz: A tool for transferring files across terminals. If you are using Xshell for remote terminal access, you can directly drag and drop files to upload to the Linux server.

apt install tree: Displays files in a directory in tree format.

2. Compilation Tool Components

Install the necessary toolchain

cmake
llvm-3.9
clang-3.9
lldb-3.9
liblldb-3.9-dev
libunwind8
libunwind8-dev
gettext
libicu-dev
liblttng-ust-dev
libcurl4-openssl-dev
libssl-dev
libkrb5-dev
libnuma-dev (optional, enables NUMA support)

One-click installation command:

sudo apt-get install cmake llvm-3.9 clang-3.9 lldb-3.9 liblldb-3.9-dev libunwind8 libunwind8-dev gettext libicu-dev liblttng-ust-dev libcurl4-openssl-dev libssl-dev libnuma-dev libkrb5-dev

Clone the CoreCLR repository

git clone https://github.com/dotnet/coreclr.git

You can see the cross folder:

[root@instance-wxxixh4k cross]# tree -L 1
.
├── android
├── arm
├── arm64
├── armel
├── build-android-rootfs.sh
├── build-rootfs.sh
├── toolchain.cmake
├── tryrun.cmake
└── x86

Pre-compilation Configuration Check

① Maximum Number of File Handles

Check the maximum number of supported file handles

sysctl fs.file-max

If it shows less than 100000, then you need to set it to a higher number, for example:

# Open file
nano /etc/sysctl.conf
# Add a line
fs.file-max = 100000

Then run

sudo sysctl -p

② Memory Must Be Greater Than 1G

If the server's memory is less than 1G, it will not be able to begin compilation, as the script requires more than 1G of memory to proceed.

Fortunately, whuanle found a solution.

Create a swap partition

sudo dd if=/dev/zero of=/swapfile bs=64M count=16
sudo mkswap /swapfile
sudo swapon /swapfile

bs: size of each partition, count: number of partitions.

After the compilation is complete, remember to delete the swap partition:

sudo swapoff /swapfile
sudo rm /swapfile

3. Compile MUSL

Install the required libraries

sudo apt-get install qemu qemu-user-static binfmt-support debootstrap

sudo apt-get install binutils-arm-linux-gnueabihf

Download the Git repository

git clone https://github.com/richfelker/musl-cross-make.git

Open the MUSL library directory and create a config.mak file

touch config.mak

In it, add the following content

TARGET = armv7-alpine-linux-musleabihel
OUTPUT = /usr
BINUTILS_CONFIG=--enable-gold=yes

Regarding the config.mak file, the format for the first line is arm[eb]-linux-musleabi[hf]

The CPU architecture/instruction set version can be obtained using the cat /proc/cpuinfo command.

4. Build Armel's Rootfs

sudo ./cross/build-rootfs.sh armel 
# Or
sudo ./cross/build-rootfs.sh armel tizen

After compilation, you can find the corresponding version of rootfs in ./coreclr/cross/rootfs/.

Using the first command, I got the corresponding rootfs, but ultimately ran into issues, and I could not use the sudo ./cross/build-rootfs.sh armel tizen command.

Issues:

umount: /var/test/0828/coreclr/cross/rootfs/armel/bin: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/boot: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/dev: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/etc: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/home: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/lib: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/media: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/mnt: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/opt: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/proc: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/root: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/run: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/sbin: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/srv: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/sys: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/tmp: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/usr: not mounted.
umount: /var/test/0828/coreclr/cross/rootfs/armel/var: not mounted.

However, there were no Error, failed, or abnormal prompts, so I was uncertain whether it had failed. Upon checking the directory, I found that the Rootfs was complete, everything seemed fine.

5. Cross-compile CoreCLR

./build.sh armel debug verbose cross
# Or 
./build.sh armel release verbose cross

Example:

root@ubuntu:/var/netcore/0828/coreclr# ./build.sh armel debug verbose cross
Commencing CoreCLR Repo build
__DistroRid: debian.8-armel
__RuntimeId: debian.8-armel
Setting up directories for build
Checking prerequisites...

After executing the command, if you see Checking prerequisites..., it indicates that you are one step closer to success.

Of course, this error may occur:

dotnet_install: Error: Could not find/download: `.NET Core SDK` with version = 3.0.100-preview6-012264
dotnet_install: Error: Refer to: https://aka.ms/dotnet-os-lifecycle for information on .NET Core support
Failed to install dotnet SDK (exit code '1').
Failed to get PGO data package version.

The reason is that it fails to download the .NET Core SDK.

In the /tmp directory, you can see many caches of failed downloads for the SDK. Because the download sources are abroad, the download speed domestically may be only a few kb.

The files to be downloaded are about 120MB, and the download time can take several hours, which may lead to interruptions or timeouts, thus causing failures.

root@ubuntu:/tmp# ls -lah
total 90M
drwxrwxrwt 10 root    root    4.0K Aug 28 01:48 .
drwxr-xr-x 24 root    root    4.0K Aug 27 09:37 ..
-rw-rw----  1 netdata netdata  240 Aug 20 05:28 as.log
-rw-------  1 root    root     34M Aug 27 06:30 dotnet.9TuQ60VOK
-rw-------  1 root    root    3.2M Aug 28 01:18 dotnet.a3WI49nPx
-rw-------  1 root    root       0 Aug 27 03:30 dotnet.CxR2scA4h
-rw-------  1 root    root       0 Aug 27 01:38 dotnet.DuAMYKCT4
-rw-------  1 root    root     16K Aug 27 01:38 dotnet.gEYeUSXYy
-rw-------  1 root    root       0 Aug 28 00:06 dotnet.HWdDnC5Hl
-rw-------  1 root    root       0 Aug 27 10:03 dotnet.hX7K3ac76
-rw-------  1 root    root       0 Aug 28 00:08 dotnet.ijQuG82B1
-rw-------  1 root    root     34M Aug 28 01:47 dotnet.ImT2CM9gt
-rw-------  1 root    root    3.9M Aug 27 10:03 dotnet.KnQlzjA4g
-rw-------  1 root    root    1.4M Aug 28 01:49 dotnet.o3nnapWHT
-rw-------  1 root    root     14M Aug 27 03:30 dotnet.XKVdUNje0
-rw-------  1 root    root       0 Aug 28 01:47 dotnet.xVEb3C7Bl

Method 1: Get your server to use a VPN

Method 2: Download the package I uploaded

Link: https://eyun.baidu.com/s/3dGnzGC5 Password: GWuV

Put the downloaded package into ./coreclr/.dotnet/ and unpack it.

Then re-execute ./build.sh armel debug verbose cross

Build Successful?

If successful, a prompt will appear.

Build succeeded. 0 Warning(s) 0 Error(s)

Time Elapsed 00:00:00.64 /var/test/0828/coreclr/bin/obj/Linux.armel.Debug /var/test/0828/coreclr Invoking "/var/test/0828/coreclr/src/pal/tools/gen-buildsys-clang.sh" "/var/test/0828/coreclr" 5 "0" armel /var/test/0828/coreclr/src/pal/tools Debug -DCLR_CMAKE_TARGET_OS=Linux -DCLR_CMAKE_PACKAGES_DIR=/var/test/0828/coreclr/.packages -DCLR_CMAKE_PGO_INSTRUMENT=0 -DCLR_CMAKE_OPTDATA_VERSION=99.99.99-master-20190716.1 -DCLR_CMAKE_PGO_OPTIMIZE=1 Unable to locate llvm-ar

Although there was a success prompt, it was too early to be happy.

The official documentation states:

As usual, the resulting binaries will be found in bin/Product/BuildOS.BuildArch.BuildType/

This means that if the build is successful, there will be a folder named [System Type].[Architecture Version].[Build Version] in the bin/Product/ directory.

Open the directory ./bin/Product

drwxr-xr-x 3 root root 4.0K Aug 28 01:12 Linux.armel.Debug
drwxr-xr-x 2 root root 4.0K Aug 28 00:06 Linux.x64.Debug

Well, it did appear!

But...

There are no files under Linux.armel.Debug. If you open the ./coreclr/cross/rootfs/armel directory to see the files inside.

root@ubuntu:/var/test/0828/coreclr/bin/Product# tree -L 3
.
├── Linux.armel.Debug
│   └── x64
├── Linux.armel.Release
│   └── x64
├── Linux.x64.Debug
│   ├── coreconsole
│   ├── corerun
│   ├── createdump
│   ├── crossgen
│   ├── gcinfo
│   │   └── gcinfoencoder.cpp
│   ├── IL
│   │   ├── System.Private.CoreLib.deps.json
│   │   ├── System.Private.CoreLib.dll
│   │   └── System.Private.CoreLib.xml
... ...

Referring to x64, successful compile directories will have these files, but there are no files under armel...

Therefore, I ultimately confirm the failure.

Due to time constraints and complexity, whuanle has already given up on treatment... ...