↓Skip to main content
  1. Posts/

Test Driving vcpkg on Apple Silicon

·8 mins·
Table of Contents
Introduction

My personal projects have become more interesting since picking up an M2 MacBook as a daily driver. For the first time in years, I needed to update my own projects to support a new architecture.

Coincidentally, it caught my attention that CLion added support for vcpkg. So I thought this might also be a good opportunity to revisit packaging in general and see what (if anything) has changed recently.

And, I’d really like the ability to develop locally on my Mac as well as remotely on machines of potentially different architectures. Oh, and I’d occassionally like to work on Windows. And it would be nice if I could use any of Clang, GCC, or Visual Studio depending on needs.

Uh oh. There goes my Saturday.

Table Of Contents

How We Got Here #

It all started with the new MacBook. You can produce both x86_64 and arm64 Mach-O binaries:

$ c++ --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ c++ main.cpp -o x64_app -target x86_64-apple-macos10.12
$ lipo -info x64_app
Non-fat file: x64_app is architecture: x86_64

$ c++ main.cpp -o arm_app -target arm64-apple-macos11
$ lipo -info arm_app
Non-fat file: arm_app is architecture: arm64

There’s also a universal binary format which is an archive and not really an executable file itself.

Thanks to the magic of Rosetta 2, you can execute either format:

$ uname -m
arm64

$ sysctl -n machdep.cpu.brand_string
Apple M2 Max

$ ./arm_app
Hello, world!

$ ./x64_app
Hello, world!

This gets even more interesting if you’re using containers. I’m presently running docker on a QEMU VM via colima. Thanks to binfmt_misc you can even run x86_64 containers on top of your aarch64 VM (with the obvious performance penalty):

$ colima status
INFO[0000] colima is running using QEMU
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/justinpratt/.colima/default/docker.sock

$ docker run -q --platform linux/arm64 hello-world | grep arm
    (arm64v8)
    
$ docker run -q --platform linux/amd64 hello-world | grep amd
    (amd64)

So I’m very interested in cross-compiling for these platforms.

CMake-ing It Easy #

Fortunately CMake makes the builds easy as well. They added Apple Silicon support in version 3.19.2.

With a bare-bones CMakeLists:

CMakeLists.txt 
cmake_minimum_required(VERSION 3.19)
project(example LANGUAGES CXX)
add_executable(main main.cpp)
set_property(TARGET main PROPERTY OUTPUT_NAME main)
target_compile_features(main PRIVATE cxx_std_20)

And a presets file that should work on either architecture:

CMakePresets.json 
{
  "version": 5,
  "configurePresets": [
    {
      "name": "linux",
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-D_FORTIFY_SOURCE=3 -fstack-protector-strong -fstack-clash-protection -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast",
        "CMAKE_EXE_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now",
        "CMAKE_SHARED_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now"
      }
    },
    {
      "name": "macos",
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-fstack-protector-strong -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast"
      }
    },
    {
      "name": "windows",
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "/sdl /analyze /analyze:external- /guard:cf /utf-8 /diagnostics:caret /w14165 /w44242 /w44254 /w44263 /w34265 /w34287 /w44296 /w44365 /w44388 /w44464 /w14545 /w14546 /w14547 /w14549 /w14555 /w34619 /w34640 /w24826 /w14905 /w14906 /w14928 /w45038 /W4 /permissive- /volatile:iso /Zc:inline /Zc:preprocessor /Zc:lambda /Zc:__cplusplus /Zc:externConstexpr /Zc:throwingNew /EHsc",
        "CMAKE_EXE_LINKER_FLAGS": "/machine:x64 /guard:cf"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "linux",
      "configurePreset": "linux"
    },
    {
      "name": "macos",
      "configurePreset": "macos"
    },
    {
      "name": "windows",
      "configurePreset": "windows"
    }
  ]
}

It works on macOS…

$ cmake --preset macos && cmake --build --preset macos
-- The CXX compiler identification is AppleClang 14.0.3.14030022
...
[100%] Built target main
$ ./build/macos/main
Hello, world!

…and it works on Linux…

$ docker run -it --rm -v "$(pwd):/usr/src" cplusplus bash
root@9773551ffd01:/usr/src# cmake --preset linux && cmake --build --preset linux
-- The CXX compiler identification is GNU 12.2.0
...
root@9773551ffd01:/usr/src# ./build/linux/main
Hello, world!

…and it works on Windows:

PS C:\Users\justin\example> cmake --preset windows && cmake --build --preset windows
...
MSBuild version 17.5.1+f6fdcf537 for .NET Framework

PS C:\Users\justin\example> .\build\windows\Debug\main.exe
Hello, world!

Nice.

Libraries and Tigers and Bears #

I’ve never been completely happy with my packaging setup. I wanted to take a fresh look at vcpkg now that it seems to be gaining steam.

vcpkg hooks into CMake via the CMAKE_TOOLCHAIN_FILE, similarly to cross-compiling.

I update the source file with an external dependency:

main.cpp 
#include <fmt/format.h>

int main() {
  fmt::print("Hello, world!\n");
  return 0;
}

Add a minimum viable vcpkg.json:

vcpkg.json 
{
  "name": "example",
  "version-semver": "0.1.0",
  "dependencies": [
    {
      "name": "fmt",
      "version>=": "9.1.0#1"
    }
  ],
  "builtin-baseline": "8daf70c56ba9581f5251a5d4675eb771b6b34957"
}

Update CMakePresets.json with the toolchain file:

CMakePresets.json 
{
  "version": 5,
  "configurePresets": [
    {
      "name": "vcpkg",
      "hidden": true,
      "cacheVariables": {
        "CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake"
      }
    },
    {
      "name": "linux",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-D_FORTIFY_SOURCE=3 -fstack-protector-strong -fstack-clash-protection -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast",
        "CMAKE_EXE_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now",
        "CMAKE_SHARED_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now"
      }
    },
    {
      "name": "macos",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-fstack-protector-strong -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast"
      }
    },
    {
      "name": "windows",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "/sdl /analyze /analyze:external- /guard:cf /utf-8 /diagnostics:caret /w14165 /w44242 /w44254 /w44263 /w34265 /w34287 /w44296 /w44365 /w44388 /w44464 /w14545 /w14546 /w14547 /w14549 /w14555 /w34619 /w34640 /w24826 /w14905 /w14906 /w14928 /w45038 /W4 /permissive- /volatile:iso /Zc:inline /Zc:preprocessor /Zc:lambda /Zc:__cplusplus /Zc:externConstexpr /Zc:throwingNew /EHsc",
        "CMAKE_EXE_LINKER_FLAGS": "/machine:x64 /guard:cf"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "linux",
      "configurePreset": "linux"
    },
    {
      "name": "macos",
      "configurePreset": "macos"
    },
    {
      "name": "windows",
      "configurePreset": "windows"
    }
  ]
}

And finally, link against fmt:

CMakeLists.txt 
cmake_minimum_required(VERSION 3.19)
project(example LANGUAGES CXX)
add_executable(main main.cpp)
set_property(TARGET main PROPERTY OUTPUT_NAME main)
target_compile_features(main PRIVATE cxx_std_20)

find_package(fmt CONFIG REQUIRED)
target_link_libraries(main PRIVATE fmt::fmt)

To my delight, this also just works.

$ cmake --preset macos && cmake --build --preset macos
-- Running vcpkg install
The following packages will be built and installed:
    fmt[core]:arm64-osx -> 9.1.0#1 -- /Users/justinpratt/Downloads/vcpkg/buildtrees/versioning_/versions/fmt/3f452404270b508daf355b72031ad3ee7d0d5751
...
-- The CXX compiler identification is AppleClang 14.0.3.14030022
...
[100%] Linking CXX executable main
[100%] Built target main

$ ./build/macos/main
Hello, world!

Beyond the Triplet #

This worked quite well, so I was curious to see how it handles cross-compiling and other real-world scenarios requiring more customization. That’s where I hit the first hurdle: vcpkg documentation.

It wasn’t obvious how to specify a compiler when using vcpkg via CMake. Typically, you can do this in one of a number of ways, but vcpkg appropriates that. I finally stumble across the VCPKG_CHAINLOAD_TOOLCHAIN_FILE option, but it’s still not totally clear what I need. Web searches are coming up surprisingly sparse. Is no one doing this? Weird.

Anyway, let’s try GCC. I first update main.cpp such that I know it won’t compile with Clang, to prove to myself that I’m actually getting the compiler I want:

main.cpp 
#include <fmt/format.h>

template <auto T>
struct Dummy {};

int main() {
  Dummy<42.0> dummy{};
  fmt::print("Hello, world!\n");
  return 0;
}

$ cmake --preset macos && cmake --build --preset macos
example/main.cpp:7:9: error: sorry, non-type template argument of type 'double' is not yet supported
  Dummy<42.0> dummy{};
        ^
1 error generated.

As expected. I’m amused at how polite clang is about this. I’m not sure a compiler has ever apologized to me.

After hunting through vcpkg docs almost randomly, I finally run across the documentation for adding your own triplets. So, I can define a custom triplet, point vcpkg at it, and then set VCPKG_CHAINLOAD_TOOLCHAIN_FILE inside that file. Finally, in my chain-loaded toolchain file, I set the compiler.

Whew. It took a while to come to that conclusion, and honestly I’m still not sure if that’s what the vcpkg maintainers had in mind. I get that sense that you’re off the beaten path if you want this level of customization.

I add the triplet and toolchain files:

triplets/arm64-apple-macos11-gcc.cmake 
set(VCPKG_CHAINLOAD_TOOLCHAIN_FILE "${CMAKE_CURRENT_LIST_DIR}/arm64-apple-macos11-gcc.toolchain")
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)
triplets/arm64-apple-macos11-gcc.toolchain 
set(CMAKE_CXX_COMPILER "g++-13")

And update the presets with macos-gcc:

CMakePresets.json 
{
  "version": 5,
  "configurePresets": [
    {
      "name": "vcpkg",
      "hidden": true,
      "cacheVariables": {
        "CMAKE_TOOLCHAIN_FILE": "$env{VCPKG_ROOT}/scripts/buildsystems/vcpkg.cmake"
      }
    },
    {
      "name": "linux",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-D_FORTIFY_SOURCE=3 -fstack-protector-strong -fstack-clash-protection -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast",
        "CMAKE_EXE_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now",
        "CMAKE_SHARED_LINKER_FLAGS": "-Wl,--allow-shlib-undefined,--as-needed,-z,noexecstack,-z,relro,-z,now"
      }
    },
    {
      "name": "macos",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "-fstack-protector-strong -Wall -Wextra -Wpedantic -Wconversion -Wsign-conversion -Wcast-qual -Wformat=2 -Wundef -Werror=float-equal -Wshadow -Wcast-align -Wunused -Wnull-dereference -Wdouble-promotion -Wimplicit-fallthrough -Wextra-semi -Woverloaded-virtual -Wnon-virtual-dtor -Wold-style-cast"
      }
    },
    {
      "name": "macos-gcc",
      "inherits": ["macos"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_COMPILER": "g++-13",
        "VCPKG_OVERLAY_TRIPLETS": "${sourceDir}/triplets",
        "VCPKG_TARGET_TRIPLET": "arm64-apple-macos11-gcc"
      }
    },
    {
      "name": "windows",
      "inherits": ["vcpkg"],
      "binaryDir": "${sourceDir}/build/${presetName}",
      "cacheVariables": {
        "CMAKE_CXX_FLAGS": "/sdl /analyze /analyze:external- /guard:cf /utf-8 /diagnostics:caret /w14165 /w44242 /w44254 /w44263 /w34265 /w34287 /w44296 /w44365 /w44388 /w44464 /w14545 /w14546 /w14547 /w14549 /w14555 /w34619 /w34640 /w24826 /w14905 /w14906 /w14928 /w45038 /W4 /permissive- /volatile:iso /Zc:inline /Zc:preprocessor /Zc:lambda /Zc:__cplusplus /Zc:externConstexpr /Zc:throwingNew /EHsc",
        "CMAKE_EXE_LINKER_FLAGS": "/machine:x64 /guard:cf"
      }
    }
  ],
  "buildPresets": [
    {
      "name": "linux",
      "configurePreset": "linux"
    },
    {
      "name": "macos",
      "configurePreset": "macos"
    },
    {
      "name": "macos-gcc",
      "configurePreset": "macos-gcc"
    },
    {
      "name": "windows",
      "configurePreset": "windows"
    }
  ]
}

Note the need for VCPKG_OVERLAY_TRIPLETS and VCPKG_TARGET_TRIPLET to select the custom triplet file.

Finally, it works:

$ cmake --preset macos-gcc && cmake --build --preset macos-gcc
...
  CMAKE_CXX_COMPILER="g++-13"
  VCPKG_TARGET_TRIPLET="arm64-apple-macos11-gcc"
...
[100%] Built target main
$ ./build/macos-gcc/main
Hello, world!

Closing Thoughts #

I love the M2. vcpkg isn’t quite as flexible as I hoped, but overall I’m happy with this setup and will probably continue to use it. The custom triplet solution meets my needs for cross-compiling or other customization, although I’d be curious if anyone knows of a better fix.

↑