envoy-cmake: Adapting YAStack build system for regular envoy

Building vanilla envoy using cmake

EnvoyCon and KubeCon in Seattle 2018

We used a Lyft to get to KubeCon/EnvoyCon from the airport in Seattle. Being closely associated with Envoy made us think of Lyft over other services. However every time we take a Lyft, we ask the driver why they choose to drive for Lyft. And every time, there is an interesting story/perspective behind it.

Envoy’s popularity and roadmap

Even more exciting was how widespread acceptance Envoy has gained. What stood out is, that it just took two years for Envoy to get to this level of popularity. This is quiet impressive compared to other open source proxies.

Using CMake for Envoy builds

While we did not present YAStack at EnvoyCon, it did come up in conversations. What also came up is the wish to use cmake to build envoy. There are people incorporating envoy in the projects they undertake. They wanted to use cmake with envoy for several reasons:

  • A library they were trying to integrate with used CMake
  • They were more familiar with CMake and less familiar with bazel.

Adapting YAStack build system

YAStack uses CMake to build the project. From a code perspective, YAStack first compiles dpdk and f-stack. It then compiles envoy and links it with dpdk and f-stack libraries.

CMake build hierarchy

The top level CMakeLists.txt is located in ‘envoy’ directory. It includes all the sub-directories that also have a CMakeLists.txt file.

cmake build tree

Enhancements

We currently have the following enhancements in mind (in no particular order).

  • Improve dependency tracking across projects
  • Create static binary for envoy-source-exe
  • Build container images
  • Using repositories.bzl file for hint, download specific version of pre-created tar/archive files for dependent library (under envoy/external). Right now they are all being compiled.
  • Start using pre-compiled libraries to improve build time.
  • Use git sub-modules to fetch and update dependencies

Troubleshooting

  • The build process assumes that protoc compiler is available. To use a compatible version, build/install the one under envoy/external/protobuf-3.5.0
  • The build process also assumes that the protoc-gen-validate binary is installed. We have verified the build with protoc-gen-validate binary built from source tagged at tag v0.0.6
  • If the build is stuck while compiling external/nghttp2–1.30.0, use ./configure — enable-lib-only to build only libraries for nghttp2

Conclusion

We have come to use cmake extensively for all our projects and find it as a very versatile tool for our build system. We hope this project will help others in the community too.

EnRoute One-Step Ingress / YAStack: Envoy on FreeBSD TCP/IP on DPDK