Google Summer of Code Project Report

Name: Matthew Nappo

Organization: The Scalable Parallel Computing Laboratory at ETH Zurich

Project: rFaaS: an RDMA-accelerated function-as-a-service platform

Project link: view here

Project size: Large (350 hours)

Dates: May 22, 2023 - September 25, 2023

Email: mnappo@u.rochester.edu

GitHub Profile: @mattnappo

What is rFaaS?

rFaaS is a function-as-a-service platform for high-performance computing. Function-as-a-service (FaaS) is a paradigm in serverless computing which allows users to easily define a set of functions, and then use those functions on-demand. Instead of deploying a dedicated server to handle requests and execute functions, FaaS providers will spin up a temporary worker to execute requested functions. This means that users are only billed for the server time actually used to execute functions. Users do not pay for idling CPU time, do not need to handle the complexities involved with deploying their own server, and get horizontal scaling out of the box. A popular FaaS platform is AWS Lambda.

One problem with FaaS platforms is the high latency of invocation requests. rFaaS addresses this issue by using Remote Direct Memory Access (RDMA) to communicate with workers. Function invocation requests, input data, and outputs are all written over an RDMA connection. Since RDMA is extremely fast, the function invocation overhead of rFaaS is significantly lower than competitors.

For more details about rFaaS, read the paper here.

Project Overview

Introduction

The rFaaS developers created rdmalib to simplify the complexity of writing RDMA networking code. This library abstracts common operations such as creating RDMA connections and reading/writing from buffers. rdmalib uses libibverbs and librdmacm for the low-level verbs programming. The rFaaS server code uses rdmalib for its RDMA networking.

While RDMA can be achieved through various communication methods, including InfiniBand, RoCE and Soft-RoCE, TCP/IP and UDP via iWarp, Cray GNI, and Amazon EFA, rFaaS presently only supports a verbs backend, such as RoCE or InfiniBand. The goal of the project is to expand this support to include a more extensive range of networking backends, as listed above.

To achieve this, the solution is to implement rFaaS using libfabric, a higher-level networking library that provides a uniform API for network operations across many RDMA protocols such as those listed above. This approach will enable rFaaS to work over all of the RDMA protocols that libfabric supports. By configuring the implementation correctly, users of rFaaS will be able to easily choose the protocol they wish to use on their cluster.

Motivation

Expanding the range of supported networking backends will greatly enhance the usability of rFaaS. Currently, the platform is limited to high-performance clusters equipped with the required verbs hardware, making it inaccessible to those without such resources. However, by adding support for TCP/IP and other protocols, such as AWS EFA, rFaaS will be more accessible and widely deployable. With TCP/IP support, rFaaS can operate reliably over standard networks, opening up new use cases for the platform. Additionally, support for AWS EFA will enable rFaaS to leverage the high-speed networking available in AWS data centers. This project has the potential to make a significant impact on the usability and accessibility of rFaaS. It will allow the platform to be used in a variety of scenarios and for a more extensive range of applications, making it more versatile and compelling to developers.

Solution

My work builds upon the partial libfabric implementation started in 2022. This base implementation was tested on a Cray machine using the GNI/verbs provider in libfabric.

My Tasks

Setup Development Environment

Before working on the project, I spent much time setting up my development environment. I did have access to RDMA hardware through the Swiss National Supercomputing Centre, but this environment was better suited for production and low-risk testing–not a development environment. Thus, I opted to setup Soft-RoCE, a software implementation of RDMA over Ethernet, since no special networking hardware is required. Much of the documentation for this process was outdated and scattered around the internet on various forums. Setting up Soft-RoCE also involved building the Linux kernel with certain kernel modules enabled. After this setup process, we were pleased to find out that rFaaS runs on Soft-RoCE out of the box.

Better Container Support

rFaaS has the option to run the function executor in a containerized environment. The current supported container systems are Docker and Sarus. However, the current implementation has a lot of temporary and hard-coded values, and no way to configure containerization without modifying and rebuilding the source code.

Solution

I solved this by adding a JSON-based configuration system for Singularity, Sarus, and Docker all in one file. This involved writing a new JSON parser for the executor manager configuration file. Once the configuration system was in place, I wrote some code to switch between launching Docker, Singularity, and Sarus containers. This takes place in the executor itself.

See the PR here for more details.

Refactoring the Initial Implementation

The existing libfabric implementation works by introducing compile-time flags to conditionally compile the libfabric or verbs code. At each point in the codebase involving RDMA networking, an #ifdef USE_LIBFABRIC is used to switch between libfabric code and ibverbs code. While this does split the codebase into two implementations, it is not a very elegant or sustainable solution.

Thus, my first task was to refactor the existing libfabric implementation to be cleaner, more readable, stable, and better for longevity. With a proper refactor, future work on the project will be much smoother, as there will be a more solid base to build off of.

Solution

The solution was to use C++ templates. Specifically, after some discussion, we decided to use the Curiously Recurring Template Pattern (CRTP). In its most basic form, CRTP involves a base class, and one or more derived classes that pass themself as a template parameter. Here is a small example:

template <typename Derived>
class Base {
    int foo() {
        return static_cast<Derived*>(this)->foo_impl();
    }
};

class DerivedA : Base<DerivedA> {
    int foo_impl() {
        ...
    }
};

class DerivedB : Base<DerivedB> {
    int foo_impl() {
        ...
    }
};

See the above link or here for more details on CRTP.

We used this pattern to separate the libfabric and ibverbs implementations. Since templates are instantiated at compile time, only the relevant code will be compiled. For example, the implementation of a libfabric connection is different from an ibverbs connection, but they share the same/similar interface. We only want to compile the libfabric code when the user wants to use libfabric, and the same for verbs. Here is a minimal example of how this works in practice:

template <typename Derived, typename T>
struct Connection {
    int connect(char* addr) {
        return static_cast<Derived*>(this)->connect(addr);
    }

    int write(T data) {
        return static_cast<Derived*>(this)->write(data);
    }
};

template <typename T>
struct LibfabricConnection : Connection<LibfabricConnection<T>, T> {
    int connect(char* addr) {
        ...
    }

    int write(T data) {
        ...
    }
};

template <typename T>
struct VerbsConnection : Connection<VerbsConnection<T>, T> {
    int connect(char* addr) {
        ...
    }

    int write(T data) {
        ...
    }
};

int main() {
    LibfabricConnection<int> conn;
    conn.write(123);
}

Note: This still needs to be tested, as there are some linker errors preventing compilation.

See the PR below.

Enabling TCP and AWS EFA Support

Much of the time spent on this project was to get additional libfabric providers to work with rFaaS. The goal was to get TCP/Sockets, and/or AWS EFA to work. This, unfortunately, did not happen. This tasks turned out to be much larger than we originally anticipated. However, it was not a complete failure, as we learned a lot about the problem, and began investigating potential solutions. You can see the research notes in the PR here.

Pull Requests

Throughout the project, I opened many pull requests:

Add better container support

Opened Apr 4, 2023

View here

This PR enhances rFaaS’s Docker container support by:

  • Adding scripts to launch a private Docker registry with authentication (credentials)
  • A rFaaS base image Dockerfile (and scripts) to enrich a Docker container with the rFaaS executor binary
  • Automatic image polling from a private Docker registry instead of DockerHub
  • Removed hard-coding of container configuration
  • Wrote a new config/executor_manager.json configuration file and parser for configuring the Docker registry, container selection, and other container configuration details

The PR also makes configuring Sarus containers easier, and laid the foundation of Singularity container configuration.

Add container configuration system from libfabric-sarus branch into master

Opened Jun 3, 2023

View here

This PR takes all the enhancements of the above PR, and adapts them to work with the master branch. The libfabric-sarus branch can no longer be merged into master as it has undergone significant changes. So, this PR allows the above container enhancements to work with master.

Fix compiler errors in the libfabric-sarus branch

Opened Aug 31, 2023

View here

This PR fixes many compiler errors and other bugs currently present in the libfabric-sarus branch. The reason for this PR is so that any future development off this branch will have a nice working base.

Libfabric alternative provider research

Opened September 24, 2023

View here

This PR documents the details of how we went about trying to support the TCP and EFA providers. This turned out to be a much larger issue than we originally expected. Even when GSoC ends, I would like to continue working on this issue.

Merge libfabric-refactor-traits into libfabric-refactor

Opened September 24, 2023

View here

All of the recent development of the refactor this summer was done on libfabric-refactor-traits. libfabric-refactor was the original branch, off of which libfabric-refactor-traits was created. This is more of an auxiliary PR to keep the repository clean.

The libfabric refactor

Opened September 25, 2023

View here

This is the main PR that introduces the template-based libfabric refactor discussed at length earlier.

Future Work

Since more work is necessary to get TCP and EFA working, I would like to continue working on this issue in the future. I am now quite comfortable with the entirety of rFaaS, having read most, if not all, of the ~10,000 lines of code. I feel connected to the project and motivated to continue working on it.

The next step is likely to contact the libfabric maintainers about how we could best use libfabric to move forward with our project.

Conclusion

Overall, working on rFaaS was a challenging but rewarding experience. I learned a lot about RDMA networking and C++ template patterns, and learned to persevere through long debugging processes.