What I Learned from Serverless Workflow

Background

I suddenly learned about a project called ServerlessWorkflow.

https://github.com/serverlessworkflow

The specific utility of this project is not important; what matters is that it provides insight into the current state of cloud-native middleware development and the knowledge that needs to be learned.

Since this project is a CNCF project, the specifications, ideas, and protocols used within it can well reflect the current trends of CNCF.

ServerlessWorkflow is a workflow protocol initiated by CNCF, and it is currently implemented using C#.

https://github.com/serverlessworkflow/synapse

After downloading the source code for synapse, I was shocked by many aspects. It wasn't because the code was exceptionally impressive; rather, it was due to the protocols, components, and ideas used within.

The structure of the synapse source code is as follows:

The working principle of synapse is roughly like this: a worker is an independent process. Whenever a workflow is launched, a worker process is started using Docker or Kubernetes, and the worker parses the workflow protocol to execute the workflow steps.

This project communicates with Docker and Kubernetes, which is one point to note but let's set it aside for now.

There are many NuGet libraries used by this worker:

Let's take a look at the ServerlessWorkflow.SDK part.

The NuGet libraries used by ServerlessWorkflow.SDK are as follows:

From this, we can understand all the NuGet packages involved.

CloudNative.CloudEvents

CloudEvents: a specification for describing event data in a standard format.

This library implements a CNCF cloud-native event bus protocol, with its official website at: https://cloudevents.io/
C# SDK usage documentation: https://github.com/cloudevents/sdk-csharp/blob/main/docs/guide.md

The usage is roughly as follows:

CloudEvent cloudEvent = new CloudEvent
{
    Id = "event-id",
    Type = "event-type",
    Source = new Uri("https://cloudevents.io/"),
    Time = DateTimeOffset.UtcNow,
    DataContentType = "text/plain",
    Data = "This is CloudEvent data"
};

CloudEventFormatter formatter = new JsonEventFormatter();
HttpRequestMessage request = new HttpRequestMessage
{
    Method = HttpMethod.Post,
    Content = cloudEvent.ToHttpContent(ContentMode.Structured, formatter)
};

Essentially, this involves using the HTTP protocol to carry certain information in the header, transforming the HTTP request into an event. The Pub/Sub framework of Dapr, as well as the CAP framework, operate similarly, with tracing frameworks also injecting SpanContext information into the header.

As we can see, a CloudEvent is created first, and then this CloudEvent is attached to the HTTP request details.

Protocol Communication

In ServerlessWorkflow, the functions (action operations) within the flow can make remote requests to external services, passing parameters from the workflow, then obtaining response results, and processing the response results by utilizing the data within the flow.

ServerlessWorkflow supports the following invocation methods:

Using Functions for RESTful Service Invocations
Using Functions for Async API Service Invocations
Using Functions for RPC Service Invocations
Using Functions for GraphQL Service Invocations
Invoking a GraphQL Query
Invoking a GraphQL Mutation
Using Functions for OData Service Invocations
Creating an OData Function Definition
Invoking an OData Function Definition

HTTP is commonly used, and gRPC is also beneficial; however, GraphQL, OData, and CloudEvents are unfamiliar to me, and I know nothing about them.

Components within Neuroglia.Serialization.* are also quite interesting, as they can simplify the protocol and the serialization/deserialization process.
protobuf-net.Grpc allows for the implementation of gRPC without writing proto.

Another interesting point is that, although they all utilize HTTP requests, the term Async API is mentioned as well.

Neuroglia.Data.Expressions.JQ is used for parsing expressions within the flow.