Introducing Scale Functions
Today, we're introducing Scale, a WebAssembly-powered function runtime that allows for performant, composable, and language-agnostic software development.
With Scale Functions you can write code in any language, then use it from any other language, environment, or runtime. State-of-the-art sandboxing qualities, startup times, and overall performance come baked in.
This initial release includes client support for Go and Rust, with runtimes for Go and TypeScript.
Why
Application developers are constantly integrating third-party services into their core business logic — most of which exist as middleware, and need to be incorporated directly into the application layer.
As the number of integrations grows, third-party services often struggle to provide support in multiple languages or runtimes, leading to compatibility and versioning issues if the library is neglected.
For Developers, this can make even a simple integration complicated.
Scale Functions open a path forward to using libraries from any language in any language, and completely wipe out this class of problems:
Does your Rust-focused team need to support an SDK written in Go? No problem.
Wanna use a TypeScript audio filter in your Java application? Scale's got you covered.
How about using a Python middleware in a Next.js application? Well... not yet, but it won't be long. 😉
Imagine building your product's SDK or middleware in whatever language is most comfortable for your team, while still supporting nearly every language under the sun. With Scale, that fantasy becomes a reality. Instead of building language-specific tools, both developers and 3rd party vendors can focus on their core business logic.
This approach also allows for a more modular approach to software development, where teams can focus on building small, composable pieces of functionality, and then stitch them together into a larger whole without worrying about compatibility issues.
Thanks to WebAssembly, for the first time developers can build software in a way that's truly language-agnostic.
“WebAssembly as an Implementation Detail[1]”
When developers think about WebAssembly, they often think about running non-JavaScript code in the browser. However, WebAssembly is a general-purpose virtual machine, and can be used for a wide variety of use-cases.
But cross-language interoperability isn't simple, and it's easy to get lost in the weeds just trying to get a simple string from one language to another. This is where Scale comes in.
Just like most users of container runtimes today don't need to have an intimate of knowledge of linux cgroups to deploy most projects, we strongly believe that WebAssembly should just be an implementation detail – not the focus – of your infrastructure.
Low-level knowledge of the technology should be abstracted away and replaced with a focus on developer experience, and with Scale Functions, we've worked very hard to make this a reality.
Functions and Middleware
Scale supports the idea of pure functions without the need for a host applications. This is similar to the concept of serverless functions, except that a serverless runtime typically requires developers to write code that specifically targets it.
With Scale Functions, user-defined business logic is executed within the Wasm sandbox, which can be embedded into any runtime or host application. WebAssembly is a perfect fit for this type of execution model, and we're excited to help further evolve the ecosystem.
We're opening up a whole new world of possibilities for developers, where the same core functions can be used inside existing applications, or as standalone serverless functions - without any changes to the code. No vendor lock-in to specific cloud providers, and no need to worry about compatibility issues.
Furthermore, Scale Functions are designed with an idiomatic .Next() method available in all guest implementations, which allows developers to easily chain together functions to create complex middleware pipelines that are completely language agnostic.
As an example pipeline, you could have a rust function to validate an incoming HTTP request, a golang function to process the request body, and a third typescript function to generate the response! You can read more about chaining functions together in our docs.
Writing a Scale Function
Writing, building, and running a Scale Function only requires 3 commands with the Scale CLI:
scale new hello
Successfully created new go scale function hello
scale build
Building scale function hello:latest ...
Successfully built scale function hello:latest
scale run hello:latest
Scale Function hello:latest listening at http://localhost:8080package scale
import (
signature "github.com/loopholelabs/scale-signature-http"
)
func Scale(ctx *signature.Context) (*signature.Context, error) {
ctx.Response().SetBody("Hello, World!")
return ctx.Next()
}Not bad right? No need to worry about the underlying runtime, or how to package your code for deployment. Just write your business logic and let Scale take care of the rest.
But how does it actually work?
How it works
Scale Functions make use of the Scale Runtime, which currently comes in two flavors: Go and TypeScript.
When a Scale Function is built, the necessary bindings for cross-language interoperability are automatically generated, and the resulting Wasm module is packaged into a format that can be executed directly by any Scale Runtime.
For Golang, the Wasm code is executed using the zero-dependency Wazero WebAssembly Runtime, from the good folks over at Tetrate Labs.
For TypeScript, the Wasm code is executed using the native WebAssembly support in Node.js and browser environments.
But how does the host runtime actually communicate with the guest language? This is where Scale Signatures come in.
Crossing the Wasm Boundary with Signatures
If you've ever worked with WebAssembly, you will have noticed that moving data between the host and the WebAssembly module is not always straightforward. For this reason, we built Polyglot, a serialization framework that's extremely efficient and facilitates sending structured data between host and guest languages.
Today, we're releasing the HTTP Signature for Scale Functions, which is meant to be used for HTTP handlers and middleware.
In the very near future we'll be giving developers the ability to write their own Signatures for Scale Functions, allowing for even more flexibility and a whole new set of use-cases.
The main benefit of the Signature model is that the data passed between the host and the guest is both typed, and type-checked, at compile time.
And as usual, at Loophole Labs, performance is the default:
An End-to-End Workflow
After you write a scale function like the above, and build it, you can also push it to the Scale Registry, which is a Docker-like repository for storing and sharing Scale Functions:
scale push hello:latest
Pushing scale function hello:latest to registry ...
Successfully pushed scale function hello:latest to registryOnce a Scale Function is in the Scale Repository, it can be shared across applications and organizations, allowing for maximum code reuse.
The Scale Registry also makes it possible to update your application's business logic at runtime, without having to rebuild or redeploy the entire application.
We are very excited for the Scale Registry to become a distributed tool for the easy sharing of application logic among teams and projects, acting as a useful, complementary layer to the interoperability and composability already provided by the Scale Runtime.
A full guide to getting started with Scale Functions can be found in our docs.
What's Next
This is just the beginning for Scale Functions; with further support for Signatures coming in the near future, and support for more Host runtimes and Guest modules already on the way, true composability and runtime agnostic software integration and development are no longer just a promise, but an imminent reality.
We'd love to hear what you think about Scale Functions, and how you think it can be improved! Please join the #scale Channel in our Discord to join the discussion, and you can also follow us on Twitter to stay up to date on all things Loophole!
[1]: Phrase from Brian Sletten's talk, “WebAssembly: The Whole Enchilada”. Brian is the Author WebAssembly: The Definitive Guide (O'Reilly, 2021)