Handler proc macro

[timClicks]

I'm concerned that developers that might be put off by the Rust runtime because it makes a poor first impression.

Let's consider a minimal example of a Lambda handler in TypeScript:

import { Handler } from 'aws-lambda';

export const handler: Handler = async (event, context) => {
    return {};
};

And a minimal example in Python:

def handler(event, context):
    return {}

And now Rust:

use lambda_runtime::{service_fn, LambdaEvent, Error};
use serde_json::{json, Value};

#[tokio::main]
async fn main() -> Result<(), Error> {
    let func = service_fn(handler);
    lambda_runtime::run(func).await?;
    Ok(())
}

async fn handler(event: LambdaEvent<Value>) -> Result<Value, Error> {
    Ok(json!({ }))
}

With some thought, we could do much better than that. Here's a sketch:

use lambda_runtime::{Event /* No need for Lambda prefix */, Error, handler};
use serde_json::{json, Value};

#[handler]
async fn handler(event: Event<Value>) -> Result<Value, Error> {
    Ok(json!({ }))
}

Previous versions of the runtime actually included such a proc macro. It was removed in #282 for a few reasons.

• Compiler error reporting is not handled correctly using this macro, so the spans where an error occurs are undefined and point to incorrect locations.
• Using this macro conflates warm and cold starts for Lambda functions, as this removes the ability to initialize global resources that might panic if initialized twice, such as a global logger.

Given that the alternative to using this macro is a single line addition to a main function, there is no real need to continue supporting this macro.

I want to address the last point first: I do think that a helper macro does remove enough heavy lifting to justify its inclusion. As mentioned earlier, we can make the developer experience much better. Hopefully we can make it so good that using the Rust runtime feels like using a scripting language.

I'll need to dig into the concern about error reporting. This seems like a very important thing to get right. I wouldn't want to proceed with my idea if we end up making things harder for developers rather than easier.

There's the question about how to do any sort of global initialization/setup. This is a valid concern. Perhaps we could add an argument to the handler attribute?

use lambda_runtime::{Event /* No need for Lambda prefix */, Error, handler};
use serde_json::{json, Value};

fn init() {
      tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .init();
}

#[handler(init = init)]
async fn handler(event: Event<Value>) -> Result<Value, Error> {
    Ok(json!({ }))
}

There's also a concerned about added confusion. Suddenly there is now an additional way to the runtime.

The other negative of having 2 pathways is the initial confusion. It takes some time to understand the difference. Should I use main or macro? What are the pro's and con's? It would be less code to maintain too if we got rid of it.

-- from #259 (comment)

I would argue that #[tokio::main] also provides a second pathway to tokio. I would like the new #[handler] attribute to be more of a short cut than a new route. It would be easy to peel back the layers.

It may seem trivial to focus on how handlers are initialized, but there will be many people looking at Rust who have come from Python/TypeScript and who want more performance. But they're nervous. They have heard that Rust is difficult to learn and get right. They have less experience with Rust and less ability to correct themselves if they get into errors.

[calavera]

I would like to have real data from people that have tried this runtime and struggled to understand its semantics.

I have several real cases where the semantics of those other runtimes lead to many performance issues during cold start because people put shared initialization inside the handler functions. I do not want to have an interface that promotes bad practices.

There's the question about how to do any sort of global initialization/setup. This is a valid concern. Perhaps we could add an argument to the handler attribute?
use lambda_runtime::{Event /* No need for Lambda prefix */, Error, handler};
use serde_json::{json, Value};

fn init() {
tracing_subscriber::fmt()
.with_max_level(tracing::Level::INFO)
.init();
}

#[handler(init = init)]
async fn handler(event: Event) -> Result<Value, Error> {
Ok(json!({ }))
}

This example is reimplementing our current interface, with worse semantics, and an extra layer of indirection. How are you passing shared resources here? With the current semantics it's much more explicit. By trying to hide a few basic Rust concepts, this is raising more questions than it really answers.

I would argue that #[tokio::main] also provides a second pathway to tokio. I would like the new #[handler] attribute to be more of a short cut than a new route. It would be easy to peel back the layers.

Can you expand on this? I don't understand what it means, which second pathway? why do we need more layers?

[greenwoodcm]

i think the handler trait would enforce the right behavior - make it easier (rather than arguably harder with the macro) to do shared things on init and unique things on invoke.

[ymwjbxxq]

Hello guys,

If I can add my 2 cents

The current implementation of Rust is ok when you get used to the verbosity of the language, but as @timClicks, maybe it can be improved to make it cleaner.
What Tim suggested would be very similar to the Golang implementation.

var authorizer *auth.Authorizer

func init() {
  authorizer = .....
}

func main() {
	lambda.Start(handleRequest)
}

func handleRequest(ctx context.Context, event events.APIGatewayCustomAuthorizerRequestTypeRequest) (events.APIGatewayCustomAuthorizerResponse, error) {
    authorizer.doSomething()
}

But I do not like the var outside of the init.

Tim syntax is nice:

use lambda_runtime::{Event /* No need for Lambda prefix */, Error, handler};
use serde_json::{json, Value};

fn init() {
      tracing_subscriber::fmt()
        .with_max_level(tracing::Level::INFO)
        .init();
        
    let config = aws_config::load_from_env().await;
    let dynamodb_client = aws_sdk_dynamodb::Client::new(&config);
}

#[handler(init = init)]
async fn handler(event: Event<Value>) -> Result<Value, Error> {
    Ok(json!({ }))
}

Now the question would be how you pass all your initialisation, like dynamodb_client, to the handler.

[yerke]

@timClicks I would argue that the current approach with #[tokio::main] async fn main() {} is more familiar to the existing Rust developers. The comparison with Python and TS/JS is not completely fair. Python example you showed is sync. In JS there is only 1 async runtime which is part of the language itself. Furthermore, for JS lambda you need to specify the corresponding runtime (nodejs18.x vs nodejs16.x, etc.) when creating a lambda explicitly, so AWS runtime would know that it should expect to find handler function in the JS exports. We don't do the same for Rust lambdas (and I personally don't think we should).

We don't have the true and only async runtime in Rust on the language level, and people who are familiar with async Rust will be more comfortable with the current approach, which is more explicit. I personally would have more questions from seeing #[handler] async fn handler(event: Event<Value>) -> Result<Value, Error> { todo!(); }. For example: Which executor is this using? How do I initialize the resources? It's a bit too vague/magical for me.

People learning async Rust are more likely see #[tokio::main] async fn main() {} in learning materials as well.

So I don't think deleting those 5 extra lines is worth it from the perspective of explicitness and clarity.

P.S. As other people pointed out you also have main function in case of Golang lambdas. Again, since Golang has async runtime in the language itself, that main function gets to be very small.

[BMorinDrifter]

With Python, Typescript, etc. I can upload a small source file as my lambda function and have it running with almost the absolute minimal amount of ceremony possible. There would need to be a considerable amount of integration work to close the gap for learning projects and small scripts, probably including AWS console work. For example, if I just need a script to tune GameLift fleet settings every few minutes, Rust isn't a great tool for the job because of the added process to make a quick adjustment. let func = service_fn(handler); is a bit of an eyesore, but cutting a half dozen or so lines of boilerplate doesn't address the biggest gaps if we did want Rust to better service learning projects and small jobs/scripts.

Where Rust really shines is when you grow a thing and start running into issues where Python and likely Typescript become a worse fit for the Lambda platform. I'm going to speak to Python because I have more experience and currently use a split of Rust and Python for serverless functions.

• As your function grows cold starts can become a challenge, particularly as you pull in dependencies due to the lack of a link step to throw out all the stuff your function isn't using.
• pip and lambda layers aren't as good for dependency management as cargo. Many Python packages are really written in C/C++, so pulling in libraries adds operational overhead that didn't exist when your function was a small number of source files. More and more, programming is becoming about combining cogs other people built to solve a specific problem vs building things with mostly the language and foundational libraries.
• Runtime binding and tossing around schema-free json documents leads to runtime errors, particularly in infrequent code paths. Building more tests for better code coverage costs much of the time saved from going with a scripting language in the first place.
• Async becomes a powerful tool when your function needs to make multiple RPCs

The gap I'd like to see closed is around some of the functionality Lambda Power Tools provides, because that helps build the things we choose to build in Rust instead of Python, etc. due to fitness of the core technology.