AWS Go lambda recevie raw http request bytes without any base64 encoding through API gateway

Question

I am trying to build a rest api with Go using lambda and api gateway. In the api, I want to process the raw request body bytes without any base64 conversion in middle.

I am able to receive base64 encoded request body from api gateway, but I am looking for a way so that I can directly process the raw body bytes without any conversion in middle.

I have as far as below:

API gateway setup: Added application/octet-stream in Binary Media Types. Lambda is integrated through AWSIntegration and direct pass through. Here is the relevant cdk code:

        const testPath = api.root.addResource('test');
        const testIntegration = new LambdaIntegration(props.testFunc, {
            proxy: false,
            contentHandling: ContentHandling.CONVERT_TO_BINARY,
            passthroughBehavior: PassthroughBehavior.WHEN_NO_MATCH,
        });
        testPath.addMethod('POST', testIntegration);

And my Go handler function looks like below:

func Handler(ctx context.Context, payload []byte) ([]byte, error) {
    fmt.Println("received...")
    fmt.Printf("received request of length: %v", len(payload))
    return nil, nil
}

func main() {
    lambda.Start(Handler)
}

But I always get a response like:

{"message": "Could not parse request body into json: Could not parse payload into json: Invalid UTF-8 start byte 0xa3\n at [Source: (byte[])\"\uFFFDWQo\uFFFD8\u0010.......\uFFFD\"[truncated 360 bytes]; line: 1, column: 6]"}%

Here is relevant logs from API Gateway

Starting execution for request: xxx
HTTP Method: POST, Resource Path: /test
(xxx) Method request body before transformations: [Binary Data]
(xxx) Endpoint request URI: https://lambda.xxxxx.amazonaws.com/2015-03-31/functions/arn:aws:lambda:xxxxx:xxxxxxx:function:test/invocatioins
(xxx) Endpoint request body after transformations: [Binary Data]
(xxx) Sending request to https://lambda.xxxxx.amazonaws.com/2015-03-31/functions/arn:aws:lambda:xxxxx:xxxxxxx:function:test/invocatioins
(xxx) Received response. Status: 400, Integration latency: 8 ms
(xxx) Endpoint response headers: {Date=xxxx GMT, Content-Type=application/json, Content-Length=1375, 
(xxx) Execution failed: Could not parse request body into json: Could not parse payload into json: Invalid UTF-8 start byte 0xa3
 at [Source: (byte[])"�WQo�8~
 (xxx) Endpoint response body before transformations: {"Type":"User","message":"Could not parse request body into json: Could not parse payload into json: Invalid UTF-8 start byte 0xa3\n at [Source: (byte[])\"�WQo�8\u0010~��\u0015\u0011�MҪ,rX����h�\u000B�R��\u0
 (xxx) Lambda invocation failed with status: 400.

I feel that the handler function definition is not correct. And somehow, in lambda it first tries to parse the request as json.

Any idea on how can I receive the raw http request body bytes in lambda?

n.b. below is the working code with base64

CDK code

        const testPath = api.root.addResource('test');
        testPath.addMethod('POST', new LambdaIntegration(props.testFunc));

Go code

func Handler(ctx context.Context, request *events.APIGatewayProxyRequest) (*events.APIGatewayProxyResponse, error) {
    if !request.IsBase64Encoded {
        return &events.APIGatewayProxyResponse{
            StatusCode: http.StatusBadRequest,
            Body: "Only raw bytes supported",
        }, nil
    }
    requestBytes, _ := base64.StdEncoding.DecodeString(request.Body)

    return &events.APIGatewayProxyResponse{
        StatusCode: http.StatusOK,
        Body: fmt.Sprintf("received request of length: %v", len(requestBytes)),
    }, nil
}

func main() {
    lambda.Start(Handler)
}