Configure grafana cloudwatch datasource with STS Temporary credentials

Question

Is there a way to configure Grafana Cloudwatch datasource using AWS STS Temporary credentials?

I am trying to configure a cloudwatch datasource in grafana by assuming a role. I have created a role CLoudwatchGrafanaRole which has policies defined to read metrics from Cloudwatch.

Next I created a IAM user grafana and defined the following policy for the user

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sts:AssumeRole"
            ],
            "Resource": "arn:aws:I am::<>:role/CloudwatchGrafanaRole"
        }
    ]
}

Also modified the trust policy of CloudwatchGrafanaRole

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:Iam::<>:user/grafana"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Next, using AWS CLI fetch the access id, key and session token

aws configure //login with grafana user 

aws sts assume-role --role-arn arn:aws:iam::<>:role/CloudwatchGrafanaRole --role-session-name "RoleSession1"

From the above command we get AWS Access ID, key and session token.

How do you use them to configure datasource in grafana?

Trying to use the Temporary Access ID and secret key directly returns the error - 1. CloudWatch metrics query failed: InvalidClientTokenId: The security token included in the request is invalid. status code: 403

Also, the .aws/credentials file is empty inside the grafana docker image

Grafana version=9.4.1