Can I Trigger a Cloud Function Based on a Pub/Sub Subscription?

Question

I'm currently working with a system that receives millions of events and I need an efficient way to handle these. My goal is to trigger a cloud function only for certain events, not every single one, to ensure efficient processing and resource utilization.

To achieve this, I was considering using a Pub/Sub model. Specifically, I'm thinking about setting up a Pub/Sub subscription that filters events based on specific criteria relevant to my use case. Then, only if an event matches these criteria, it should trigger a Cloud Function. This way, I can avoid unnecessary invocations of the function and focus only on the relevant events.

My questions are:

Is it possible to trigger a Cloud Function based on a filtered Pub/Sub subscription? If yes, how can I set up this filtered subscription to ensure only specific events trigger the Cloud Function? Are there any best practices or limitations I should be aware of when implementing this approach? I'm aiming for a solution that can handle high throughput efficiently while minimizing unnecessary executions of the Cloud Function. Any insights or guidance on this would be greatly appreciated.

Thanks in advance for your help!

Answer 1

Indeed, when you create a Cloud Function on a topic, a subscription is created but without any filter. And you can't update the subscription filter after the creation. So, in that way you are blocked.

The alternative is not to use the built in mechanism, but to do it manually:

• Create a HTTP cloud functions (not a background function)
• Create a PubSub subscription with the correct filter and a service account
• Be sure the service account of the PubSub subscription has the permission to invoke the Cloud Functions

It requires a like rework on your side, but it's more generic by this way.

Answer 2

Cloud Functions can be triggered by Pub/Sub but by its topic rather than the subscription [Docs Here ]

You could add a filtered subscription As explained here attached the sub to the topic that when receiving the filtered message will trigger your CF.

As for the limitations, I'd take into consideration the amount of messages that you might received as it might scale the CF more than you'd like to. Also, if you have other subscriptions filter those ones to NOT receive these messages.

As best practices, I think that the usual, use a SA that has IAM permissions to publish to the topic, to invoke the CF, configure the CF to be internally if does not has to be public (If it has to, use a LB) and so forth.