Properly charting cumulative, monotonically increasing metrics in MQL

Question

I'm emitting metrics in google cloud counting how often certain jobs are executed and succeed vs. fail. These are cumulative metrics, so the actual values emitted should be monotonically increasing.

When I graph these metrics, for each job I would expect to get 3 lines, each generally increasing. Instead of graphing what's being emitted, it seems MQL is destroying the cumulative nature of these metrics.

visualized metrics, not cumulative for some reason

I would expect to see more of a step function, where the last value charted is 6 not 2.

The relevant MQL is:

fetch_cumulative gce_instance
| metric 'custom.googleapis.com/opencensus/foo_job.counter'
| group_by [metric.eventName],
    [sum(value.counter)]

My understanding is that fetch would convert cumulative metrics to delta metrics and thus result in this behavior. But fetch_cumulative should avoid that.

Answer 1

I believe fetch_cumulative should do the job. However, according to the footnotes, the charting in the UI might convert it automatically into delta streams. Maybe try the API call directly to confirm this behaviour.