How can I stop AVVideoComposition from pre-rendering frames?

Question

I have a custom compositor that renders frames while applying filters. When I change the filters properties with a slider, the change is delayed since the video composition renders ahead of time and holds the old filter values for those pre-rendered frames. I can fix this by resetting the videoComposition - playerItem.videoComposition = videoComposition. This works well and fixes the issue above, however it creates a new problem. With something like a slider, the change happens so fast that this operation overloads the CPU resulting in a jerky video. I can improve this by adding a step to the Slider and only changing by larger increments, but this is still not perfect and creates a noticeable lag. I would also rather allow the user to make those minuscule changes so this fix just isn't worth it. Does anyone know a good way to disable this feature on AVVideoComposition or custom compositors? It has been a pain in the ass.