SteamVR’s New Frame Throttling Motion Smoothing Feature (And RTX)

[Adil3tr]

SteamVR got an update that lets you not just motion smooth with 1/2 the frames interpolated, but up to 5/6 frames, It actually is quite playable on an index running at 144hz, so you just need to generate 24 frames and a little overhead. Does this work with VorpX, it runs in the steamVR compositor so I would expect it to but there could be little issues.

Also really interesting is what kinds of opportunities this creates. Are there games that work in vorpx but only at very low framerates? Do any RTX games work in VorpX as G3D, especially like Quake 2 RTX? It would be amazing as a demonstration.

[Ogrescar]

It may work with vorpx but I seriously doubt it – anything that messes with headset synchronization is probably going to make vorpx nauseous. Also doesn’t work with Rift or WMR headsets so probably isn’t worth the effort to make it compatible.

Quake 2 RTX uses the vulkan api, which vorpx doesn’t currently support. As far as ray tracing in general, I don’t think there’s a GPU in existence that can do G3D and RTX at the same time at a reasonable frame rate and resolution.

[Adil3tr]

It’s still worth it, there are hundreds of thousands of Index’s and Vive’s, and WMR may add it eventually. As for whether you could get it to do RTX, the whole point of this is that you have to render a fraction of the frames. If you can hit 24fps with at least 90% SS, you’re in.

[senoctar]

The issue with any kind of interleaving reprojection is that the VR framework interacts with vorpX and is largely unaware of the game. When the framework limits frame rate (ex. half the display refresh) in order to interleave synthetic frames, vorpX itself will be limited but the game is not. This induces a larger and variable delta between a frame render time and display time. The GPU is also overloaded with frames that are just dropped.

Fortunately vorpX has two settings to help with this:

Headset Sync

When set to “Safe” vorpX will sync game frames with the VR rendering. I presume this isn’t possible to do perfectly. There’s always going to be some overhead in syncing and the VR distortion/render itself. So the game FPS will be lower than needed for smooth motion even if the GPU could push more frames. In my experience this is better than nothing but still exhibits some stuttering.

Direct Mode FluidSync

When set to “On” vorpX will limit the game frame rate to half the display refresh. It’s important vorpX uses the display refresh rate not the VR framework limit. With 1/2 synthetic frames the frame rate that vorpX requests from the game and the rate the framework requests from vorpX will be the same.

Thus leaving Headset Sync as the default “Device (Fast)” and changing Direct Mode FluidSync to “On” will work great with 1/2 reprojection. At least if your GPU can keep a stable FPS.
Personally I’m more susceptible to stuttering and low FPS than to reprojection artifacts. So I’m using this setup for most games. I haven’t noticed a quality difference between vorpX and native VR games. This is on the WMR platform. My understanding is reprojection in SteamVR and Oculus can also use depth information so there might be some difference there.

Just to note some platforms have “auto” reprojection which enables when the framework detects the game can not keep up with the device’s refresh. This won’t work with Direct Mode FluidSync because vorpX duplicates every frame and the VR framework will think the game is doing just fine.

For 2/3 or more synthetic frames I don’t think there is an ideal solution with vorpX right now. We could lobby Ralf to add additional options for Direct Mode FluidSync like “On 1/2”, “On 2/3”.
I don’t have this on WMR but I hope it’s implemented at some point and vorpX will be able to use it.

[developer remark: Unfortunately the above descriptions of the two vorpX settings aren’t fully correct (largely correct for ‘FluidSync’, not correct at all for ‘Headset Sync’. To avoid confusion please see the post below.]

[senoctar]

Reprojection with more than 1/2 frames should work with vorpX because it’s managed by the VR platform.

While most games should work well by default, there can be some issues due to synchronization. If you notice stuttering try to set “Direct Mode Async Render” to “Off” (in vorpX display settings).

For fixed 1/2 reprojection the better option is to keep async mode On and set FluidSync to On as well. Again, if you notice stuttering (with default settings vorpX will adjust frame rate by itself and should be fine most of the time).

[Ralf]

Short remark regarding senotcars post two posts above: It almost correctly describes what FluidSync does, but the part about the ‘Headset Sync’ setting is not correct at all.

Headset Sync

That’s a rather technical setting which defines which method vorpX uses to synchronize the game and the headset thread. Always leave this setting at ‘Fast’ unless you encounter situations where the headset frame rate can’t keep up (usually a sign of GPU overload). The noticeable effect of ‘Safe’ is more leeway for the GPU on the headset thread at the expense of game frame rate. Don’t use unless necessary.

The ‘Headset Sync’ setting does not affect the motion smoothing behavior in any way.

FluidSync

The noticeable effect of this setting is capping the game frame rate at half the headset refresh rate. There’s a bit more to it behind the scenes, but that’s what you should be aware of. Depending on the headset it may also cap the headset thread frame rate (but usually doesn’t).

General Remark

I’ll take a look at the new SteamVR smoothing shortly, but unless it solved inherent artifacts that so far plagued each and every video smoothing technique, I’d recommend to not use it with vorpX. Just as your TV’s ‘soap opera mode’ VR motion smoothing is more or less bound to fail in situations where there either are conflicting motion directions within a small space (obvious examples: legs of running people, galloping horses, cars crossing each other etc.). Same for static elements in front of fast moving stuff (e.g. TV station logos or in case of games, even worse, crosshairs or HUD elements directly in the center of your view). Situations like these often cause pretty ugly artifacts when a motion estimation based frame interpolation algorithm takes its guesses.

vorpX per default tries to always keep its headset thread frame rate at the full refresh rate and maps potentially lower game frame rates via head rotation interpolation to the higher headset frame rate. That doesn’t interpolate the whole image, but it interpolates what is most important for comfort in VR (head rotation) without any artifacts at all.

[senoctar]

I confused Headset Sync with Direct Mode Async Render. I played around with async some time ago but did not find it necessary to change the default. So I never used it and forgot which one it was.

Since posting I’ve been testing the settings again and realised most of what I wrote is wrong. Should have done that before. Thanks for highlighting this in the post.

As for reprojection itself it seems rather subjective and depends on context. In FUll VR with G3D I find it worse than just FluidSync. In Cinema with Z3D I find it better, and with G3D it’s a coin toss.
It also depends on the headset. There’s a point where increasing resolution/details has diminishing returns due to the headset’s display. With the Rift S my GPU could maintain a decent frame at that point, with the Reverb G2 it can’t.

[Author]

Posts