Yes, the original DX9 version is usually better suited for vorpX unless you use huge amounts of mods/texture packs that exceed the 32bit memory limit of the DX9 Skyrim.
Usually the original Skyrim performs better, even more important is that it allows actual multisampling antialising instead of the very blurry post process AA of the Special Edition, which just doesn’t work really well in VR.
I find the stability of SE to be much higher, but it has this weird fog effect which seems like it’s between you and what you’re looking at. It isn’t just blurry, it’s worse – it’s like looking through a filter or something.
I have both versions and have gone back and forth between them at the same place and same time in the game world, and regular Skyrim just looks crisper and more real. Just load the mods you really want, and don’t go crazy with textures.
I have found – and this may just be me – that some of the crash fix mods for regular Skyrim seem to cause Direct VR to not work. So another reason to be selective in what mods you load.
All of the above said, regular Skyrim with VORPX is the best VR experience I’ve had, and it plays beautifully. Do hot key Edge Peek to something easy to reach to make the menus workable.