Did you run the Direct VR scanner after entering the actual gameworld to adjust field of view and head tracking?
Direct VR games (incl. Skyrim) are the ones that work best. Current games with Direct VR support are: Fallout 4, Skyrim, Skyrim Special Edition, Half-Life 2, Left 4 Dead 2, Portal 2, Borderlands 2, Bishock (original), Bioshock 2 (original), Bioshock Infinite, Aliens: Colonial Marines, Fallout 3, Fallout New Vegas, Black Mesa Source. More to come shortly.
it might be my processor, the performance wasn’t great. Maybe bioshock is more my level for the time being
Oh, the other thing that weirded me out was that head tracking was weird, rotating to the sides were like looking at a big screen, then when my eyes hit the edge of the screen the rotation starts. Is that how Vorpx is? or is there a way to set it so that as soon as my head rotates the camera rotates?
Adjust the detail settings in the Skyrim launcher. Running the game (any game) with vorpX is a lot more demanding than running it on your monitor, mostly due to the stereo rendering. You should make sure that games run at 45fps, which you can check with ALT+F. 45fps is what is required for a comfortable experience.
Also make sure to run the Direct VR scanner in games with Direct VR support (you see a message about that when starting a game if that is the case). That takes care of head tracking.