Resolution:
vorpX is using the resolution set in the game options. For a crisper image you would have to select a higher resolution there. Be aware that a higher resolution looks better, but in turn costs additional performance. So you’ll have to find the right balance here.
BTW: The custom resolutions section in the vorpX help has a guide regarding adding higher resolutions than the ones your monitor can display.
Performance:
With your GPU it might make sense to switch to Z3D in vorpX ingame menu. That doesn’t look as natural as the default G3D, but is a lot faster. If you want to stick to G3D, with your GPU you definitely will have to follow the suggestion in the top/left corner of the game window and reduce the game’s graphics detail to medium or low.