Normal 2D video to 3D SBS

[Minabe]

Does anyone knows of a way to get good results? Most of what i’ve seen is pretty crappy and most “converters” suck, as in they just make one eye be a frame ahead to “fake” the 3D. Mostly out of curiosity as i do have a method of getting somewhat better results using ffmpeg only. By cutting from the sides of the video on each eye to fake the 3D, it works great in some videos but not all.

Figured i lose nothing by asking.

The method i use, uses the following commands:

ffmpeg -i in.mp4 -filter:v "crop=out_w:out_h:x:y" out.mp4 #example code
ffmpeg -i in.mp4 -filter:v "crop=iw-~40:ih:0:ih" right_eye.mp4
ffmpeg -i in.mp4 -filter:v "crop=iw-~40:ih:~40:ih" left_eye.mp4

ffmpeg -i inputleft.avi -vf "movie=inputright.avi [in1]; [in]pad=iw*2:ih:iw:0[in0]; [in0][in1] overlay=0:0 [out]" sbs.mp4 #merger code

[Ralf]

Some 3D TVs had algorithms built in that tried to guess a depth map by figuring out what is foreground and what is background via analyzing motion and/or color/contrast, but those never really worked well. Except for special cases like e.g. a horizontically panning camera, which happens to contain the stereo information a few frames apart, you will mostly get odd results using such methods. Trying to guess information that just isn‘t there is more or less bound to fail more often than not.

Sounds like a potentially good application for an accordingly trained neural network (AKA ‘AI’) though. I’m not sure if such a thing exists, especially freely available, as training such a network until it’s good enough would clearly be a pretty huge task. Nonetheless searching for a neural network based solution is probably your best bet, maybe someone was crazy enough to try.

[Minabe]

Yeah, i would’ve tought such a neural network would have been created already but so far nothing… that’s the only thing lacking imo as we already have good enough ai resizers. Topaz Suite being the best one out there afaik

[Ralf]

Out of curiosity I searched a bit myself yesterday and found some related papers/projects. At least one even including code and training data. Not a simple point and click solution, but if you want to invest serious time and effort it might be worth a closer look. Googling ‚2d 3d conversion movie neural network‘ should get you started.

[Minabe]

Haha, just found it too (Deep3D), tho it seems it’s from at least 2016 as there are 2 githubs (the other from 2018) with the same files that were updated 4 years ago.

I’ll probably try it, need to build the libraries it uses from source but luckily i already have installed most of the things i need. I’ll let you know if it’s worth it.

[Author]

Posts