3D reconstruction of a moving object from two camera video stream

Question

I have two synchronized video streams of a moving deformable object from two different cameras (same brand/model type) but two different views.

How can I do the 3D reconstruction of this moving object?

The 3D reconstruction needs not be real-time. I already have done the capture and need to do this as a postprocessing step.

A few more details:

My camera is FLIR SC8000 camera which has thermal videography. The current format of video is sfmov (SAF movie) which has 2 bytes (RGB+ count values -as for thermal aspect) and can also be converted to RGB (1 byte per pixel) too in WMV format. I am also able to convert the video stream using th ResearchIR Max+HSDR software to a series of lossless JPEG images that are called Radiometric JPEG (2 bytes per pixel).

Answer 1

If the cameras are stationary it should work to use Structure from Motion (https://github.com/mapillary/OpenSfM). Failing that the cameras are stationary you could attempt to create a factor graph in gtsam to solve for the camera movement, object movement, and projections between the cameras and object. See a simple SFM setup here: https://gtsam-jlblanco-docs.readthedocs.io/en/latest/StructureFromMotion.html or here http://nghiaho.com/?p=2379

Answer 2

First you have to use OpenGL to capture images and process fast While the pictures are JPG technology Very slow to process a single frame taking longer Second, anything that is in the frame of the picture must be transformed into waves An example of this Triangle shape Beginning, rows of pixels of the color type are combined from right to left, and the result is a waveform number one Then rows of pixels of color type are combined from top to bottom, and the result is a vector number two Then, router number one and router number two enter the database, and here the program knows that this data is that this is a triangle shape Even if you enter a large or small triangle, the program will tell you that this is the shape of a triangle And so on the rest of the geometric shapes