I want to try out drone visual odometry using a thermal camera and ORB-SLAM3 running on ROS and Jetson. The device would likely be Orin Nano or AGX Xavier but I'm flexible on this and will use whatever works.

I'm completely lost in the interfaces terminology though. Jetson devices seem to support CSI-2 but also V4L2 and MIPI are mentioned. I'm also pretty sure RTSP is mentioned as the easiest way to publish video to be consumed by ROS and ORB-SLAM3. I haven't been able to find any thermal cameras or dev boards that would support RTSP.

Unfortunately, cheaper thermal cameras like Foxeet don't specify much about their output and I haven't found any development boards for them. My understanding is that the vast majority are analog CVBS output meant to be connected directly to VTX (video transmitter). Forum discussions about trying to convert CVBS to anything readable by Jetson are unconclusive and mention a lot of latency and incompatibility problems so it seems to be a no-go.

I'm also looking at FLIR Boson and while there seem to be some videos demonstrating it working, I'm not sure how exactly they are wired, whether it would work with a specific camera and computer pair, and which interfaces are used.

Seek Thermal Mosaic Core cameras seem to have a USB breakout board but discussions mention it not being recognized as a universal UVC device, which seems to mean that a custom driver is needed which is way above my skillset.

Cheaper USB cameras are probably easier to integrate but I would look into at least a 320H resolution and don't see any that satisfy this.

I would appreciate any help, pointers, or success stories on using thermal cameras with Jetson hardware where somebody would share which hardware stack would give the best results with minimum headache for a beginner.

Alternatively, any 101 on how to determine whether any camera would work with Jetson and what technologies are involved would be a great help

Thank you!

  • 1
    $\begingroup$ I might be wrong, but (unfortunately) this seems like a cross between a shopping question, and a question that will attract opinion based answers. There may not be a "correct" answer, as each of the options will have their own benefits and disadvantages. Personally, I don't know much about thermal camera, but as a pointer look at (and get familiar with) gstreamer - if you aren't already. This video (and all of Paul McWhorter's other videos) is an excellent intro: Jetson Xavier NX Lesson 4: Understanding Gstreamer for Absolute Beginners $\endgroup$ Mar 27 at 20:07
  • 1
    $\begingroup$ @Greenonline Thank you!I was just listing my research and trying to make sense at what will work and what won't. Much appreciate the link, it will help me make sense and ask something more specific. $\endgroup$
    – Alex Bausk
    Mar 29 at 12:05
  • 1
    $\begingroup$ You are welcome. Paul McWhorter has some other videos about gstreamer, I think on the Jetson Nano tutorial series. Both of his Jetson Nano and the Xavier series are worth watching all the way through. Good luck! :-) $\endgroup$ Mar 29 at 12:58