2
$\begingroup$

We have a high-resolution Riegl laser scanner and mounted atop it a Resonon Pika L, which is a hyperspectral camera which records one spatial column at a time, using the second dimension of the sensor for the wavelength spectrum from 400 to 1000 nanometers.

Now we would like to label each scan point with hyperspectral data. For this, we

  • record a fixed angular window with the laser scanner
  • stitch together the individual spatial columns from the hyperspectral camera rotating with the scanner
  • Identify chessboard corners in the point cloud
  • also find them in the stitched hyperspectral panorama (reduced to rgb).
  • Using OpenCV's calibrateCamera we try to get a camera matrix, which I assume will only be valid for this particular panorama size (if at all)

and then we could in theory obtain rotation and translation between scanner and panorama coordinates using solvePnP.

Is this a valid way to go about solving this problem? There doesn't seem to be much prior art in this regard.

Improve this question
$\endgroup$
2
  • $\begingroup$ @Chuck So you mean finding some marker directly in the "1d" image from the line camera? $\endgroup$
    – oarfish
    Jun 15, 2017 at 19:30
  • $\begingroup$ I realized I was getting chatty in the comments so I just moved everything to an answer below. $\endgroup$
    – Chuck ♦
    Jun 15, 2017 at 19:54

1 Answer 1

Reset to default
1
$\begingroup$

I converted my comments to an actual answer:

If I understand your setup correctly, you're saying you have a line scan camera mounted to the top of the rotating head of a laser scanner, and all you're looking for is the one-time transform matrix between the scanner and camera coordinate frames. Is this correct? If so, why not just perform the test in a controlled environment where the features are easy to discern?

[The spectral camera doesn't really have] a 1d image if you have the encoder feedback from the Riegl. But yes, if you could generate a shape (easily detectable by the laser scanner) that also had some interesting spectral content (easily detectable by the hyperspectral camera), then you could compare encoder counts where the same feature were located in both systems.

As your spectral camera appears to cover the visible spectrum, you could use an exterior door (presumably the wall is one color, and outside is "noisy" colors), or a table, or an open window, or a or a monitor or any colored, physical object.

I don't know if you're trying to mass produce something, but probably the easiest way to do this is just to manually align the two outputs until you have good coherence. I don't know if this is the point of what you're trying to do or not, but again you have visible spectrum data from your line camera. You could segment out the RGB information from that data, use it to texture your point cloud, and then tweak the transform values until your texture overlay "fits" with the depth information you've got from the laser scanner.

Of course if you're mass producing the thing and/or you plan on constantly re-aligning the data, then you would probably want some way to automate this, but automation is not always the answer.

It would be really neat if it could align itself, but please please consider the time cost of just doing the alignment manually versus the time cost of writing a custom alignment algorithm for scratch. If you have a rigid coupling between the two sensors (your project doesn't seem to be hurting for money!!) then you shouldn't need to do the alignment more than once.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I will consider your points later, but one remark I should make is that the line camera will be detached and reattached frequently and the placement isn't completely consistent (deviations should be small, but still). Therefore, a semiauto process would be great. $\endgroup$
    – oarfish
    Jun 16, 2017 at 18:28
  • $\begingroup$ What do you mean by encoder feedback? The problem is that the laser scanner creates a complete scan and the line camera outputs a continuous feed of images throughout the entire time, and it is not clear, whether the scanner's laser beam is exactly at the center of the window recorded by the camera. Not sure if I can rely on the marker being visible in the camera if it's directly in front of the scanner, there may be an offset. $\endgroup$
    – oarfish
    Jun 18, 2017 at 8:25
  • $\begingroup$ @oarfish - You are correct in that it will be difficult (nearly impossible) to get the line camera exactly aligned with the beam emitted by the laser scanner, BUT as long as the camera is rigidly connected to the rotating scanner head, then you can use the current scan angle provided by the laser scanner as a position encoder for the line camera, too. When the initial position repeats, the camera has made one revolution. Now you have a cylinder of points from the scanner and a cylinder of images from the camera. Rotate the two by hand until the colors correctly relate to the depths. $\endgroup$
    – Chuck ♦
    Jun 19, 2017 at 0:15
  • $\begingroup$ My point was that you should only have to do this once and you'll be set until you need to remake the rigid connection. I don't know how accurate you're trying to be with your alignments, but bear in mind that the angular alignment (manually rotating the cylinders) is much more important, especially at distance, than the z translation between optical centers of the devices. Translation error is fixed; angular error grows as r$\theta$. You're manually finding the offset angle that relates scanner angle to camera angle. Once you find that you're golden. Then do the same with the z-axis offset. $\endgroup$
    – Chuck ♦
    Jun 19, 2017 at 0:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.