Smart camera image processing on embedded GPU vs. in the cloud

Question

I'm working on a smart camera that does key point prediction (predicting locations of wrists, elbow, shoulder, ears, eyes, nose, etc.) for gesture recognition.

Right now, the neural net is running on the embedded GPU (a Jetson TX2) and performance isn't ideal (<3 FPS). So I'm exploring whether it makes more sense to constantly upload images to the cloud, doing the predictions there, and sending the results back to the device.

I'm curious what approach others would recommend for a smart camera. Specifically:

1. Performance differences between using an embedded GPU vs. the cloud?
2. Cost differences between using an embedded GPU vs. the cloud?
3. What other smart cameras are doing (Nest, Lighthouse, etc.)?
4. If there is an alternative better than the Jetson TX2 if going the embedded path?

Any advice for any of the questions would be great.

Answer 1

Performance differences between using an embedded GPU vs. the cloud?

For your context (to my mind, gesture recognition would ideally be realtime), the latency of going over the cloud would push me towards hardware.

Cost differences between using an embedded GPU vs. the cloud?

Depends on your volume. The cloud is not necessarily cheaper than hardware in the long run, although in some ways it's less headache especially in the short run. For gesture tracking, though, the latency would be too annoying to put up with forever.

What other smart cameras are doing (Nest, Lighthouse, etc.)?

If you're considering a Lighthouse - we use a lot of Realsense here, and find it quite satisfactory. Looking at getting one of the new ones any day now.

If there is an alternative better than the Jetson TX2 if going the embedded path?

I'm not sure it's the TX2 that's the problem - 3fps, what sensor are you using? The TX2 is a beast, and given a half-decent sensor (say, Realsense R200) you almost don't need one. That stated, have you looked into incorporating a Movidius?

HTH.

Answer 2

If there is an alternative better than the Jetson TX2 if going the embedded path?

You can try https://developer.qualcomm.com/software/snapdragon-neural-processing-engine

I did not see any comparison between different Snapdragon's and Jetson and I am not sure where you can find top level Snapdragon 835 except some fresh Android phones.