0
$\begingroup$

Rosanswers logo

I currently have a robot running Ubuntu with ROS installed and an Asus Xtion Pro Live attached. What I'm trying to do is to capture the video and push it to a server, and then have the server re-stream it so that other clients that connect to my server can view the stream. The server is running Windows, has a public ip and does not have ROS installed.

What I currently have now is, while the robot is on the same wifi as a client, use mjpeg_server, and have the client connect directly to the robot to view the stream. What I'm trying to overcome is, if I wanted to use my robot outside and use a 3G/4G dongle, I wouldn't be able to connect to it directly. Or, if I wanted to view the video stream from outside the network. So I want it to push the stream to my server, and then have clients connect to the server to view the stream through a webapp. The server currently runs a webapp on tomcat, using java, javascript jsp, jquery, mssql. I want to add on to the webapp to allow viewing the video stream from the Asus Xtion on my robot, without having to install ROS on the server if possible.

I roughly know that there will be three things I have to do:

  1. Stream the Asus Xtion camera feed to the server,
  2. Receive the stream on my server somehow, and
  3. Have my server re-stream the video.

But I don't really have any idea on how to go about doing this. Can anyone help?

EDIT: I know that with VLC, I can pull a video stream, and then re-stream it. However this won't work for me because I won't be able to pull from the robot; it doesn't have a public ip so it wouldn't be accessible.

Originally posted by Andrew.A on ROS Answers with karma: 324 on 2014-01-20

Post score: 3

Original comments

Comment by Tim Sweet on 2014-01-21:
It seems like this is unrelated to ROS, where would you want to fit ROS in? If you just need a rough implementation, it is possible to stream on OpenCV matrix over TCP/UDP sockets, but don't expect excellent performance for the resolution of the Xtion. If you need more fidelity I'd use videolan.org

Comment by Andrew.A on 2014-01-21:
It doesn't necessarily have to use ROS, but I was thinking perhaps I could use ROS to get the stream produced by mjpeg_server, and then push it out to the server.

Comment by Andrew.A on 2014-01-21:
Also, as far as I know, VLC can only pull a video stream, and then re-stream it. This isn't possible for me because the robot doesn't have a public IP, as I've mentioned I'll be using 3G or 4G. I've also stated in my question that I need to push to the server, not pull from the server.

Comment by Tim Sweet on 2014-01-22:
The mjpeg_server package is serving the OpenCV matrix over TCP, so you could try it. To solve the public IP problem I'd suggest using openvpn (openvpn.net). Either way your going to have terrible quality and latency over 3G/4G, so I would do some processing on the robot to reduce the data size.

Comment by Andrew.A on 2014-01-22:
I do not want clients connecting directly to the robot, because its specs are quite weak. I want the robot to push the stream to the server, and then have clients connect to that server instead. Do you know how I can do this?

Comment by Tim Sweet on 2014-01-22:
I'm trying to say your results over 3G/4G are going to be very poor, likely in the order of a frame every few seconds, plus lag of a few seconds. But if you set up openvpn on the robot and server you will be able to use mjpeg_server to do exactly that.

Comment by Tim Sweet on 2014-01-22:
You could implement it directly by serializing each image (using opencv) over a TCP socket. The server would open a port for listening, and the robot would connect to it and send the image, and the server would deserialize it. That would avoid openvpn assuming the server has a public IP

Comment by Andrew.A on 2014-01-22:
Could you perhaps write an answer with more details on doing this? I get that the results over 3G/4G might be poor, but where does openvpn come in? How will this help? Also, in your second comment, would that mean writing my own ROS package to do the serializing? Lastly, server already has public ip

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Rosanswers logo

Okay here we go:

To clarify, this is what I believe your setup is:

  • Robot has ROS installed and has a cellular (3G/4G) dongle. It has a camera on board and you need to stream the image to multiple computers on a different network.
  • Server is publicly addressable (ie it has open ports on the internet, and you have the authority to open those ports). It does not have ROS installed. It will get the image from the robot's camera and make it publicly available. I will assume the server runs Ubuntu, you could adapt the commands below for Windows
  • Multiple clients do not have anything special installed, and need to view the camera stream.

On the cellular thing: I really think you'll get less than one frame per second with this setup.

Anyway, here is how I would set this up:

  1. On the server, open UDP port 1194 to the internet
  2. On the server, install OpenVPN with: sudo apt-get install openvpn
  3. Update: these directions are more accurate: Follow the instructions at: https://github.com/OpenVPN/easy-rsa/blob/master/README.quickstart.md to generate a server key and client key (note you'll need to git easyrsa onto both machines from https://github.com/OpenVPN/easy-rsa). Put the keys and certificates you generate into the /etc/openvpn/ directory on each machine
  4. On the server, copy the server configuration file at http://openvpn.net/index.php/open-source/documentation/howto.html#server to a new file at: /etc/openvpn/server.conf
  5. On the server, update the ca cert key and dh fields in server.conf to match the keys you generated in step 2
  6. On the robot, download OpenVPN with: sudo apt-get install openvpn
  7. On the robot, copy the client configuration file at http://openvpn.net/index.php/open-source/documentation/howto.html#client to /etc/openvpn/client.conf
  8. In that client.conf file, change "my-server" in the line remote my-server-1 1194 to the public IP address of your server.
  9. On both the server and robot run: sudo service openvpn restart to have them both connect through the VPN tunnel.
  10. To test the configuration: on the robot, ping the server with: ping 10.8.0.1, then on the server ping the robot with: ping 10.8.0.6. If one of these fails, let me know and we'll debug it.
  11. On the server, install mjpeg_server with sudo apt-get install ros-hydro-mjpeg-server
  12. Set up mjpeg_server to start streaming (I've never used it...but let me know if you have problems and we can take a look together). I'll continue under the assumption this is working and you can view the image by going to http://localhost:8080/stream?topic=/IMAGE_TOPIC
  13. On the server, go to: http://10.8.0.6:8080/stream?topic=/IMAGE_TOPIC and you should see the images. I'd suggest trialing this over Ethernet or WiFi before trying to use the cellular adapter

From here I assume you can figure out the streaming to clients part. You could either re-encode the stream before sending it to clients, or just set your server up so that when a client connects to port 8080 the traffic is redirected through the OpenVPN tunnel to 10.8.0.6. I believe this is pretty simple, just like redirecting any other kind of traffic similar to a WiFi access point does port forwarding.

And that's my simple 14-step process for robot to client video streaming :) Good luck, let me know if (and when) you run into any difficulties.

-Tim

Originally posted by Tim Sweet with karma: 267 on 2014-01-24

This answer was ACCEPTED on the original site

Post score: 3

Original comments

Comment by Andrew.A on 2014-01-26:
My server is running Windows though. Would this mean a big change in your solution? I'll follow what I can and ask you once I get stuck.

Comment by Andrew.A on 2014-01-26:
Also, for point 11, do you mean to install mjpeg_server on the robot? Because I haven't got ROS installed on my Windows server.

Comment by Andrew.A on 2014-01-28:
I've managed to setup openvpn on Windows. Just some small changes: the server file is server.ovpn, the keys and server config file should be placed in the config dir C:\Program files\OpenVPN\config. Setting up on the robot now...

Comment by Andrew.A on 2014-01-28:
I'm having some trouble creating the tunnel, I think. I can't ping from server to client and vice versa. However, on the Windows server, I can ping 10.8.0.1 and get a response. On the robot, I can't ping 10.8.0.6; there's no response at all. Seems like there's some problem on the client side.

Comment by Andrew.A on 2014-01-28:
I'm getting some error, require nsCertType=Server. Full output here: http://pastebin.com/s9uUR5nk thanks

Comment by Andrew.A on 2014-02-04:
Okay I managed to fix the error. I had to replace the ns-cert-type server line in client.conf with remote-cert-tls server. But now I'm getting another error about operation not permitted, and some error getting interface flags: no such device. Full output at http://pastebin.com/ZgwUB9Ve thanks

Comment by Andrew.A on 2014-02-04:
Okay it was a simple problem. I just had to be root to do it. Everything seems fine and should be working right now... but I can't ping from server to client, or client to server. Each can ping themselves though. Dyou know what's wrong?

Comment by Andrew.A on 2014-02-16:
Forgot to add the step here. Just needed to run the windows OpenVPN GUI as administrator, then everything worked. Thanks!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.