2
$\begingroup$

Are there any specific tools or practices which robotics companies use in order to keep track of errors in:

  1. Sensors/Actuators malfunctioning or returning unexpected results
  2. Programming errors or exception states

The aim of tracking this diagnostics information would be to:

  1. Remote diagnosis of errors
  2. Root cause analysis
  3. Understanding of errors which 'cascade' from devices/robots to platforms and cloud services

I'm interested specifically in the parallels between normal software tools, e.g. Sentry or Rollbar for exception tracking, APM systems such as Dynatrace and the distributed and long lived state of the autonomous systems world. These systems seem to be better suited to errors in a request/response format, where the history of a device does not have as much affect on it's future as in the autonomous systems world.

So far the only tool I've seen is the diagnosis tools built into the ROS messaging bus, however it does not seem well suited to production.

How are teams currently doing this in production?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Company size and market expectations shape the specific tools used. Almost always kept confidential to some degree.

Issue tracking, remote diagnosis (and device security), and stochastic monitoring of hardware and software are common practice.

Failure prediction modeling is common where the economics make failure costly.

There are no specific tools and best practice is sector specific.

You might gain some insight looking at the practices of academic projects that require a large number of robots to be created and shared with other universities (iCub for instance); or by looking at robotics companies that have a strong open source commitment. But projects like these are biased by what tools the project leaders are willing to pay for.

If you are part of a team in a new company with no experience in this you are probably better off hiring an expert or two in your sector than reinventing 100 wheels.

EDIT Note that robotics hardware products can take 5 to 10 years to develop into a mature product, be deployed for another decade or two, and best practices will be from the era of development.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks hauptmech. I'm not currently convinced that academic projects will provide reasonable indications of how monitoring/debugging systems work due to the scale. Am I wrong? You also mention that there's a lot of variability in the tools used etc, do you have any examples of how you've seen this done (e.g. log files pushed to s3)? $\endgroup$
    – hookd
    Oct 1, 2017 at 17:32
  • $\begingroup$ Scale is important, there are academic projects with a large(ish) scale, 10's or 100's of systems deployed of a high enough complexity to need what you are interested in. The linked project is one such. $\endgroup$
    – hauptmech
    Oct 1, 2017 at 22:22
  • $\begingroup$ The problem with robotics is that it is so broad. If you have a specific sector in mind, you should talk to someone from that sector. If you are fishing for whether there is a market need, you need to define what subset is the market you are after and then talk to them. If you are looking for help on best practice for a project you are part of, you need to narrow your question to specify the type of robotics you are doing. $\endgroup$
    – hauptmech
    Oct 1, 2017 at 22:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.