0
$\begingroup$

I have a robot that takes a measurement of its current pose in the form

$$ z = \begin{bmatrix} x\\ y\\ \theta \end{bmatrix} $$ $x$ and $y$ are the coordinates in $XY$ plane et $\theta$ the heading angle.

I also have ground truth poses $$ z_{gt} = \begin{bmatrix} x_{gt}\\ y_{gt}\\ \theta_{gt} \end{bmatrix} $$ I want to estimate the standard deviation of the measurement. As the standard deviation formula is:

$\sigma=\sqrt{\frac{1}{N}\sum_{i=0}^{i=N}(x_{i}-\mu)^2}$

Is it correct to calculate in this case where I have only one measurement, i.e

$N=1$:

$\sigma_{xx}=\sqrt{(x-x_{gt})^2}$

and the same for $\sigma_{yy}$ and $\sigma_{\theta\theta}$ ?

Edit The measurement is taken from a place recognition algorithm where for a query image, a best match from a database of images is returned. To each image in the database is associated a pose where this image has been taken, i.e $z_{gt}$. That's why I only have one measurement. In order to integrate it into the correction step of Kalman filter, I want a model of measurement standard deviation to estimate the covariance having the following form:

$$ \Sigma = \begin{bmatrix} \sigma_{x}^2 & 0 & 0 \\ 0 & \sigma_{y}^2 & 0 \\ 0 & 0 & \sigma_{\theta }^2 \\ \end{bmatrix} $$

Improve this question
$\endgroup$
2
  • $\begingroup$ Are the query images taken on-line, or are they from the database? If the former, then you can have many measurements by using on-line pictures, and the error corresponds (roughly) to the average area of space associated with each image in your database. $\endgroup$
    – combo
    Sep 22 '17 at 22:10
  • $\begingroup$ The query is taken online, but in order to validate my model, I only have two sequences of the same path. The first is used in the learning step to construct the database and the second is used to recognize the route. Should I use a machine learning method to validate my error model like cross validation? $\endgroup$
    – Daphnee
    Sep 22 '17 at 22:17
1
$\begingroup$

In short; no, you can't estimate standard deviation from a single measurement. Your last equation, assuming $\mu = x_{gt}$ is invalid because you have no guarantee that the mean measurement will match the ground truth (the sensor could have bias error).

The first formula you give is used only if you sample the entire population of possible values, which means the distribution would have to be discrete. I'm guessing in your case the distribution includes all real numbers (a continuous distribution) so the best you can do is estimate the real standard deviation via the sample distribution. In this case the formula you want is: $$s = \sqrt{\frac{\sum_{i=1}^N \left ( x_i - \bar{x} \right )^2}{N-1}}$$ where as $N \rightarrow \infty, s \rightarrow \sigma ^{(a)}$. Since the denominator is zero for $N=1$ the sample distribution would be undefined.

You can approximate it in a few ways. Most common is to take many measurements and apply the sample standard deviation formula. Another rough option is to look for accuracy specs from the vendor and assume that full range means 6 standard deviations (e.g. $\pm 1$cm -> range of 2cm -> $\sigma = \frac{2}{6}$cm). This is further assuming a normal distribution and the accuracy spec covers 99% of all values.

$^{a}$This isn't exactly true, even for the unbiased estimate I gave, but it's close enough to true to be the most commonly used form

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the answer. Please see my edit. For the example $\sigma=\frac{2}{6}$ the $6$ stands for $3\sigma$? $\endgroup$
    – Daphnee
    Sep 22 '17 at 21:31
  • $\begingroup$ yes, going from $-3\sigma$ to $+3\sigma$. Regarding your edit: it's hard to say since I can't quite picture the details what your filter is actually doing, but in most cases the covariance used by a Kalman Filter is 'tuned' rather than computed. i.e. people will increase the covariance (rely on the model more) if the prediction is too noisy or decrease the covariance if the prediction is lagging too far behind. $\endgroup$
    – ryan0270
    Sep 23 '17 at 0:56
2
$\begingroup$

Short answer: You need at least two measurements and you should use many more.

Definitions

The standard deviation is defined as $$\sigma = \mathbb{E}[x^2]-\mathbb{E}[x]^2,$$ which is the difference between the first and second moments of $x$.

The sample standard deviation estimates this using $N+1$ samples, $$\hat{\sigma} = \sqrt{\frac{1}{N}\sum_{i=0}^N (x_i-\bar{x})^2}.$$ Which converges to $\sigma$ as $N\to\infty$. Note that if you use a single sample, then $\bar{x} = x$, and so you get $\hat{\sigma} = 0/0$, which is undefined. This makes sense, since you cannot infer variance (or standard deviation) from a single point.

Convergence Rate

The problem is that $\hat{\sigma}$ is just an estimate of $\sigma$, and that estimate can be very bad for small $N$. Wikipedia has a good article on the convergence properties of the sample standard deviation. For the normal case, the variance of $\hat{\sigma}^2$ is $2\sigma^4/(N-1)$, but in general it may not converge this quickly.

Noise distribution

One last (very important) point: instead of trying to find the standard deviation of the measurement, you should be looking at the standard deviation of the error. So in this case, your $x_i$ are the difference between the ground truth and observed value, and $\sigma_{xx}$ is the standard deviation of the error. The reason to consider error is that this way you don't conflate motion with noise. Furthermore, since you have the Kalman filter tag, the covariance that the filter is expecting is for the errors.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for the answer. Concerning the standard deviation of the error, should I calculate the error, $e_{x}=\sqrt{(x-x_{gt})^2}$ then see the range of the error wrt $x_{gt}$, let's say $\pm10$, then $\sigma_{x}=\frac{10}{30}$. I do the same for $\sigma_{y}$ and $\sigma_{\theta}$ and the filter expects the follwing covariance matrix: $$ \Sigma = \begin{bmatrix} \sigma_{x}^2 & 0 & 0 \\ 0 & \sigma_{y}^2 & 0 \\ 0 & 0 & \sigma_{\theta }^2 \\ \end{bmatrix} $$ ? $\endgroup$
    – Daphnee
    Sep 22 '17 at 22:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.