Combining multiple constraints in optimization

Question

Let's say we have two equations for a pose $\textbf{R},\textbf{t}$ optimization problem.

For ICP constraint $$\textbf{e}{_{I}}=\textbf{n}^\top(\textbf{p}_d-(\textbf{R}\textbf{p}_s+\textbf{t}))$$

For epipolar constraint $$\textbf{e}{_{c}}=\textbf{u}_s^\top[\textbf{t}]_{\times}\textbf{R}\textbf{u}_d$$

Then, for the optimization we define an objective function as follows and find $\textbf{R},\textbf{t}$ that minimize it. $$\mathbf{J}=\alpha\frac{1}{2}\sum_{v=1}^{V}{\textbf{e}{_{I_v}}^\top\boldsymbol{\Sigma}_{I_v}^{-1}\textbf{e}{_{I_v}}}+\beta{\frac{1}{2}}\sum_{j=1}^{J}{\textbf{e}{_c{_j}}^\top\boldsymbol{\Sigma}_{c_j}^{-1}\textbf{e}{_{c_j}}}$$

My question is that what is the common practics to decide the weights $\alpha,\beta$. The scales of each constraint are too different. If we do not set it properly, the one with very small scale will be ignored. I have been finding these values manually, but I believe that there are better ways.

Answer 1

Typically you would try to 'normalize' the 2 quantities so that they have the same magnitude and can be compared.

Let's say you know the worst case error or the average error, or some thing of the sort, for the to quantities. Then you simply divide both quantity by that.

In the case you know the maximal error it is going to give you a number between 0 and 1.