I am posting this here because my background in estimation theory and optimization has been developed entirely through my experience in robotics.
TLDR: What makes it so that any time you put something in an exponential you can suddenly call it a probability distribution?
The Deets: I thought of this question while reading A General and Adaptive Robust Loss Function by Jonathan Barron. The paper describes a single parameter loss function that can subsume the most commonly used individual loss functions:
$$f(x, \alpha, c)$$
Where $x$ is parameter, $\alpha$ is the "single parameter", and $c$ is a distance scale that describes the "quadratic bowl" around the origin which all the loss functions share in common. When $\alpha = 2$ it is squared error loss, $\alpha =1$ it is essentially absolute error loss, and when $\alpha = 0$ it is cauchy loss.
My confusion starts when the paper goes on to say "With our loss function we construct a general probability distribution".
$$ p(x|\mu,\alpha, c) = \frac{1}{cZ(\alpha)}\exp(-f(x-\mu, \alpha, c)) $$
My question is, why does dividing by a normalization constant, and taking the exponential of the negative loss function suddenly make it a pdf? I understand the results, I have used these same ideas in my estimation theory courses, but I just don't get how we can make the jump from loss to pdf OR why it is so intuitive to do so.
Cheers, and happy holidays :)