While the books give general formulas, you need to have enough understanding to adjust them based on your specific application. The textbook definition of error is:
error = goal - position
position is your current state and
goal is the state you wish you were at. While this definition is ok for a cartesian space, in others, as you have observed, is not the best thing.
With your own system, where the
position are angles, you have in fact infinite possible values for
$goal = 2k'\pi+\alpha$
$position = 2k''\pi + \beta$
$error = 2k\pi + (\alpha - \beta)$
In your case, the
error could be 340, 700, 1060, -20, -380 etc. Clearly, depending on what value you choose for the
error (all of which are correct), the controller behaves differently. If you choose 1060 over 340 for example, the controller thinks it's much farther from the
goal and therefore tries to get to it in much more of a hurry.
You would usually want the minimum movement to reach your
goal, so from the values above you would want to choose -20. That's quite easy. Simply choose the value of
error in the range $[-\pi, \pi)$ (or $[-180, 180)$ if you want). That would have the minimum absolute value among the infinite choices.