If you aligned the IMU "well", then this should be a relatively trivial problem. Consider your statement that you can reconstruct the path using only "robot encoders" (wheel encoders?) and the yaw rate. Now, if you have the IMU configured such that the X-axis is the forward/reverse direction, then double-integrating the x-axis accelerometer should duplicate the wheel encoder information. The only information the Y-axis accelerometer should tell you is if the robot is sliding sideways.
If you have the IMU aligned "poorly" then you could have some projection of the x- and y-axis along the forward/reverse axis, at which point you would need to take the correct projection from each and add them together to get the forward distance.
Regarding rotation matrices, whatever you're doing currently to apply it to the (wheel encoder?) robot encoder should work the same when you convert the acceleration data to distance.
:EDIT:
Assuming your yaw angle is a z-axis rotation, your rotation matrix should look like:
$$
R = \left[ \begin{array}{ccc}
\cos{\theta} & -\sin{\theta} & 0 \\
\sin{\theta} & \cos{\theta} & 0 \\
0 & 0 & 1 \end{array} \right] \\
$$
You should have some starting heading, $\theta_0$. You also need a starting velocity $v_0$ and a starting position, $p_0$. Now, you get samples at some update rate, where the time between samples, the sample time, is $dT$. So, from physics, you can calculate your new distance and velocity:
$$
p_N = p_{N-1} + v_{N-1} dT + \frac{1}{2} a_N dT^2 \\
v_N = v_{N-1} + a_N dT \\
$$
where $N$ indicates the current sample and $N-1$ indicates the previous sample. In this instance your initial conditions, $p_0$ and $v_0$ are used and you get your first update, $p_1$ and $v_1$ from your first accelerometer reading. I've used $p$ for position instead of $x$ to try to keep the concepts separate - the accelerometer will give you some position, but it doesn't get to be a coordinate until you've reconciled it back to your world coordinates. This is done with the rotation matrix:
$$
\left[ \begin{array}{ccc}
x_N \\
y_N \\
z_N \end{array} \right] = R_{N-1} * \left[ \begin{array}{ccc}
p_N \\
0 \\
0 \end{array} \right] \\
$$
Here I've shown the $N-1$ rotation matrix, the rotation matrix based on the previous yaw reading (or starting heading, if you're considering the first sample), but your choice of which angle to use (previous sample, current sample, an average, etc.) shouldn't actually make a big difference for the same reason that trapezoidal integration and rectangular integration both converge on the same result. As long as the sample time is sufficiently short relative to the motion of your vehicle, any errors accumulated going one direction should generally be canceled out going the other.
I would say if you wanted the most reasonable yaw estimate, use the average of the previous and current headings:
$$
R = (R_{N} + R_{N-1})/2 \\
$$
So, to summarize, I would try the following:
$$
\theta_N = \mbox{read(yaw)} \\
a_N = \mbox{read(x-axis IMU)} \\
$$
$$
\theta_{\mbox{mean}} = (\theta_N + \theta_{N-1})/2 \\
R = \left[ \begin{array}{ccc}
\cos{\theta_{\mbox{mean}}} & -\sin{\theta_{\mbox{mean}}} & 0 \\
\sin{\theta_{\mbox{mean}}} & \cos{\theta_{\mbox{mean}}} & 0 \\
0 & 0 & 1 \end{array} \right] \\
$$
$$
p_N = p_{N-1} + v_{N-1} dT + \frac{1}{2} a_N dT^2 \\
v_N = v_{N-1} + a_N dT \\
$$
$$
\left[ \begin{array}{ccc}
x_N \\
y_N \\
z_N \end{array} \right] = R * \left[ \begin{array}{ccc}
p_N \\
0 \\
0 \end{array} \right] \\
$$
Now, as a final note, again, I would leave the y-axis accelerometer out of everything because you'll find one of two scenarios:
- The IMU is located on the center of rotation of the vehicle, so you don't actually ever get a y-axis acceleration during normal operation, or
- The IMU is not located on the center of the rotation of the vehicle, so you get a y-axis acceleration any time you turn. In this instance, any y-axis acceleration only indicates rotation, which you already know from the z-axis rotation.
In both cases, assuming the vehicle isn't sliding sideways, the y-axis accelerometer isn't providing you any forward/reverse information. It's only providing you either nothing (case 1) or a more obfuscated form of rotation (case 2) that you already get from your yaw angle. So, just leave it off.