NOTE: the answers below are wrong. That is, IMU measurements give you $$\frac{{}^O d^2}{dt^2}\left({}^O p^B_B\right)$$ and not $$\frac{{}^B d^2}{dt^2}\left({}^O p^B_B\right)$$
I will answer each of my own questions individually.
Question 1
My understanding is that the linear acceleration measurements coming from IMU B correspond to measurements of the second time-derivative of ${}^O p^B_B$, which is the position vector from the origin of frame O to the origin of IMU B's frame expressed in frame IMU B's frame. In other words, it is
$$
\frac{{}^B d^2}{dt^2}\left({}^O p^B_B\right)
$$
where $\frac{{}^B d}{dt}(\cdot)$ is the time derivative operator in frame B.
Is my understanding correct?
This is not correct. Instead, the measurements coming from IMU B are measurements of $$\frac{{}^O d^2}{dt^2}\left({}^O p^B_B\right)$$ which is the second time-derivative, with respect to frame $O$, of ${}^O p^B_B$. That is, time-derivatives of ${}^O p^B_B$ are computed while assuming that the only fixed frame is frame $O$ (i.e. the basis vectors associated with frame $O$ have a velocity of $0$ with respect to frame $O$).
Moreover, if we let $O_x(t)$ and $O_y(t)$ be the time-varying basis vectors associated with frame $O$, such that ${}^O p^B_B = a(t) \cdot O_x(t) + b(t) \cdot O_y(t)$ for some scalar functions $a(t)$ and $b(t)$, then
$$
\begin{align}
\frac{{}^O d}{dt}\left({}^O p^B_B\right) &= \frac{{}^O d}{dt}\left(a(t) \cdot O_x(t) + b(t) \cdot O_y(t)\right) \\
&= \frac{{}^O d}{dt}\left(a(t) \cdot O_x(t)\right) + \frac{{}^O d}{dt}\left(b(t) \cdot O_y(t)\right) \\
&\stackrel{(1)}{=} \left(\frac{da}{dt}(t) \cdot O_x(t) + a(t) \cdot \frac{{}^O dO_x}{dt}(t)\right) + \left(\frac{db}{dt}(t) \cdot O_y(t) + b(t) \cdot \frac{{}^O dO_y}{dt}(t)\right) \\
&\stackrel{(2)}{=} \frac{da}{dt}(t) \cdot O_x(t) + \frac{db}{dt}(t) \cdot O_y(t)
\end{align}
$$
where $(1)$ follows because of the chain-rule and $(2)$ follows because $\frac{{}^O dO_x}{dt}(t) = 0$ and $\frac{{}^O dO_y}{dt}(t) = 0$.
For more information on why this is the case, see sections 2.1, 2.2, and 2.3 in the paper
Manon Kok, Jeroen D. Hol and Thomas B. Schön (2017), "Using Inertial Sensors for Position and Orientation Estimation", Foundations and Trends® in Signal Processing: Vol. 11: No. 1-2, pp 1-153. http://dx.doi.org/10.1561/2000000094
which is also available for free here. More specifically, in section 2.3, the author writes
The accelerometer measures the specific force $f$ in the body frame $b$ [147]. This can be expressed as $$f^b = R^{bn}(a_{ii}^n − g^n), \tag{2.3}$$
The term $a_{ii}^n$ is similar to $\frac{{}^O d^2}{dt^2}\left({}^O p^B_B\right)$.
Question 2
Similarly, my understanding is that the angular velocity measurements obtained from IMU B correspond to measurements of the time-derivative of ${}^O\theta^B_B$, which is the angular displacement vector of frame B relative to frame O expressed in frame B. In other words, the angular velocity measurements correspond to
$$
\frac{{}^B d}{dt}\left({}^O\theta^B_B\right)
$$
Is my understanding correct?
The term "angular displacement vector of frame B relative to frame O expressed in frame B" turned out to be misleading and imprecise. To see why, let us expand Fig. 1 into Fig. 2 below.
Fig. 2
We can see from Fig. 2 that the "angular displacement of IMU B's frame expressed in the fixed frame of reference O" is ${}^O \theta^B$, which is positive counter-clockwise.
We can see that, because angular displacement is measured with respect to lines in 2D and planes in 3D, as opposed to points for linear displacement in 2D and 3D, then there is no need to specify a "relative to" frame and an "expressed in" frame at the same time for angular displacements. Instead, only the source frame (frame O in this case) and target frame (IMU B's frame in this case) need to be specified.
Similarly, the angular displacement of frame O in frame of IMU B is ${}^B \theta^O = -{}^O \theta^B$. Additionally, the angular velocity of frame B in frame O is
$$
\frac{{}^O d}{dt}\left({}^O \theta^B\right) = {}^O \omega^B
$$
Question 3
My goal is as follows: given measurements from IMU B of the vectors
$$
\frac{{}^B d^2}{dt^2}\left({}^O p^B_B\right)
$$
and $\frac{{}^O d}{dt}\left({}^O\theta^B\right)$, compute estimates of the vectors
$$
\frac{{}^A d^2}{dt^2}\left({}^O p^A_A\right)
$$
and $\frac{{}^O d}{dt}\left({}^O\theta^A\right)$ without using measurements from IMU A.
How can I go about doing this?
First, note that (see here for details on spatial algebra rules)
$$
\begin{align}
{}^O p^A_A &= {}^O p^B_A + {}^B p^A_A \\
&= {}^A R^B \cdot {}^O p^B_B + {}^B p^A_A
\end{align}
$$
where ${}^A R^B$ is the orientation of IMU B's frame relative to IMU A's frame (which is realized as a rotation matrix). So,
$$
\frac{{}^A d^2}{dt^2}\left({}^O p^A_A\right) = \frac{{}^A d^2}{dt^2}\left({}^A R^B \cdot {}^O p^B_B + {}^B p^A_A\right) \tag{1}
$$
Because ${}^A R^B$ is time-invariant, then
$$
\begin{align}
\frac{{}^A d^2}{dt^2}\left({}^O p^A_A\right) &= \frac{{}^A d^2}{dt^2}\left({}^A R^B \cdot {}^O p^B_B + {}^B p^A_A\right) \\
&= {}^A R^B \cdot \left[\frac{{}^A d^2}{dt^2}\left({}^O p^B_B\right) + \frac{{}^A d^2}{dt^2}\left({}^B p^A_A\right)\right]
\end{align}
$$
Additionally, because ${}^B p^A_A$ is time-invariant, then $\frac{{}^A d^2}{dt^2}\left({}^B p^A_A\right) = 0$, and so
$$
\frac{{}^A d^2}{dt^2}\left({}^O p^A_A\right) = {}^A R^B \cdot \frac{{}^A d^2}{dt^2}\left({}^O p^B_B\right)
$$
Finally, because IMU A and IMU B are connected via a rigid body, such that ${}^A R^B$ is time-invariant, then
$$
\begin{align}
\frac{{}^A d^2}{dt^2}\left({}^O p^A_A\right) &= {}^A R^B \cdot \frac{{}^A d^2}{dt^2}\left({}^O p^B_B\right) \\
&= {}^A R^B \cdot \frac{{}^B d^2}{dt^2}\left({}^O p^B_B\right) \tag{2}
\end{align}
$$
Equation (2) is what we are looking for, since we have measurements for $\frac{{}^B d^2}{dt^2}\left({}^O p^B_B\right)$.
Note that the above only holds if both ${}^A R^B$ and ${}^B p^A_A$ are time-invariant (that is, the two IMUs are moving together through a rigid transformation and the frames themselves do not change their orientations relative to each other). Otherwise, you would need to take the Coriolis effect into account (since IMU A's frame would be moving relative to IMU B's frame. See this answer, this page, and this page for details). For example, if IMU A is fixed in-place and IMU B is rotated around it, then ${}^B p^A_A$ is no longer time-invariant.
