Assume we have camera poses data and IMU data(which include gyr and acc data) with different timestamps, now I want to estimate this value, I known that this can be done in many ways(EKF,MSCKF), but I wanna to do this use factor graph model, how can we model this problem as an factor graph model and do optimization?
There is no off the shelf implementation you can use for computing the temporal offset in GTSAM. You will have to implement the algorithms yourself. For the specifics of the algorithm I know 3 different ways.
One is you can implement the same equations as implemented in the EKF/MSCKF framework. Factor graphs and Extended Kalman Filters actually solve the problem in the exact same way(just factor graphs are iterative and are in the Information form). Only problem with this approach is that it is inefficient due to the needing to relinearize multiple times.
For factor graphs to be more efficient with IMU processing you typically use an algorithm called IMU preintegration. This can be modified to also estimate the temporal offset as seen in this paper.
Final option is to implement the algorithm from VINS Mono. It works a little bit different then the above mentioned approaches in that it estimates a velocity for each landmark. Again you would have to implement this algorithm yourself, but it has the advantage that the code is open source and so it can be easier to code up.