I am modifying an implementation of SLAM with single camera, MonoSLAM . Instead of image patches, I want to use features points (ORB) to track landmarks. MonoSLAM uses a EKF framework. So for every feature tracked, I have a covariance matrix which defines the uncertainty of the measurement in current camera frame.
Had this been an image patch, as in Davidson et al , one has to center a sliding window with every pixel inside the ellipse defined by the covariance. Then check for correlation of pixel intensities between the tracked patch and the patch from the new frame. Some thing like this figure below.
For features, should I search in a similar fashion by calculating the descriptor for each pixel in the ellipse?
Since features come with their own detectors, should I leave it to the detector to find the list of features within the ellipse and then match them with my previous frame feature?
: Davison, Andrew J., et al. "MonoSLAM: Real-time single camera SLAM." IEEE Transactions on Pattern Analysis & Machine Intelligence 6 (2007): 1052-1067.