In the covariance matrix of the EKF SLAM, there are components that specifies the correlation between landmarks. Are those values important? Because while we bulding map of landmarks, we only draw the ellipse around each landmark (which correspond to the covariance of that landmark). I'm quite new to SLAM so any correction and help would be great. Thank you.
Drawing the error ellipses is more a tool for you the human to see how the system is working. It is a proportional representation of the error, not an exact value. So it is OK to ignore the cross correlation.
The cross covariances are important for the EKF to work correctly. That is why we still keep them in the Filter and propogate/update them accordingly. But you don't have to care about them when drawing the error ellipse.