I need to get my drone flying still enough that I can rest a glass of water on it.

I've tried a few KK boards and APM 2.6 (3.1 software). I've balanced props, set PID settings, auto-trim / autotune and the drone still tends to inconsistently drift a little one way or another.

What is a plausible way to completely isolate drift?

Using existing resources:

Very similar answer to @SteveO - I'd focus on tuning the PID control loop for each motor individually, if possible, to eliminate any instability in the power delivery which would result in the aircraft leaning and therefore drifting.

A great video tutorial on PID control that helped me through a control exam

For the control responsiveness, there are also many items that may help. Of course, faster control loop rates, and highly accurate sensors help. What is the loop rate of your controller? Can you program it more efficiently? Or use a faster micro? And, is the most current sensor data getting to the controller quickly? I suspect a loop rate of around 1 kHz, and sensors with that type of bandwidth, would be required to achieve the type of stability you desire. I would also look at the responsiveness and accuracy of your altitude and attitude sensors. If you are using an IMU, could you add other sensors to get more accurate and/or faster data? What about a downward-pointing (possibly laser) rangefinder to get very peppy altitude measurements? Does your power supply and controller have enough slew rate to accommodate the control output changes needed for stability? Finally, what model are you using in your controller? A basic PID controller should be pretty good if the above items are okay, but if you still have nonsteady motion you might need to look into feedforward control, or maybe H$\infty$ control to get the stability you need.