STOP USING YOUR BATTERY IMMEDIATELY!
First of all you shouldn't let the voltage of a lipo battery drop below 3V per cell which in your case comes to 12V. Lipo Batteries are extremely dangerous and the way in which you are over-discharging it is one of the ways you definitely should NOT handle a lipo battery.
Please read the instructions given here completely before you start using your battery again -
http://www.dronethusiast.com/ultimate-drone-battery-care/
I don't want to scare you, but you should understand the various high-risk situations involved in using a lipo battery as part of a quadcopter.
Second, check for individual cell voltage of your lipo battery to make sure none of them are over-discharged.

Now, if their voltage is below 3V, be very careful as you might find links online to articles that claim they can help you 'restore' the lipo battery.
An average lipo can take upto 500 recharge cycles and if you've been consistently over-discharging them their capacity will decrease; ie, recharge faster but drain faster.
If this is your first lipo battery, I suggest you get another one and handle it with care from start, or if your current battery is not damaged and all the cells attain nominal voltage, you can continue using it,
But first get a voltage indicator that visually displays the voltage reading on an 7-segment led display.
Take care... No seriously...Take care of the battery.