I'm looking for an equation (or set of equations) that would allow me to predict (with fair accuracy) how heavy a payload a quadcopter is capable of lifting.
I assume the main variables would be the weight of the copter as well as the size + power of the 4 rotors. What general approach can one use to make such a determination?
Similar answer to this post: Choosing motor for a tricopter
This depends on what you want to do with your copter.
Hovering: 100-150 Watt/kg
ScaleFlight: 200-300 Watt/kg
Some chilling acrobatic flight: 350-400 Watt/kg
3D acrobatics 500+ Watt/kg
Hardcore 3D 1000 Watt/kg
Just take your motors and your flight style and you got some values for the max weight of your copter
I also have a source for this but i know only a German one: http://www.heli-planet.de/index.php?section=auslegung_von_e_motoren
This values are pretty accurate. There are mathmatical ways to calculate the lift of your quadrotor but they are a very theoretical approach. You have to think about the enviorment, the wind, air pressure, reaction speed, controlability.
I would go further than TobiasK, even if the order of magnitude is correct.
The main characteristic for a drone dynamic is its Thrust/Weight Ratio (TWR). To be able to fly, your TWR shall at least equal 1. To be able to take off, you need TWR>1. In my opinion, TWR=2 gives a fairly dynamic quadcopter.
For a given propeller, you can get its thrust vs rpm (round per minute) ratio, so you can get the target rpm (if you have 4 motors and want TWR=2, each motor should achieve a thrust equal to half the drone weight). You can also get its torque vs rpm characteristic, and get the mechanic torque you'll get at the target RPM.
Then you can choose your motor. The target RPM gives you the Kv of your motor. The Kv indicates the amplitude of the back EMF of your motor, according to its rotation speed. If you use a 3S LiPo battery, your battery may drop done to 9.6V, so, if you want to keep your dynamic during the whole flight, you shall choose a motor that may reach the rpm target at 9.6V. Let's say you need 10k rpm to get your TWR=2, your motor's Kv shall be at least 1040 rpm/V. You should choose your Kv a bit higher, as Kv is given with no load, and the actual speed under load is a bit slower.
The output torque of your motor is proportional to the current you feed in. I looked for "quadcopter motor" on google, and quickly found a motor with 1300Kv and 19A max (I don't know this motor and can't say if it's good quality or not). 10k rpm divided by 1300 rpm/V gives 7.7V. 7.7V will be the average voltage at the output of your ESC when driving the motor at 10k rpm. The max current is 19A, this means the maximum power at 10k rpm is 7.7*19=146W. This kind of motor has a typical efficiency around 0.7 when on load (if you take a motor with a much higher Kv, you usually decrease its efficiency on load). Therefore, I expect the output mechanical power to be 100W. Mechanical power is speed times torque (checks the units you uses). The max torque has to be higher than your propeller torque at TWR=2.
"Hardcore 3D" was mentioned. In my opinion, doing intensive 3D flight with a 5kg drone is insane. I think you should target a total weight under 1kg (400g is good for 3D acrobatics). 3D flight faces an other drawback: propellers are usually optimized to run in one way, and you usually want to use it in both, so you'll either have a much poorer TWR in reverse, or have a poorer efficiency in both way (symmetrical and better than a classical propeller in reverse, nonetheless).
Short answer is : it depends. Variables such a motors, propellers and body construction are very important. I have been building drones in my garage as a hobby for years. You'd be surprised what difference can propellers make.
Since building heavy lift drone can be extremely hard, most of people should be satisfied by buying some heavy lift drone. There are several models available on amazon so it's probably good idea to check what market has to offer.
