I'm afraid not, actually. Even if you have a perfect friction compensation and a perfect gravity compensation, you still need to deal with inertia. It will be extremely hard to swing it in arbitrary direction due to large inertia (and Coriolis and centrifugal forces, if your robot arm is multi-dof system).
Using equations, a generic manipulator equations of motion is given as:
$$M(q)q"+C(q,q')q'+g(q)+tau_{fric}=u_motor + u_{user}$$
Suppose perfect gravity/friction compensator, i.e., $u_{motor} = g(q) + tau_{fric}$. You are left with
$$M(q)q"+C(q,q')q'=u_{user}$$
To generate arbitrary motion, i.e., arbitrary $q(t)$, you need sufficient amount of effort to overcome inertia $M(q)$ and Coriolis/centrifugal $C(q,q')$, both of which are approximately proportional to mass.