Let me give you a mathematician's perspective on the difference between the two kinds problems.
Forward kinematics asks the question: given a certain input (i.e. control command), what will be the output (i.e. robot configuration, pose, etc.). Inverse kinematics asks the reverse question: given a certain desired output, what is the necessary input.
Inverse problems are usually much harder to solve (e.g. ill-conditioned or otherwise ill-posed) than forward problems1. This is true not only of kinematics, but also dynamics and a wide range of computational problems as well.
Consider the following analogy. Let's assume we are given a basic algebraic function $f(x)=x^3-5x+1$. Forward kinematics is like asking to find $f(2)$. It's easy to do because the equation is already given and all we need to do is substitute the input to get the output. Kinematic equations are also this way: designed to convert inputs into outputs easily. Inverse kinematics is like asking to solve the equation $f(x)=2$. It's not as easy to find the answer. Going backwards to find the input that gives a specific output is non-trivial. An answer may not even exist. If it does exist, it may not be unique. You may require additional information (e.g. regularization) to obtain a unique solution with desirable characteristics.
Often, it's impossible to solve inverse kinematics analytically. This is especially for true for serial arm linkages. This means that you will need a numerical method to solve a system of non-linear equations of the form $F(X)=0$. Borrowing from calculus, we can use Newton's method. In higher dimensional spaces, Newton's method requires formulating the Jacobian (or at least the approximate Jacobian) of $F(X)$.
1. For parallel arms (such as a Delta arm or Stewart platform), the difficulty is reversed (i.e. inverse kinematics is straightforward, whereas forward kinematics is more tricky). In general, one of the two problems (either forward or inverse) will always be harder to solve than the other.