The simplest method line following robots use is 3 sensor system:
Consider the 2 block sensors, let's assume they sense color. The middle one senses white, the remaining two senses grey. This robot follows the white line in the image.
X
XXXXX
XXXXX
XX XXXXX
XXXXXX XX
XXX XX
XXX X
X X
X XX
X XX
X X
X X
X X
^ X X
| +---+ X +---+ X +---+
| | | X | | X | |
| +---+ X +---+ X +---+
| X X
| X X
+ X X
X X
As the robot advances, towards the turn, the left sensor evetually will sens white color. This is the trigger which makes the robot turn to the left. If the middle sensor senses grey intead of white, the robot knows that it has to turn at a sharper angle. As the left one senses grey again, and the middle one does not sens white, the robot knows that it is off track and has to turn even more sharply to the left. So there are 3 stages of triggers for turning even sharper curves to the left. If you consider the sensors not binary, but being able to sense the "closenes" of the white stripe (as it starts to reflect more light if the sensors on the side approach it) you can derive a signal which is more contous and you can create a PID line following controller based on "how much white does the side sensor sense).
Special logic is required to get back to the track once the robot left the stripe completely. In this case the sensor which last sensed white gives the direction of the line search behaviour. Once the line is found, the line following controller can be re-activated.
This is similar to wall fallowing. In that case distance to a wall gives a similar signal to which a wall following controller (e.g. a PID controller) can be tied to.