For an arm with series elastic actuators I'm not aware of an open source design. There are many mechanisms you can find in research papers if you search for them.
For force control you have a few other options.
You can use an efficient backdrivable transmission (direct drive or spur or planetary or capstan all work) and control the current in a permanent magnet motor which will have a linear relationship between current and torque/force.
If you only interact with the world (payload) at the end-effector, you can put a force-torque sensor there.
For safety you can use clutches instead of compliance in many cases.
In all cases you will need to invest some time and learning. Maybe buddy up with a mechanical engineer.
As a side note, there are cheaper ways to measure the deflection (the original used a potentiometer) and finding the correct geometry and size of spring to match the operating conditions is a challenge in itself. These guys used compression spring for a rotary actuator and you could probably find enough public information plus some simple equations to use that approach in your own design.
A final comment is that since you tagged visual servoing in your other question and your are talking about machine learning, you might consider keeping things more simple. Don't worry about force control, add whatever compliance you need for safety in any form (a rubber section in the link perhaps) and let the learning algorithm handle the control details...