We are having an issue where after toggling the create 2 from passive to active, then waiting a few seconds and toggling back to passive while on the charger, all serial communication fails a certain percentage of the time. (Links to images of current and serial communication down below illustrating it).
Detail
We are working to have the Create 2 power a NUC, auto-charge and stay alive for long (multi-month) periods of time.
Because we are taking power directly off of the battery (Need higher current), we found that we have to toggle the mode from passive
to full
and back so that it checks the voltage and goes back from trickle charge to full charging if it has been a few hours and the computer has discharged the battery. This is because the trickle charge algorithm does not check voltage as it assumes the load on the roomba is not high enough to discharge.
We started with it failing this way and have now verified that if we just plug a laptop into the create through the usb serial cable with nothing else plugged in and then toggle passive/full/wait/passive, a percent of the time we also get comm failures without any separate hardware.
Because the create 2 turns off after 5 minutes of no communication, we have tried toggling it every 4 minutes and see that reliably happening, so this is not an issue where the create 2 is just going into sleep mode.
Reproducibility
We are seeing a very repeatable issue where the serial communication (Using current generation USB serial cable from iRobot) loses serial communication with the robot some percentage of the time exactly at the point where it starts charging the robot again. After that, no packets come in from the robot for a long while. Some time later, when we have repeatedly tried to toggle from passive to full and back, communication re-appears.
We have verified this using the ROS stack (create_autonomy) and also by directly echoing characters on the command line echo -e "\x07" > /dev/ttyUSB0
to reset things.
When it is in this state, no software resets work, unplugging and replugging the serial cable does not work, etc. We have to take it off the charger physically (or wait for a long time until our system toggles it again and it does work.
We have verified that this occurs (When the create 2 is on its charger) on:
- Mac only plugged in over the create cable with no power taken from the system.
- NUC taking power directly off the battery through a dc/dc converter.
- NUC taking power off an intermediate battery charging from the create.
Mitigating it
- We have added ferrite beads on all the cables to mitigate any high frequency noise and that does not seem to solve it either.
- We have also tried 3 different create cables w/ different serial/usb chips and even built our own from scratch and not fixed it.
- We have tried separately powered usb hubs, unpowered ones and plugging directly in to the computer.
- We have tried it on 3 separate create 2's.
I have attached links below with graphs of the output data when it fails. When there is a straight line on current/voltage, that is repeated data, showing that it was stuck and not updating. It always happens right when the system steps up the current to start charging again, so it seems to be linked to a current spike - but only breaks it 1 / 5 or 10 times - it just happens that once broken, it stays that way for a very long time.
Communication fails after multiple successful toggles
Detail showing exactly when communication fails
After many minutes, it is working again
When it has been broken, I have tried turning off our software and addressing the system by echoing data to it directly for OI to turn off, on, reset, etc and do not see communication resume correctly.
If we unplug the roomba from the charger by hand (pull it off), then many times it does work, but not always. If we then reset everything by holding down the buttons, it does seem to work 95% of the time and sometimes we have even had to pull the battery itself to make it work.
If someone has encountered this before, advice would be helpful. We had thought that it was ground loop issues, but given it is on so many different cables, roombas, power situations and compute devices, we are trying to figure out what is fundementally wrong with our setup.
Thanks!