I am currently working on a prototype device to automate cloth ironing task. So basically the mechanical design involves 4 manipulators and a separate manipulator arm with a hot pad (iron) to manipulate a given cloth to perform the ironing task. I understand its a very complicated task and I have come up with the following steps to potentially go about solving this problem. I would appreciate any kind input on the plausibility of this idea (mechanical and software challenges).
For my first EB (Engineering build), I'm solely focusing on solving this problem for a T-shirt, probably extend this solution to other cloth types with modifications.
From a pile of clothes, choose a certain cloth for ironing using background subtraction and colour based image segmentation (already programmed this part and its working!)
Classify or recognise the chosen cloth into one of any known classes using CNN (Convoluted neural networks). I personally think this part is quite achievable given that I have enough training data to train my network.
This is where the problem begins!. After classifying the given cloth, find coordinates of points of interest to manipulate the cloth in a certain 'predetermined' mechanical movement to facilitate the manipulation of cloth. What I mean by points of interest here is, for example, in the case of a T-shirt (see image below); the algorithm should localise the locations the following parts 1) Midpoint of T-shirt hem (see image for details) 2) Corner point of T-shirt hem (see image for details) 2) Start of yoke 3) End of the yoke
I believe (through my mental visualisation) that these are the key points to place the T-shirt flat on the surface (a known STATE). When the T-shirt is flat on the surface, it could be stretched just enough to make the cloth wrinkle free to facilitate ironing through some kind of force sensing mechanism (Haven't really though about it, but I presume its not huge challenge).
- When the cloth is in a known STATE, proceed to ironing using path planning algorithms.
I do understand that this is a very complex problem, just want to get some inputs to see if I am on the right path. I am really not sure if STEP 4 is mechanically achievable since non-rigid body manipulation is a huge problem for robots. I am quite confident on the software aspect of the project, I believe a large enough data set and well tested algorithm will help me identify the afore-mentioned points of interest in a test cloth. But I would like hear the mechanical challenges and plausibility of this project from design engineers standpoint. Hope I'm clear about my query.
Any input is appreciated. Thank you so much and cheers :)