mathematical calculation using speech

Question

i want my robot to do simple calculation like addition or subtraction on my voice command how to proceed ? :( the code that i am trying is as follow

#!/usr/bin/env python

import roslib; roslib.load_manifest('pi_speech_tutorial') import rospy

from time import gmtime, strftime from geometry_msgs.msg import Twist from std_msgs.msg import String from math import copysign import re

Small = { 'zero': 0, 'one': 1, 'two': 2, 'three': 3, 'four': 4, 'five': 5, 'six': 6, 'seven': 7, 'eight': 8, 'nine': 9, 'ten': 10, 'eleven': 11, 'twelve': 12, 'thirteen': 13, 'fourteen': 14, 'fifteen': 15, 'sixteen': 16, 'seventeen': 17, 'eighteen': 18, 'nineteen': 19, 'twenty': 20, 'thirty': 30, 'forty': 40, 'fifty': 50, 'sixty': 60, 'seventy': 70, 'eighty': 80, 'ninety': 90 }

Magnitude = { 'thousand': 1000, 'million': 1000000, 'billion': 1000000000, 'trillion': 1000000000000, 'quadrillion': 1000000000000000, 'quintillion': 1000000000000000000, 'sextillion': 1000000000000000000000, 'septillion': 1000000000000000000000000, 'octillion': 1000000000000000000000000000, 'nonillion': 1000000000000000000000000000000, 'decillion': 1000000000000000000000000000000000, }

class TalkBack:

def __init__(self):

def init(self, msg):

   # Exception.__init__(self, msg)
rospy.on_shutdown(self.cleanup)
      
    self.voice = rospy.get_param("~voice", "voice_don_diphone")
    self.wavepath = rospy.get_param("~wavepath", "")
    
    # Create the sound client object
    self.soundhandle = SoundClient()
    
    rospy.sleep(1)
    self.soundhandle.stopAll()
    
    # Announce that we are ready for input
    self.soundhandle.playWave(self.wavepath + "/R2D2a.wav")
    rospy.sleep(1)
    self.soundhandle.say("Ready", self.voice)
    
    rospy.loginfo("Say commands...")

    # Subscribe to the recognizer output
    rospy.Subscriber('/recognizer/output', String, self.talkback)

   
    # A mapping from keywords to commands.
    self.keywords_to_command = {'plus': ['addition', 'add'],
                                'minus': ['subtraction', 'subtract'],
                                'devide': ['devision', 'by'],
                                'multiply': ['multiplication', 'into'],
                                }
    
   # rospy.loginfo("Ready to receive voice commands")
    
                        
        
def get_command(self, data):
for (command, keywords) in self.keywords_to_command.iteritems():
        for word in keywords:
            if data.find(word) > -1:
                return command


def text2num(self, s):
a = re.split(r"[\s-]+", s)
    n = 0
    g = 0
    for w in a:
        x = Small.get(w, None)
        if x is not None:
            g += x
        elif w == "hundred":
            g *= 100
        else:
            x = Magnitude.get(w, None)
            if x is not None:
                n += g * x
                g = 0
            else:
                raise NumberException("Unknown number: "+w)
return n + g

def talkback(self, msg):        
    command = msg.data
    rospy.loginfo("Command: " + msg.data)
    
    if 'plus' in command.split():
    var1,var2= command.split("plus")
        var1 = self.text2num(var1)
    var2 = self.text2num(var2)
    var3=sum(var1,var2)
    self.soundhandle.say(var3, self.voice)
    

    else:
        return

 def cleanup(self):
    # When shutting down be sure to stop the robot!  Publish a Twist message consisting of all zeros.
rospy.loginfo("Shutting down talkback node...")     

if name=="main": rospy.init_node('talkback')

try:
    TalkBack()
rospy.spin()

except:
    pass

---

Originally posted by karan4515 on ROS Answers with karma: 73 on 2012-11-09

Post score: 0

Answer 1

Interesting question, you can reference this tutorial.

For your scenario, I think you need to do the folowings:

1.Construct the language model for keywords like "plus", "minus"...etc
2.Launch the recognizer and let the recognizer referencing your language model
3.Compute the value according to the parsed 

For each part, I can give you some hints,

1.To construct the language model, write the words you need in a text file, and upload it to this tool.

2.To launch the recognizer using language model, you can reference this command after you go through the tutorial I gave.

roslaunch pocketsphinx robocup.launch 

3.To compute the value, you need a simple calculator, you can either write one by using Flex&Bison or just simply use other's calculator.

If you want to let the robot speak out the answer, you can reference the last part of the tutorial.

---

Originally posted by Po-Jen Lai with karma: 1371 on 2012-11-10

This answer was ACCEPTED on the original site

Post score: 1

---

Original comments

Comment by karan4515 on 2012-11-11:
i have read that tutorial and my language model is also complete but i am not able to think how will robot recognize whether it is a addition or subtraction problem

Comment by Po-Jen Lai on 2012-11-11:
A simple method is to map the recognized output to keywords and then perform calculation. One possible implementation is to store the recognized output in a string, send this string to a service which can parse this string and perform calculation. The work of recognize "plus" is done in the service.

Comment by karan4515 on 2012-11-15:
send this string to a service which can parse this string and perform calculation? which services are this that you are referring to sir..

Comment by Po-Jen Lai on 2012-11-16:
I mean write one by yourself....this is not that hard.Programs which can perform this function is already out there on the web, just look for them and add service in that program.