Adding voice recognition
Invent Logo
AI

You've probably heard about AI, or Artificial Intelligence. It's the technology that helps Alexa and Siri answer your questions, suggests videos on YouTube, and even lets self-driving cars know where to go. But why should you learn about it?

AI is becoming a big part of our world. It's used in almost everything - from video games to helping doctors find out what's wrong with people. By understanding AI, you can help shape the future! Imagine creating a robot that can help with homework or an app that suggests fun activities based on what you like. Learning about AI lets you create amazing things that can make life easier and more fun.

Plus, AI isn't just about programming. It's about learning how to think creatively, solve problems, and understand how things work. These skills are super useful, no matter what occupation you go into. And who knows? Maybe one day you'll create the next big AI invention that everyone will use!

AI
Voice recognition

Have you ever used Siri, Alexa, or Google Assistant and wondered how they understand your voice? That's thanks to voice recognition technology, which uses AI to figure out what you're saying.

Here's how it works: When you speak, your voice creates sound waves. The AI system captures these sound waves and converts them into digital data - basically a bunch of numbers that represent different sounds. Then, it uses something called a neural network, which is like a super-smart system that's been trained to recognize patterns in speech.

This neural network breaks your speech down into tiny pieces, like the sounds of each letter and word, and tries to match them to words it knows. The cool part is that it's been trained on millions of voice samples, so it can understand you even if you have a unique way of speaking or an accent.

After identifying the words, the AI uses another process to figure out what you mean. For example, if you ask, "What's the weather like today?" it knows you're looking for a weather update and not just saying random words.

So, voice recognition is like having a personal assistant that's constantly learning and improving, making technology more interactive and convenient!

Voice recognition
How we'll do it

We're going to get the robot to listen to our voice and carry out instructions. The robot doesn't have a microphone so we need to use the microphone on our computer. The microphone will listen to our voice, work out what we're saying and send instructions to the robot.

This is an advanced Python tutorial and requires programs running in Python on both your robot and a desktop (or laptop) computer. We'll assume you know how to use Python already on your robot.

Most laptops have in-built microphones. If you're using a desktop computer you might need to plug in a separate microphone or use a headset or set of earphones that has an inbuilt microphone.

How we'll do it
Using cloud computing

For our first experience of AI, we're going to use Google's voice recognition services that run on it's 'Cloud'. This makes the task of voice recognition much easier because it takes a lot of computing power.

Cloud computing is like having a super powerful computer on the internet. Instead of storing data or running programs on your own device, you use remote servers (the "cloud") to do the heavy lifting. This lets you access files, apps, and services from anywhere with an internet connection. Think of it like streaming a movie: you don't need to download it because it's stored on Netflix's servers. Similarly, cloud computing gives you access to massive storage and computing power without needing expensive hardware. It's flexible, scalable, and makes sharing and collaboration super easy!

Using cloud computing
Required software

If you're using a computer of your own you'll need to install the 'Python' programming language, some 'libraries' that allow Python to recognise your voice, and ideally a code editor to make editing and running your code easier.

If you're using a school computer that already has these things installed you can skip to the step 'The Computer Code' below.

If you're using your own computer, follow the next steps to install 'Thonny' which is a code editor that already comes with Python.

Installing Thonny

Go to thonny.org and download and install Thonny for your system.

Installing Libraries in Thonny

Go to the 'Tools' menu and select 'Manage Packages'.

Now enter 'setuptools' into the search and click on the first option that's found labelled 'setuptools'. On the next page there should be an 'Install' button to install this package. (If there isn't and it looks like it's already installed you can skip to the next package).

Repeat this process for the library packages called 'SpeechRecognition' and 'PyAudio'.

You've now installed Thonny (with Python), and the three libraries you need, so you can proceed to the step 'The Computer Code' below.

Installing Libraries in Thonny
Notes to IT support staff

Note to IT support staff :

If your school already uses Python and an alternate code editor such as Visual Studio, you may need to install the three libraries via the command line instead using the 'PIP3' command :

  • pip3 install setuptools
  • pip3 install SpeechRecognition
  • pip3 install PyAudio
The Computer Code

Now let's write the code for your computer to recognise your voice.

Copy and paste this code into Thonny (or whatever code editor you're using).

import socket
import speech_recognition as sr

# Place your robots IP address here!
server_ip = '192.168.178.61'
port = 2309

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((server_ip, port))
print("connected to robot")

recognizer = sr.Recognizer()
microphone = sr.Microphone()

while True:
  with microphone as source:
    recognizer.adjust_for_ambient_noise(source)
    audio = recognizer.listen(source)
  try:
    sentence = recognizer.recognize_google(audio)
    print("You said: {}".format(sentence))
    s.sendall(sentence.encode())
  except:
    print("Error recognising")
Add the Robot's IP Address

Our programme needs to know the IP address of our robot so it can send the instructions to it.

First get the robot to read out it's IP address. Do this by holding in your robot's button whilst you switch it on and letting go when it says "wifi". When it connects to the wifi and the light turns green, it should read out the IP address.

Find the code right at the end that's shown below and put your IP address inside the quotes. Our IP address was '192.168.178.61' so we've entered that here.

# Place your robots IP address here!
server_ip = '192.168.178.61'
port = 2309

Now save your complete code as 'voicesend.py'. Don't run it yet - it won't work until we've run code to listen for our instructions on the robot.

Robot code

Now switch to the inventblocks website to program the robot. You'll need to connect to the robot like you normally do and then create a new file in your text editor, copy and paste the code below into it, save it to your computer with a name like "voicereceive.py" and then upload it to your robot.

import socket

while True:
  print("starting socket")
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
  s.bind(('', 2309))
  s.listen(5)
  print("Listening for connections...")
  conn, addr = s.accept()
  print('Connected with', addr)
  while True:
    try:
      data = conn.recv(1024)
      print('Received:', data)
      data = data.decode()
      if data=="move forward":
        move(15,True)
    except:
      s.close()
      break
  time.sleep(0.1)
Running your robot code

Once you've uploaded your code, you'll need to run it on the robot.

Running your robot code
Is your code running?

To check your code is running, go to the 'Console' tab. You should see text that says:

"starting socket"

and

"Listening for connections..."

This means that code on your robot is awaiting commands ...

Is your code running?
Running your computer code

Now run the code that transmits your voice commands using the 'play' button on your computer.

Running your computer code
Is your code running?

If your console looks like our and says 'ready', you're ready to try talking to your robot!

Try saying 'move forward' and your robot should follow your command!

N.B. If instead of 'ready' you get some red writing and it says 'ConnectionRefusedError', your computer couldn't reach your robot over the network. Try restarting your robot and running the code again on your robot, and then finally running the code again on your computer. The code on your robot must be running before you start the code on your computer!

Is your code running?
How the computer code works - 1

Let's look at the start of the code running on our computer. We first import the libraries we need - one for the network communications ('socket') and one for speech recognition :

import socket
import speech_recognition as sr

We then define the IP address of our robot .to communicate with and the port number. The port number we've picked here is pretty much an arbitrary number, albeit one we know isn't used by other standard things running on a typical network:

# Place your robots IP address here!
server_ip = '192.168.178.61'
port = 2309
How the computer code works - 2

Now we're going to setup the 'socket' (the way we communicate with another machine) and connect that socket using the IP address of the robot:

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((server_ip, port))
print("connected to robot")

Finally, we setup the speech recognition system - we need to create a speech recognition 'recognizer' object and one for the microphone as well:

recognizer = sr.Recognizer()
microphone = sr.Microphone()
How the computer code works - 3

Now we're all set up, we can just loop around getting an audio sample from the microphone and trying to recognise the text. The 'adjust_for_ambient_noise' function is used to minimise the effect of background noises when we're trying to look for speech.

The hard work is done in the 'recognizer.recognize_google(audio)' function using Google's API to recognise the speech. Then we just print it out and send it over the network to our robot!

while True:
  with microphone as source:
    recognizer.adjust_for_ambient_noise(source)
    audio = recognizer.listen(source)
  try:
    sentence = recognizer.recognize_google(audio)
    print("You said: {}".format(sentence))
    s.sendall(sentence.encode())
  except:
    print("Error recognising")
How the robot code works - 1

Now let's look at the code on the robot.

  • We start off with importing the socket library for network communications, and then open this socket.
  • The 's.setsockopt' function tells the network system that we can reuse the port number (2309) if we stop and start the programme. If we didn't do this, we'd get an error the second time we ran the programme.
  • Then the 'bind' function makes sure we're running on the right port number we set on the computer (2309).
  • The 's.listen' function starts listening for communications.
  • Finally, the 's.accept' function waits until the code running on the computer has connected to us
import socket

while True:
  print("starting socket")
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
  s.bind(('', 2309))
  s.listen(5)
  print("Listening for connections...")
  conn, addr = s.accept()
  print('Connected with', addr)
How the robot code works - 2

Now the hard work is done, we can just loop around checking for any received messages and decoding them.

When we receive a message, we print it out and then act on the command.

The 'try: except:' block catches any errors. The main reason we use this, is to close down the network connection so that it can be restarted later if the code running on the computer is stopped. This is also why we have two 'while True:' loops inside of each other. The first one is just waiting for a connection, and the second is waiting for a message to be received.

  while True:
    try:
      data = conn.recv(1024)
      print('Received:', data)
      data = data.decode()
      if data=="move forward":
        move(15,True)
    except:
      s.close()
      break
  time.sleep(0.1)
Add to the code

At the moment the only command the robot will respond to is to "move forward".

Try adding some more commands to the if statement at the bottom of the file.

Sample Robot Code

Here's my code with some more commands added.

Feel free to add some of my commands to your code as well! Note that we're using the obstacle sensor module here that has the LED circle on it so we can have a 'smile' command!

import socket

while True:
  print("starting socket")
  s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
  s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR,1)
  s.bind(('', 2309))
  s.listen(5)
  print("Listening for connections...")
  conn, addr = s.accept()
  print('Connected with', addr)
  while True:
    try:
      data = conn.recv(1024)
      print('Received:', data)
      data = data.decode()
      if data=="forward" or data=="move forward":
        move(15,True)
      elif data =="reverse" or data=="move backwards" or data=="move back":
        move(-15,True)
      elif data =="turn left":
        turn(-90,True)
      elif data =="turn right":
        turn(90,True)
      elif data =="smile":
        facenp[0]=(10,100,30)
        facenp[1]=(10,100,30)
        facenp[2]=(10,100,30)
        facenp[3]=(10,100,30)
        facenp[4]=(10,100,30)
        facenp.write()
    except:
      s.close()
      break
  time.sleep(0.1)