Skip to content

Home

Jump to bottom
Anirban Banik edited this page Aug 13, 2018 · 12 revisions

Introduction to the project

This is basically a project to rebrain the original Speak and Spell by Texas Instruments, first introduced in 1978. The main improvement over its predecessor is the addition of Speech to Text functionalities in addition to Text to Speech. Not only this but I have tried to increase the complexity of the games further, as well has worked on building game GUIs with pygame modules.

In the Project

The project mainly consists of four Games:

  • Spell It! -> It asks the user to spell out a word correctly within a limited number of trials, and accordingly generates his score out of 10.
  • Hangman -> This asks the user to guess a word correctly when it is given in the form of dashes, like beagle is given as ----. The scores are generated accordingly.
  • Encrypter -> This game consists of three parts, Encode, Decode, Guess. By selecting options between 1, 2, and 3 the user selects to play any one of the games. In Encode and Decode, an example is given how the encoding or decoding is carried out. The user guesses the shifting key and then accordingly encodes or decodes a word given. In the Guess game, the user has only to enter the correct shifting key as the answer.
  • Crossword -> This is basically an extension of Hangman, where the user has to guess four words, given in rows and columns of a 4x4 matrix. The meanings are accordingly given for all of them so the user finds it easier to get to the result. The games are both in terminal and GUI modes as of now.

It also consists of a main Game launcher script which voice-launches all the game GUIs.

Quick-Start Guide:

  • Setup the hardware as described in the HW wiki page. Deploying the project on your PC will be easier, but to do so on the board follow the intricate details of the setup description carefully.
  • To voice-launch the games, go to ./Game/Game_launcher, start run_launcher.py and enter voice commands START or GENERATE (Game-Name) to launch the game-GUIs. For example to launch Hangman GUI, enter command START HANGMAN. For more information regarding this go to the description of the Game-launcher.
  • You can also manually launch the games, just go to the specific directories of the games, and follow the description given for further details. One example given: For manually launching Encrypter, migrate to ./Game/Games/Encrypter and run:
  1. For the terminal mode:
python3 Encrypter.py
  1. For the GUI mode:
python3 Encrypter_gui.py

Project structure description

The four games in the project can be run both manually or automatically run by calling them via the Game_launcher script. All the games are Voice-I/O based games, and require no input from the keyboard. The game codes have been structured so that the game GUI scripts uses core functionalities from the main game script while the terminal version of the game can be run from the main scripts themselves, subject to making an instance of the game class.

GSoC work done:

This summarises the progress made during GSoC 2018.

Hardware

The hardware used for this project are:

  1. A PocketBeagle board.
  2. A USB Audio device.
  3. A microphone with Automatic Gain Control.
  4. A USB breakout board.
  5. A Breadboarding kit consisting of wires and connectors to power everything.

Aim

The final aim of this project to be fulfilled during GSOC are:

  1. Create a Voice User-Interface for the project, and an API to simplify the application of Voice Input in the games, giving various options to the future developers to create games efficiently, and update the API without making any changes to the game code.
  2. Make all the games as present in the original Speak and Spell device with appropriate GUIs.
  3. Develop game-launching functionality to launch the games using voice input.
  4. Deploy the entire project on the PocketBeagle.

Progress

The progress during the ongoing GSOC period is summarised below:

  1. Succeeded in making the VUI and interacting with the games efficiently. I initially thought of using live speech decoding interface for this purpose, but felt that to efficiently record it continuously and operating on the recorded files would be quite productive.
  2. Almost finished working on the games, some UI refinements are still needed, and worked on voice-launching the games.
  3. There were some issues with the hardware, it couldn't be fixed earlier, solved it after some soldering. As far as of now the audio works on the pocketbeagle, subject to some changes in the configuration files of ALSA, as described in the Hardware setup wiki page.

To Do

  • Support the voice launching of the terminal modes of the games as well.
  • Test the working of the game GUI on the pocketbeagle.
  • Refine the terminal modes and suppress any unwanted logs.
  • Improve the API structure further to help in building better VUI architectures.
  • Auto-starting the game-launcher script on boot-up.

Future goals

  1. The main aim of this project is to provide a base for future work on building smart speakers/assistants with pocketbeagle.
  2. Also to build more complex VUI architechtures in the future.

Useful links

  1. Project Proposal Draft
  2. Code
  3. Documentation
Clone this wiki locally