新しい Twinmotion プロダクトが無料で利用可能になりました

Speech To Life

Offline Speech Recognition for Windows and Android platforms including Quest 1 / 2

  • サポートされたプラットフォーム
  • サポートされたエンジンバージョン
    4.26 - 4.27, 5.0
  • ダウンロードのタイプ
    エンジン プラグイン
    この製品には、コード プラグインが含まれており、ビルド済みのバイナリと Unreal Engine に統合される全ソースコードが完備されています。任意のエンジン バージョンにインストールし、プロジェクト毎に有効化することが可能です。

Speech To Life is a voice recognition plugin for Unreal Engine which gives you performant offline voice recognition for Windows and Android platforms including Quest 1 / 2. It includes a sentence fuzzy matching library to allow conversation selection like gameplay and easily supports voice command control found in games where your voice causes actions/events to occur in the game world. This can be a very powerful mechanic in VR games where it adds an extra layer of 'presence'.


Some examples of usage:

  • Choose dialog choices with NPCs using your voice
  • Have NPCs react to things you say, give commands
  • Ghost hunting game questions to the ghost to get responses / make the ghost do things.
  • Add an extra layer of controls for VR/AR applications


Considerations:

  • Speech recognition is completely offline and doesn't require an internet connection.
  • A speech model is around 40-80mb in size on average and needs to be completely loaded into memory to be used. You should include this in your memory budget when using this plugin.
  • This plugin only supports Windows and Android platforms. Linux, MacOS, and iOS will come at a future time.
  • The plugin uses Vosk for voice recognition, but if there is interest / if I feel the need I will add new recognition systems. Vosk is great because it is offline, works on all platforms, and is fast.


Speech To Life can support over 20 languages using the exceptional voice recognition supplied by Vosk.


This plugin contains a BONUS Fuzzy String Matching library exposed to blueprint for using a number of fuzzy string matching methods to match a voice recognition result sentence (or the words of) to a number of strings you are checking for. You can use this library for purposes other than voice recognition matching as well.


The plugin can be simply implemented into your own project and contains a fully working example found here.

NOTE: The example project is for Engine 4.27 and greater. Let me know on the Solar Storm Interactive Discord if you need a version for 4.26 and we can arrange something.

Engine 5.0 Example Project

Engine 5.0 Quest Specific Example Project

Engine 4.27 Example Project


Demos can be found here:

Documentation can be found here

Join the Solar Storm Interactive Discord here to get support!


This plugin would not be possible without the amazing work done on the Kaldi and Vosk projects. This plugin also contains the RapidFuzz C++ library for matching recognizer output sentence strings to strings pre-defined by the designer to be checked against.


Vosk : https://alphacephei.com/vosk/

RapidFuzz : https://github.com/maxbachmann/RapidFuzz

テクニカルノート

Features:

  • Offline, no internet required, voice recognition
  • Runs on Mobile Android Devices
  • Supports 20+ languages
  • Performant threaded implementation
  • Fuzzy matching library included for large sentence matching

Code Modules:

  • SpeechToUnreal (Runtime) Contains a game instance subsystem for globally controlling the current locale (language) used, a component to feed audio for voice recognition and get results, a function library for matching results in different ways, an extended audio device capture source for feeding microphone audio into the recognizer component, a bonus audio generator synth component for taking the microphone (or any audio generator source) and mixing it back into the games audio or to feed through mixers for creating walkie-talkie voice FX for example.
  • SpeechToUnrealRecognizers (Runtime) Recognizer implementation, currently includes Vosk but might contain more in the future if requested.
  • SpeechToUnrealEditor (Editor) For editor settings extension.
  • RapidFuzz (Runtime) A fuzzy string matching library

Number of C++ Classes: 10

Supported Development Platforms: Windows 64-bit

Supported Target Build Platforms: Windows 64-bit, Android, Oculus, Steam VR

Nativizable: Yes

Documentation: Here

Example Project: Here