Mycroft plugins

Mycroft is extendable by plugins. These plugins can add support for new Speech To Text engines, Text To Speech engines, wake word engines and add new audio playback options.

TTS - Base for TTS plugins

class mycroft.tts.TTS(lang, config, validator, audio_ext='wav', phonetic_spelling=True, ssml_tags=None)

TTS abstract class to be implemented by all TTS engines.

It aggregates the minimum required parameters and exposes execute(sentence) and validate_ssml(sentence) functions.

Parameters

• lang (str) –

• config (dict) – Configuration for this specific tts engine

• validator (TTSValidator) – Used to verify proper installation

• phonetic_spelling (bool) – Whether to spell certain words phonetically

• ssml_tags (list) – Supported ssml properties. Ex. [‘speak’, ‘prosody’]

begin_audio()

Helper function for child classes to call in execute().

clear_cache()

Remove all cached files.

end_audio(listen=False)

Helper function for child classes to call in execute().

Sends the recognizer_loop:audio_output_end message (indicating that speaking is done for the moment) as well as trigger listening if it has been requested. It also checks if cache directory needs cleaning to free up disk space.

Parameters

listen (bool) – indication if listening trigger should be sent.

execute(sentence, ident=None, listen=False)

Convert sentence to speech, preprocessing out unsupported ssml

The method caches results if possible using the hash of the sentence.

Parameters

• sentence – (str) Sentence to be spoken

• ident – (str) Id reference to current interaction

• listen – (bool) True if listen should be triggered at the end of the utterance.

get_tts(sentence, wav_file)

Abstract method that a tts implementation needs to implement.

Should get data from tts.

Parameters

• sentence (str) – Sentence to synthesize

• wav_file (str) – output file

Returns

(wav_file, phoneme)

Return type

tuple

init(bus)

Performs intial setup of TTS object.

Parameters

bus – Mycroft messagebus connection

load_phonemes(key)

Load phonemes from cache file.

Parameters

key (str) – Key identifying phoneme cache

load_spellings()

Load phonetic spellings of words as dictionary.

modify_tag(tag)

Override to modify each supported ssml tag.

Parameters

tag (str) – SSML tag to check and possibly transform.

static remove_ssml(text)

Removes SSML tags from a string.

Parameters

text (str) – input string

Returns

input string stripped from tags.

Return type

str

save_phonemes(key, phonemes)

Cache phonemes

Parameters

• key (str) – Hash key for the sentence

• phonemes (str) – phoneme string to save

validate_ssml(utterance)

Check if engine supports ssml, if not remove all tags.

Remove unsupported / invalid tags

Parameters

utterance (str) – Sentence to validate

Returns

validated_sentence

Return type

str

viseme(phonemes)

Create visemes from phonemes.

May be implemented to convert TTS phonemes into Mycroft mouth visuals.

Parameters

phonemes (str) – String with phoneme data

Returns

visemes

Return type

list

STT - base for STT plugins

class mycroft.stt.STT

STT Base class, all STT backends derive from this one.

abstract execute(audio, language=None)

Implementation of STT functionallity.

This method needs to be implemented by the derived class to implement the specific STT engine connection.

The method gets passed audio and optionally a language code and is expected to return a text string.

Parameters

• audio (AudioData) – audio recorded by mycroft.

• language (str) – optional language code

Returns

parsed text

Return type

str

static init_language(config_core)

Helper method to get language code from Mycroft config.

class mycroft.stt.StreamingSTT

ABC class for threaded streaming STT implemenations.

abstract create_streaming_thread()

Create thread for parsing audio chunks.

This method should be implemented by the derived class to return an instance derived from StreamThread to handle the audio stream and send it to the STT engine.

Returns

Thread to handle audio data.

Return type

StreamThread

execute(audio, language=None)

End the parsing thread and collect data.

stream_data(data)

Receiver of audio data.

Parameters

data (bytes) – raw audio data.

stream_start(language=None)

Indicate start of new audio stream.

This creates a new thread for handling the incomming audio stream as it’s collected by Mycroft.

Parameters

language (str) – optional language code for the new stream.

stream_stop()

Indicate that the audio stream has ended.

This will tear down the processing thread and collect the result

Returns

parsed text

Return type

str

class mycroft.stt.StreamThread(queue, language)

ABC class to be used with StreamingSTT class implementations.

This class reads audio chunks from a queue and sends it to a parsing STT engine.

Parameters

• queue (Queue) – Input Queue

• language (str) – language code for the current language.

abstract handle_audio_stream(audio, language)

Handling of audio stream.

Needs to be implemented by derived class to process audio data and optionally update self.text with the current hypothesis.

Argumens:

audio (bytes): raw audio data. language (str): language code for the current session.

run()

Thread entry point.

HotWordEngine - Base for Hotword engine plugins

class mycroft.client.speech.hotword_factory.HotWordEngine(key_phrase='hey mycroft', config=None, lang='en-us')

Hotword/Wakeword base class to be implemented by all wake word plugins.

Parameters

• key_phrase (str) – string representation of the wake word

• config (dict) – Configuration block for the specific wake word

• lang (str) – language code (BCP-47)

found_wake_word(frame_data)

Check if wake word has been found.

Checks if the wake word has been found. Should reset any internal tracking of the wake word state.

Parameters

frame_data (binary data) – Deprecated. Audio data for large chunk of audio to be processed. This should not be used to detect audio data instead use update() to incrementaly update audio

Returns

True if a wake word was detected, else False

Return type

bool

stop()

Perform any actions needed to shut down the wake word engine.

This may include things such as unloading data or shutdown external processess.

update(chunk)

Updates the hotword engine with new audio data.

The engine should process the data and update internal trigger state.

Parameters

chunk (bytes) – Chunk of audio data to process

AudioBackend - Base for audioservice backend plugins

class mycroft.audio.services.AudioBackend(config, bus)

Base class for all audio backend implementations.

Parameters

• config (dict) – configuration dict for the instance

• bus (MessageBusClient) – Mycroft messagebus emitter

abstract add_list(tracks)

Add tracks to backend’s playlist.

Parameters

tracks (list) – list of tracks.

abstract clear_list()

Clear playlist.

lower_volume()

Lower volume.

This method is used to implement audio ducking. It will be called when Mycroft is listening or speaking to make sure the media playing isn’t interfering.

next()

Skip to next track in playlist.

pause()

Pause playback.

Stops playback but may be resumed at the exact position the pause occured.

abstract play(repeat=False)

Start playback.

Starts playing the first track in the playlist and will contiune until all tracks have been played.

Parameters

repeat (bool) – Repeat playlist, defaults to False

previous()

Skip to previous track in playlist.

restore_volume()

Restore normal volume.

Called when to restore the playback volume to previous level after Mycroft has lowered it using lower_volume().

resume()

Resume paused playback.

Resumes playback after being paused.

seek_backward(seconds=1)

Rewind X seconds.

Parameters

seconds (int) – number of seconds to seek, if negative jump forward.

seek_forward(seconds=1)

Skip X seconds.

Parameters

seconds (int) – number of seconds to seek, if negative rewind

set_track_start_callback(callback_func)

Register callback on track start.

This method should be called as each track in a playlist is started.

shutdown()

Perform clean shutdown.

Implements any audio backend specific shutdown procedures.

abstract stop()

Stop playback.

Stops the current playback.

Returns

True if playback was stopped, otherwise False

Return type

bool

abstract supported_uris()

List of supported uri types.

Returns

Supported uri’s

Return type

list

track_info()

Get info about current playing track.

Returns

Track info containing atleast the keys artist and album.

Return type

dict

class mycroft.audio.services.RemoteAudioBackend(config, bus)

Base class for remote audio backends.

RemoteAudioBackends will always be checked after the normal AudioBackends to make playback start locally by default.

An example of a RemoteAudioBackend would be things like Chromecasts, mopidy servers, etc.