LipSurf User ManualChanging or Adding Languages
🇯🇵 🇺🇸 🇫🇷 🇪🇸 🇷🇺 🇬🇧 🇧🇾 🇨🇦
Changing Language
Say the language name with LipSurf on (eg. Japanese or Russian) or change the language in the settings (open settings by saying options or right clicking the extension icon and clicking "options").
Adding Languages
Languages can be added and improved by anyone.
Steps
1) Install the Chrome Extension and get a feeling for how it works.
2) Translate the name: LipSurf - Voice Control for the Web and description:
Browse the web with your voice -- eating, multitasking or just to give your hands a break. Speech recognition for executing voice commands in the browser that would normally require a mouse and keyboard.
Excellent as an assistive technology.
Voice control integrations:
* Gmail - select & mark messages, compose emails, dictate emails, etc.
https://youtu.be/o1zIHxkg2KU?t=37
* Youtube & other video sites - play, pause, fast-forward etc.
* Duolingo & WaniKani - answer questions completely based on voice control (handsfree)
https://www.youtube.com/watch?v=pBtFl8qHgpE
* Reddit - speech recognition for upvote, visit posts, comment etc.
Features:
* High performance speech recognition
* Speech to text (STT)
* Shows live transcript as you speak
* Wake-word (say "hey LipSurf" to activate hands-free) (Premium)
* Custom shortcuts - map any phrase to a command
* Text to speech (TTS)
* Works in Google Sheets (Premium)
* Limited commands on PDFs
General voice commands:
* Write and edit text (Premium)
* Scroll up, down, little down, little up, autoscroll, bottom, top, left, right...
* Annotate links to click them with voice
* Open multiple links
* Spell mode (Premium)
* Write emails and dictate forms (Premium)
* Open websites/URLs (Premium)
* Weather lookup
* Dictionary
* Control videos: play, pause, mute etc.
* Add custom corrections, shortcuts, homophones or synonyms by clicking the transcript!
* Voice activation via push-to-talk key
* Click things by saying "click [thing text]" or "tag" then saying the annotation in the yellow box next to it.
* Navigate tabs: next tab, previous tab, select tab 3, new tab, etc.
* Zoom in, zoom out
* Multi-language support (Spanish, French, Russian and Japanese - more languages may be added by anyone via our forum)
* 100+ built-in commands that can be expanded to infinity with additional plugins and tons of options.
Languages:
* 🇬🇧 English
* 🇪🇸 Spanish
* 🇫🇷 French
* 🇺🇦 Ukrainian
* 🇷🇺 Russian
* 🇯🇵 Japanese
* 🇧🇷 Portuguese (Duolingo only)
* 🇩🇪 German (Duolingo only)
* 🇫🇮 Finnish (Duolingo only)
* 🇳🇱 Dutch (Duolingo only)
Please go through the tutorial to make sure you configure everything properly, and just say "help" to see the commands that are possible for the current page you're on! Enjoy!
See information about updates along with video demos here: https://www.lipsurf.com/updates
Forum where you can make feature requests and ask questions: https://discuss.lipsurf.com3) Translate the commands.
TIP
- Languages can be added incrementally. Since there are many commands, you can start with the most essential commands, and add the rest as time permits.
- It's better to start by translating one plugin, then testing (step 4). You will get a feel for how command words work that way and it will save you from needing to do more readjusting later.
If you're comfortable editing JavaScript files, you can edit the translation files directly and submit a pull-request on Github. You don't need to understand programming, you just need to keep the structure of the file in-tact when you edit it.
Otherwise, you can just write your translations out in a document and we'll take care of programming it in for you.
Each plugin optionally has translation files (eg. for the Scroll.ts plugin there is Scroll.ja.ts, Scroll.ru.ts, Scroll.fr.ts etc.) Here is a fragment of a translation file:
...
Scroll.languages!.ru = { // notice the same 2 character language code as in the filename
niceName: "Браузер",
description: "Контроль действий в браузере, как-то: создание новой вкладки, навигация по странице (назад, вперед, вниз), вызов справки и т.д.",
authors: "Hanna, Miko", // add your name here
homophones: { // format: "[misheard homophone]": "[correct command word/phrase]", see step 4
"вниис": "вниз",
},
commands: {
"Scroll Down": { // the original English name of the command we're translating
name: "Прокрутить вниз",
match: ["вниз"],
},
...
For each plugin the following need to be translated:
niceName
description (if it's not obvious)
For each command the following need to be translated:
name
description (if it's not obvious)
match (command words)
This is what the user says to execute the command (they are the most important thing!)
Case and hyphen insensitive
an
*represents wildcard (anything can be said in it's position. Eg.google *will match google no bake cheesecake)- Command words cannot start with an
*, but they may have an*in the middle.
- Command words cannot start with an
a
#represents any numeral or ordinal number (eg. the command word for changing tabs istab #and# tabso you can say things like tab 1 or first tab).Aim for short - especially for the most used commands (eg.
topinstead ofscroll to top of the page,auto scrollinstead ofautomatically scroll down).There may be more than one match phrase. It's good to have at least all the intuitive variations.
Try to avoid using command words that match the beginnings of command words for different commands.
4) Test the commands.
Sometimes the speech recognizer will pick up different homophones for certain words. Take note of these homophones and add them to the translation file so they can be automatically corrected.
Sometimes certain commands will be very hard for the speech recognizer - you might need to change them to work around this.
Command words should be adjusted based on these real-world tests (back to step 3).