LipType Word-Initial Silent Speech (Su et al., 2025)
Measured by Su, Fang & Rekimoto · ACM CUI 2025 (2025)
Inputs
The measured or assumed values behind the calculations, each with its source.
- rate = 41.6 word/min
- Average of the reported LipType text-entry rates in the user study: 42.96 WPM two-handed and 40.23 WPM one-handed. The paper's WPM definition is characters/5 divided by input time, so this is an end-to-end text-entry speed including candidate selection and correction.
- P = 0.973
- 1 - 2.69% final uncorrected WER, averaging the reported LipType values: 2.64% two-handed and 2.74% one-handed.
- H = 5.0 bits/word
- Shannon per-word entropy of English; same word-level convention used for speech and silent-speech text-output entries.
Strictest ITR
Each scoring method is an upper bound on the channel, so the headline is the strictest (smallest) one for this entry. Use the score selector on the home page to view any single method across entries.
-
Measured end-to-end text-entry rate
(42.96 + 40.23) / 2 = 41.6 word/min
The paper defines WPM as characters/5 over input time, including the interaction cost of candidate selection and correction.
-
Apply final uncorrected word accuracy
P = 1 - ((2.64% + 2.74%) / 2) = 0.973; 41.6 x 0.973 = 40.5 net word/min
This uses final uncorrected WER from the text-entry study, not the raw pre-correction VSR output.
-
Information transfer rate
40.5 word/min × 5.0 bits/word ÷ 60 s/min ≈ 3.37 bits/s
Raw recognizer WER before correction was much higher (18.89%), so this should be read as a complete text-entry interface, not bare lip-reading throughput.
What counts as a bit depends on the action space. The number of distinguishable actions and how likely each one is are design choices of the task, not the sensing hardware. The same modality can present a fixed set of targets, a set pruned per step by a grammar or language model, or a continuous control space. Each of these changes how many actions are live and how the probability mass is spread, and therefore the information per selection. Read the action space below before comparing headline numbers across entries.
Action space
What the user can produce at each step, and how those options are distributed.
- Structure
- Context-dependent (the live set changes per step)
- Size
- Continuous
- Prior
- Context-conditioned: likelihoods depend on prior actions
- Notes
- LipType is a multimodal silent-speech text-entry system from Rekimoto's group. A phone camera captures lip motion while the user silently articulates a phrase and types each word's initial letter; an LLM-conditioned visual speech recognizer proposes candidates, and the user corrects the result. The typed initials and candidate interface are part of the measured channel, so this should be read as a complete multimodal text-entry system rather than bare lip-reading throughput. It is a cleaner atlas entry than the former Diener synthesis because WPM, final WER, raw recognizer WER, and the interaction protocol all come from one paper and one user study.
Comparability The strictest bound here is the Shannon entropy of the output text, under one predictor held constant across the whole atlas (≈1 bit per character). That shared predictor makes it directly comparable to every other text entry (keyboards, spellers, silent speech and speech BCIs) regardless of prior or vocabulary size. For most text interfaces it comes out tighter than the raw-selection bounds, but not always. Where a small vocabulary makes Wolpaw tighter, that wins instead. Any Fitts, Wolpaw or log₂(N) figure shown below is another bound on the same channel. Switch the home-page score selector to compare one across entries.
Source
- Authors
- Su, Fang & Rekimoto
- Publication
- ACM CUI 2025, 2025
- Paper
- 10.1145/3719160.3736612
- Reference
- Project/publication page, Shitao Fang