NeuroTechML

Can muscle signals distinguish speech sounds?

NeuroTechSC investigated whether surface electromyography could support phoneme recognition. We collected signals from facial and neck muscles, segmented labeled recordings, and trained recurrent models. The data collection grew from a small phoneme set to 44 phonemes plus a rest class, with four recording channels.

My work connected the data and model stages: cleaning and normalization, labeled-window generation, LSTM training, performance visualization, and integration with a streaming application. The team presented the research at the California Neurotechnology Conference.

The deployment assumptions changed the result

The most useful findings came from trying to move beyond the recorded dataset. Training on individually pronounced sounds did not transfer cleanly to continuous speech or subvocalization. Electrode placement and muscle selection changed the input distribution, and the amount and diversity of data were limiting factors.

One concrete mismatch was normalization. The offline pipeline could normalize using an entire recording; the live application received only a stream. Retraining with segment-local normalization did not recover the hoped-for performance. The repository retains that unsuccessful experiment and the resulting changes to our data-collection recommendations.

What the work established

The project produced documented data-processing and training pipelines, saved models, visual analysis, and a live integration attempt. Its lasting lesson is methodological: an offline score is meaningful only in relation to the recording, labeling, split, and deployment conditions that produced it.