A Spotify patent details the company's plans to incorporate speech recognition into its content recommendation methods. The company is apparently exploring ways to use voice data to determine a user's emotional state, gender, age, and accent.

The patent, discovered by Music Business Worldwide, was filed in February 2018 and was just granted earlier this month. It details a system in which "provided audio signals" (presumably microphone input) are analyzed to determine the speaker's personal characteristics and hints of their physical and social environment; Spotify then suggests content that it believes is most appropriate for the user and their surroundings.

Spotify is well known for its ability to recommend content based on a user's listening habits, but it has never attempted to build a personal profile of a particular user to the extent detailed in this patent. The proposed system could, for example, use speech recognition to check whether a person is in a positive or negative mental state, or analyze ambient sounds to determine whether the person is alone or with friends.

The system would suggest brighter content if it thinks you are in a good mood and with friends, and relaxing music if it thinks you are alone and stressed.

"Prosodic information (e.g., intonation, stress, rhythm, etc.) can be combined and integrated with acoustic information within a hidden Markov model architecture. Using this architecture, the prosodic information can be used to detect and classify the emotional state of the speaker.

As is usual with newly discovered patents, especially those filed nearly two years ago, they do not necessarily reflect the final product. Spotify may have modified or even completely abandoned its speech recognition plans in the meantime.

Still, any system that would determine a user's personal characteristics and current surroundings raises serious privacy concerns. If Spotify moves forward with these or similar plans, we hope that the final product is at least an opt-in feature.