Meta on Tuesday released SeamlessM4T, the world’s first all-in-one multimodal and multilingual artificial intelligence (AI) translation and transcription model.
What it does: SeamlessM4T allows users to communicate with ease through speech and text across nearly 100 languages. It supports the following features:
- Speech recognition (nearly 100 languages)
- Speech-to-text translation (nearly 100 input and output languages)
- Speech-to-speech translation (nearly 100 input languages and 36 output languages, including English)
- Text-to-text translation (nearly 100 languages)
- Text-to-speech translation (nearly 100 input languages and 35 output languages, including English)
The big picture: Meta said SeamlessM4T is its latest step in ongoing efforts to build AI technology that helps connect people across languages. The model draws on findings from previous projects, such as No Language Left Behind, Universal Speech Translator and Massively Multilingual Speech.
What’s next: Researchers and developers are expected to build on SeamlessM4T, which Meta calls a “foundational” model. The company also released the metadata of its training dataset SeamlessAlign, which Meta described as the largest of its kind to date, having 470,000 hours of mind speech and text alignments.
