Mistral AI: Mistral-Common v1.9.0 - Audio Streaming Support | SignalBreak

Mistral AI: Mistral-Common v1.9.0 - Audio Streaming Support | SignalBreak | SignalBreak

Source text

Mistral-Common can now process streaming requests import numpy as np from mistral_common . audio import Audio from mistral_common . protocol . instruct . chunk import RawAudio from mistral_common . protocol . transcription . request import ( StreamingMode , TranscriptionRequest , ) from mistral_common . tokens . tokenizers . mistral import MistralTokenizer # 1. Load the tokenizer with audio support tokenizer = MistralTokenizer . from_hf_hub ( "mistralai/Voxtral-Mini-4B-Realtime-2602" ) # 2. Create sample audio data (or load from a file) sampling_rate = 16_000 duration_s = 2.0 audio_array = np . random . uniform ( - 1 , 1 , size = int ( duration_s * sampling_rate )). astype ( np . float32 ) audio = Audio ( audio_array = audio_array , sampling_rate = sampling_rate , format = "wav" , ) # 3. Create the streaming transcription request request = TranscriptionRequest ( audio = RawAudio ( data = audio . to_base64 ( "wav" ), format = "wav" , ), streaming = StreamingMode . ONLINE , # or StreamingMode.OFFLINE language = None , ) # 4. Encode the request tokenized = tokenizer . encode_transcription ( request ) # 5. Access the results print ( f"Tokens: { tokenized . tokens } " ) print ( f"Number of tokens: { len ( tokenized . tokens ) } " ) print ( f"Number of audio segments: { len ( tokenized . audios ) } " ) See https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602 for more info. What's Changed Add new token logic asrstr by @patrickvonplaten in #172 [Backward comp] Still need the _control_tokens for vLLM by @patrickvonplaten in #173 Release 1.8.8 by @juliendenize in #174 [Audio] Update padding by @patrickvonplaten in #175 [Audio] Improve padding for streaming by @patrickvonplaten in #177 Add audio_encoder to Tokenizer V13 by @amosyou in #180 Release v1.9.0 - Audio streaming by @patrickvonplaten in #179 Fix image tests with downloads by @juliendenize in #181 Enhance accessibility by @juliendenize in #176 New Contributors @amosyou made their first contribution in #180 Fu

View original source

Mistral-Common v1.9.0 - Audio Streaming Support

More from Mistral AI

Get alerts for Mistral AI