Feed-loop stage that turns a voice media event into a %{text: transcript}
event before it is routed.
A platform adapter normalizes a voice note to
%{media: %{kind: :voice, ref: ref, ...}} (e.g.
Raxol.Telegram.GatewayAdapter for message.voice updates). This stage
fetches the audio behind ref, converts it to raw PCM, transcribes it, and
hands the transcript on as the ordinary %{text: binary} event every
Raxol.Gateway.Handler already understands. Any non-voice event passes
through untouched, so the stage composes unconditionally:
with {:ok, route, event} <- MyAdapter.normalize_event(raw),
{:ok, event} <- Transcribe.run(event, fetch_fn: fetch) do
Raxol.Gateway.SessionRouter.route(router, route, event)
endWhy a feed-loop stage, not a Session concern
A Raxol.Gateway.Session records the raw inbound event to its :log
before the handler sees it, and it processes events one at a time. Running
STT inside the session would log opaque media references instead of
transcripts and block the per-chat mailbox for the whole recognition run.
In the feed loop the logged event is the transcript, and only the feed
callback (e.g. the poller's :on_update) waits.
Options
:fetch_fn-(media_map -> {:ok, binary} | {:error, term}). Fetches the audio bytes behind the media:ref; platform-specific, so there is no default (a voice event without one is dropped). For Telegram:fn media -> Raxol.Telegram.GatewayAdapter.fetch_media(conn, media) end.:convert_fn-(binary -> {:ok, pcm} | {:error, term}). Converts the fetched audio to raw f32le mono 16kHz PCM (whatRaxol.Speech.Recognizer.recognize/1takes). Default:convert_with_ffmpeg/2with these options.:recognize_fn-(pcm -> {:ok, text} | {:error, term}). Default:Raxol.Speech.Recognizer.recognize/1when the optionalraxol_speechdependency is loaded and its Recognizer process is running.:max_bytes- drop audio larger than this before converting (default 20MB, the Bot APIgetFileceiling). Checked against the media's:size_bytesmetadata before fetching and against the fetched binary.:max_duration_s- drop voice notes longer than this (default 300, matching the raxol_speech Listener's recording cap). Checked against the media's:duration_smetadata before fetching, enforced in the default converter (ffmpeg-t), and re-checked against the decoded PCM size.:max_bytesalone cannot bound the decode: f32le mono 16kHz is 64000 bytes/s, so a low-bitrate note under the compressed ceiling would otherwise expand without limit in memory.:ffmpeg_path,:cmd_fn,:tmp_dir- seeconvert_with_ffmpeg/2.
A stage function that raises or exits (e.g. a recognizer process dying
mid-call) is caught and handled as that stage's failure; only a shape
summary of the crash reaches the log and telemetry, never the arguments
(a GenServer.call exit term embeds the full audio binary).
Failure mode
Fail-open per event: any stage failure (no :fetch_fn, fetch/convert
errors, STT unavailable, empty transcript) drops that one voice event with
:ignore, a Logger.warning, and [:raxol_gateway, :transcribe, :error]
telemetry (metadata %{stage, reason, kind}). Text traffic is unaffected;
a dropped voice note never takes the chat down. Successful runs emit
[:raxol_gateway, :transcribe, :done] with the audio and transcript sizes.
Neither audio bytes nor transcript text ever reach the logs.
Latency
The stage blocks its caller for the whole fetch + convert + recognize run.
The first recognition after boot also pays the XLA graph compile (minutes
on CPU); give Raxol.Speech.Recognizer a generous :recognize_timeout_ms
via Raxol.Speech.Supervisor's :recognizer_opts or warm the serving up
front, otherwise every cold call times out, aborts the compile, and the
next call starts it over.
Summary
Functions
Default :convert_fn: decode any audio ffmpeg understands to raw f32le
mono 16kHz PCM.
Transcribe a voice media event; pass any other event through unchanged.
Types
Functions
Default :convert_fn: decode any audio ffmpeg understands to raw f32le
mono 16kHz PCM.
System.cmd/3 cannot feed stdin, so the input bytes go through a
temporary file (deleted afterwards, success or not):
ffmpeg -hide_banner -loglevel error -i <tmp> -t <max_duration_s> -f f32le -ac 1 -ar 16000 pipe:1Options
:ffmpeg_path- executable path (defaultSystem.find_executable("ffmpeg")). Only a binary namedffmpegthat exists on disk is accepted.:cmd_fn-(path, args -> {output_binary, exit_status})runner override for tests (defaultSystem.cmd/3):tmp_dir- where the temporary input file goes (defaultSystem.tmp_dir!()):max_duration_s- decode at most this many seconds (-t, default 300), bounding the PCM held in memory regardless of how small the compressed input is
Transcribe a voice media event; pass any other event through unchanged.
Returns {:ok, %{text: transcript, source: :voice}} for a transcribed
voice event, {:ok, event} untouched for everything else, and :ignore
when a voice event cannot be transcribed (logged, never raised).