Unlocking the Power of Voice with Flask

Voice interfaces stopped being a novelty a while back, and clients keep asking for them in places we did not expect 'em a few years ago — internal support tools, warehouse apps, even accessibility layers bolted onto existing dashboards. Flask is our go-to when the ask is "add a voice front end to this thing quickly" because it gets out of the way. This post walks through the actual request flow we use: capture audio in the browser, ship it to Flask, transcribe it, hand the text to a chat engine, and get a response back to the user.

Capturing Audio in the Browser

The MediaRecorder API does the heavy lifting client-side. You request microphone access with getUserMedia, record into chunks, and on stop you assemble a Blob and POST it as multipart form data. We default to audio/webm because Chrome and Edge support it natively without extra codecs, and we transcode server-side if a client needs WAV.

let mediaRecorder;
let chunks = [];

async function startRecording() {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/webm' });
    chunks = [];

    mediaRecorder.ondataavailable = (e) => chunks.push(e.data);
    mediaRecorder.onstop = async () => {
        const blob = new Blob(chunks, { type: 'audio/webm' });
        const formData = new FormData();
        formData.append('audio', blob, 'utterance.webm');

        const res = await fetch('/api/voice', { method: 'POST', body: formData });
        const data = await res.json();
        document.getElementById('reply').innerText = data.reply_text;

        if (data.audio_url) {
            new Audio(data.audio_url).play();
        }
    };

    mediaRecorder.start();
}

function stopRecording() {
    mediaRecorder.stop();
}

The Flask Endpoint

On the server, the uploaded blob lands in request.files. We save it to a temp path, convert it to WAV with pydub (which shells out to ffmpeg), then run it through SpeechRecognition's Google Web Speech recognizer for quick prototypes, or swap in Azure Cognitive Services / AWS Transcribe for production workloads that need better accuracy and privacy guarantees.

import os
import tempfile
import speech_recognition as sr
from pydub import AudioSegment
from flask import Flask, request, jsonify

app = Flask(__name__)
recognizer = sr.Recognizer()

@app.route('/api/voice', methods=['POST'])
def handle_voice():
    audio_file = request.files.get('audio')
    if audio_file is None:
        return jsonify({'error': 'no audio uploaded'}), 400

    with tempfile.TemporaryDirectory() as tmp_dir:
        webm_path = os.path.join(tmp_dir, 'input.webm')
        wav_path = os.path.join(tmp_dir, 'input.wav')
        audio_file.save(webm_path)

        AudioSegment.from_file(webm_path).export(wav_path, format='wav')

        with sr.AudioFile(wav_path) as source:
            audio_data = recognizer.record(source)
            transcript = recognizer.recognize_google(audio_data)

    reply_text = generate_chat_reply(transcript)

    return jsonify({
        'transcript': transcript,
        'reply_text': reply_text,
    })

Feeding the Chat Engine and Closing the Loop

generate_chat_reply is intentionally a black box in the snippet above — that is where the transcript hits whatever conversational engine backs the product, whether that is an LLM call, a rules-based intent matcher, or a retrieval pipeline over internal docs. Once you have reply_text, you have two options for getting sound back to the user: return text and let the browser's SpeechSynthesis API speak it locally (free, zero server cost, slightly robotic), or generate an audio file server-side with a TTS provider and return a URL the browser plays back. We use the browser API for internal tools and a hosted TTS voice for anything customer-facing.

• Keep audio chunks small (a few seconds) if you want live-transcription feel instead of record-then-send
• Always validate mimetype and size on the Flask side — browsers lie about codec support more than you would like
• Cache STT results during development so you are not burning API quota on every reload
• Run ffmpeg/pydub conversion in a background thread or task queue for anything beyond a handful of concurrent users

Wrapping Up

None of this is exotic — it is three well-understood pieces (MediaRecorder, a Flask upload route, an STT call) wired together in the right order. The part worth spending your design time on is everything after the transcript: latency budget, fallback text input for noisy environments, and making sure the chat engine gets enough context to produce a reply worth speaking back.