mediaplayer/sentence_api

Oral Trainer Video Service

FastAPI service for the Android oral-training SDK. It provides:

• chunked video upload with SHA-256 calculation;
• files stored under data/v/ and metadata stored in SQLite;
• after the first processing pass, videos can be uploaded automatically to Volcano Engine Object Storage (TOS / 火山 OSS) and the catalog/share UI will use that public URL;
• background sentence transcription through MOSS-Transcribe-Diarize;
• a browser-based administration page at /admin;
• video catalog and HTTP Range playback;
• sentence-boundary lookup compatible with the existing Android SDK;
• student recording upload and asr-fluency-v1 assessment;
• signed-in user history for test results and dubbing shares, including deletion;
• AI sentence explanation for signed-in mobile users;
• the legacy JSON boundary index as a read-only fallback.

Local Run

python3.12 -m venv .venv-sentence-api
. .venv-sentence-api/bin/activate
python -m pip install -r sentence_api/requirements.txt

export ADMIN_API_KEY=local-development-key
export MOSS_TRANSCRIBE_URL=http://127.0.0.1:8001/v1/audio/transcriptions
python -m uvicorn sentence_api.main:app --host 0.0.0.0 --port 8000

Open:

http://127.0.0.1:8000/admin
http://127.0.0.1:8000/docs

When MOSS_TRANSCRIBE_URL is empty, uploaded videos still receive silence-based boundaries, but no reference transcript is produced and assessment is disabled.

When all VOLCANO_OSS_* credentials in .env.example are configured, the first processing pass saves sentence boundaries, uploads the source video, and only then marks it ready. stream_url and the legacy /content endpoint then point to the OSS/CDN URL. If the upload fails, the video is marked failed so clients do not fall back to server-local playback.

Main Endpoints

GET    /healthz
GET    /api/v1/videos
GET    /api/v1/videos/{sha256}
GET    /api/v1/videos/{sha256}/content
GET    /api/v1/videos/{sha256}/sentence-boundaries
POST   /api/v1/audio/sentence-boundaries
POST   /api/v1/sentences/explain
POST   /api/v1/videos/{sha256}/sentences/{index}/assessments

POST   /api/v1/admin/videos
PUT    /api/v1/admin/videos/raw
POST   /api/v1/admin/videos/{sha256}/process
POST   /api/v1/admin/videos/{sha256}/repair-gaps
GET    /api/v1/admin/videos/{sha256}/subtitle-translation
POST   /api/v1/admin/videos/{sha256}/subtitle-translation
PUT    /api/v1/admin/videos/{sha256}/sentences/{index}
PUT    /api/v1/admin/videos/{sha256}/sentences/{index}/split
PUT    /api/v1/admin/videos/{sha256}/sentences/{index}/boundary
PUT    /api/v1/admin/users/password
DELETE /api/v1/admin/videos/{sha256}

Admin endpoints use X-Admin-Key when ADMIN_API_KEY is configured. Assessment requests use X-Client-Key when CLIENT_API_KEY is configured.

Administrators can set a local password for an SSO-linked account with:

curl -X PUT https://your-server/api/v1/admin/users/password \
  -H "X-Admin-Key: $ADMIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"username":"bobcoc","password":"new-password"}'

POST /api/v1/audio/sentence-boundaries accepts an audio file from an authenticated client, generates boundaries in memory/temp storage only, and returns a sentence document. It does not register or archive the uploaded audio.

AI Sentence Explanation

Configure an OpenAI-compatible chat-completions endpoint in sentence_api/.env. A base URL ending in /v1 is also accepted and is resolved to /v1/chat/completions:

AI_API_URL=https://api.example.com/v1/chat/completions
AI_API_KEY=replace-with-your-ai-api-key
AI_MODEL=gpt-4o-mini
AI_TIMEOUT_SECONDS=45

The endpoint requires a signed-in user and accepts only the sentence and course context. AI_API_URL and AI_API_KEY remain on the server and are never sent to the mobile clients.

Chinese captions are stored on the course in the server database and shared by every user. GET /api/v1/videos/{sha256}/subtitle-translation returns the saved captions without calling the AI, including complete and status so clients can poll long videos. POST starts missing-caption generation in the background and returns quickly; it translates in small windows with neighboring context, runs several windows in parallel, retries and splits a window if the model response is invalid, and writes each finished window immediately so later viewers can reuse it. English sentence edits clear the saved Chinese captions for that course.

Desktop playback of a local file can POST the same endpoint with the locally generated English cues (duration_ms, algorithm_version, sentences). If that SHA-256 is not already a course, the server stores the English subtitles and translates them; later GET/POST calls poll and reuse the saved Chinese captions. Official course sentences are never overwritten by a local upload.

Assessment

curl -X POST \
  http://127.0.0.1:8000/api/v1/videos/{sha256}/sentences/0/assessments \
  -F audio=@student.wav \
  -F language=en

The current score intentionally measures reading content and fluency:

overall = content * 80% + fluency * 20%
fluency = duration * 35% + pauses * 40% + speech rate * 25%

pronunciation_score and prosody_score remain null until a phoneme/GOP model is connected.

Tests

python -m pytest sentence_api/tests -q

See DEPLOYMENT.md for Docker, MOSS/Whisper, Nginx HTTP 回源, large-file upload, backup, and production operation instructions.