mediaplayer/android

Oral Trainer Android SDK

Android SDK for online oral-training video playback. It provides an embeddable Media3/ExoPlayer player, streaming cache, tablet gestures, sentence navigation, the server video catalog, and remote imitation-quality assessment.

Modules

• oral-trainer-sdk: Android library module to publish as an AAR.
• sample-app: Minimal Android app showing SDK integration.

Default Gesture Mapping

• Single tap on the playback area: play / pause.
• Swipe left: previous sentence; if there is no sentence data, rewind by the configured seek step.
• Swipe right: next sentence; if there is no sentence data, fast-forward by the configured seek step.

The sample app's previous/next sentence buttons use the same fallback behavior. Actual sentence navigation requires SentenceBoundary timestamps supplied by the course service or extracted from subtitle/speech analysis data.

These defaults match the desktop player's arrow-key workflow while fitting a tablet touch screen.

Sentence Playback Modes

PlayerConfig.continuousPlayback controls how playback crosses sentence boundaries:

• true (default): the video plays continuously through all sentences.
• false: playback stays on the current sentence; when its boundary is reached, playback loops back to the start of the same sentence until the student swipes to the previous/next sentence.

Toggle it at runtime with controller.setContinuousPlayback(enabled).

controller.playCurrentSentenceAndStop() plays only the current sentence and automatically pauses once it finishes — useful for a "play the reference sentence" button in a recording test flow.

Basic Integration

val sdk = OralTrainerSdk.init(context)
val controller = sdk.createController(
    playerConfig = PlayerConfig(
        sentenceMode = true,
        defaultSeekStepMs = 10_000L,
        autoPlay = false,
    )
)

val playerView = OralTrainerPlayerView(context)
playerView.bind(controller)

controller.loadItem(
    TrainingMediaItem(
        id = "lesson_01",
        title = "Lesson 01",
        uri = Uri.parse("https://cdn.example.com/lesson_01.mp4"),
        customCacheKey = "lesson_01",
        sentences = listOf(
            SentenceBoundary(0, 0L, 4200L, "Listen and repeat."),
            SentenceBoundary(1, 4200L, 9000L, "Swipe to jump by sentence."),
        ),
    )
)

Online Streaming And Cache

The SDK uses AndroidX Media3 SimpleCache through CacheDataSource.Factory. It supports regular MP4 streams plus HLS and DASH through Media3. Configure the cache location and maximum size at initialization:

OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(maxCacheBytes = 1024L * 1024L * 1024L)
)

Sentence Boundary API

The SDK defaults to https://videoservice.d1kt.cn and requests:

GET /api/v1/videos/{sha256}/sentence-boundaries

For a known hash:

sdk.sentenceBoundaryApi.fetch(videoHash, callback)

For a local content:// video, the SDK can hash the file in streaming chunks before querying the API:

sdk.sentenceBoundaryApi.fetchForUri(videoUri, contentResolver, callback)

Override the service only when a staging or private deployment is required:

OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(
        sentenceBoundaryApiBaseUrl = "https://videoservice.d1kt.cn",
        assessmentApiKey = BuildConfig.ORAL_TRAINER_CLIENT_KEY,
    )
)

Video Catalog

The SDK loads ready videos from GET /api/v1/videos and resolves both absolute and relative stream URLs:

sdk.videoCatalogApi.fetch(object : VideoCatalogCallback {
    override fun onSuccess(videos: List<TrainingVideoSummary>) {
        val video = videos.first()
        sdk.sentenceBoundaryApi.fetch(video.videoHash, object : SentenceBoundaryApiCallback {
            override fun onSuccess(result: SentenceBoundaryApiResult) {
                controller.loadItem(video.toTrainingMediaItem(result.sentences))
            }

            override fun onError(error: Throwable) {
                controller.loadItem(video.toTrainingMediaItem())
            }
        })
    }

    override fun onError(error: Throwable) = Unit
})

Sample App Configuration

The sample app points at the cloud service through the SERVER_BASE_URL constant at the top of MainActivity.kt:

const val SERVER_BASE_URL = "https://videoservice.d1kt.cn"

Change it to your real deployment address before building the APK. For a LAN or development server, an http://<server-ip>:<port> address works because the sample app permits cleartext HTTP (see res/xml/network_security_config.xml); production should keep HTTPS.

The sample app also enables allowHttpFallback: if an https:// request fails with a TLS error (for example a misconfigured or unstable gateway/certificate), the SDK automatically retries the same request over plain HTTP. This is a convenience for private deployments only — fix the TLS edge in production.

The app has two modules, switched by the 训练 / 测试 tabs at the top:

• 训练: browse the cloud course list and do sentence-by-sentence playback practice. The old built-in "示例课程" card was removed; only courses served by GET /api/v1/videos appear here.
• 测试: record your reading of the current sentence and submit it to the server's assessment endpoint for AI scoring. Only cloud courses are supported (the server must know the video hash and reference sentences).

The video's bottom-right corner has a small rotation icon that forces landscape or portrait orientation, so rotation works even when the device's auto-rotate switch is turned off. The progress bar below the sentence panel is draggable, so the student can seek to any position by dragging it.

For the test module, fill in the client key that the server expects in the X-Client-Key header (the CLIENT_API_KEY value from the server's sentence_api/.env):

const val ASSESSMENT_API_KEY = "replace-with-your-client-api-key"

The app requests microphone permission at runtime on first use.

Troubleshooting "云端暂不可用"

That label means GET /api/v1/videos failed. Since the fix, the sample app shows the underlying error next to the label (for example DNS resolution, timeout, TLS, or an unexpected HTTP status). Common causes:

• The APK was built with the wrong SERVER_BASE_URL for your deployment.
• The phone cannot reach the domain/IP (check with a browser on the phone, and verify the server with curl https://<server>/api/v1/videos).
• The gateway only exposes plain HTTP, so the app must use http://... instead of https://....
• The server certificate is invalid/self-signed on the phone; use HTTP for internal testing or install a valid certificate at the gateway.

Local Video Testing

The computer path /Users/... is not visible to an Android device. For a quick test, copy the video to the device or use the 选择本地视频 button in the sample app. The Android document picker returns a content:// URI, which the SDK supports without requesting broad storage permissions.

For production courses, keep the video on an HTTPS CDN or object-storage service and pass its URL as TrainingMediaItem.uri. The SDK streams it and caches downloaded ranges locally. Copying large 4K files to each device is better reserved for explicitly offline courses.

Remote Imitation Scoring

Create the controller with the built-in remote assessor:

val remoteAssessor = sdk.createRemoteImitationQualityAssessor()
val controller = sdk.createController(
    imitationAssessor = remoteAssessor
)

Then call assessCurrentSentence(recordingUri, callback) after the student records a sentence. The current TrainingMediaItem.id must be the server video SHA-256, which is already true for items created by TrainingVideoSummary:

controller.assessCurrentSentence(
    recordingUri = recordingUri,
    locale = "en",
    callback = object : ImitationAssessmentCallback {
        override fun onResult(result: ImitationAssessmentResult) {
            val passed = result.passed == true
            val score = result.overallScore
            val durationRatio = result.durationRatio
        }

        override fun onError(error: Throwable) = Unit
    }
)

The server returns content, completeness, fluency, duration, pause, and speech rate scores. Phoneme pronunciation and prosody are nullable until the dedicated models are enabled.

Build

Prerequisites: JDK 17 or newer and Android SDK Platform 36.1. Open the android/ directory in Android Studio, or run:

ANDROID_HOME="$HOME/Library/Android/sdk" ./gradlew :oral-trainer-sdk:assembleDebug

If the Gradle ZIP has already been downloaded, extract it and select the extracted directory in Android Studio under Settings > Build, Execution, Deployment > Build Tools > Gradle > Gradle distribution > Local installation. For example, the local installation directory on the development machine is /Users/liushuming/Downloads/ssss/gradle-8.13.