Oral Trainer Android SDK

Android SDK for online oral-training video playback. It provides an embeddable Media3/ExoPlayer player, streaming cache, tablet gestures, sentence navigation, the server video catalog, and remote imitation-quality assessment.

Modules

oral-trainer-sdk: Android library module to publish as an AAR.
sample-app: Minimal Android app showing SDK integration.

Default Gesture Mapping

Single tap on the playback area: play / pause.
Swipe left: previous sentence; if there is no sentence data, rewind by the configured seek step.
Swipe right: next sentence; if there is no sentence data, fast-forward by the configured seek step.

The sample app's previous/next sentence buttons use the same fallback behavior. Actual sentence navigation requires SentenceBoundary timestamps supplied by the course service or extracted from subtitle/speech analysis data.

These defaults match the desktop player's arrow-key workflow while fitting a tablet touch screen.

Basic Integration

val sdk = OralTrainerSdk.init(context)
val controller = sdk.createController(
    playerConfig = PlayerConfig(
        sentenceMode = true,
        defaultSeekStepMs = 10_000L,
        autoPlay = false,
    )
)

val playerView = OralTrainerPlayerView(context)
playerView.bind(controller)

controller.loadItem(
    TrainingMediaItem(
        id = "lesson_01",
        title = "Lesson 01",
        uri = Uri.parse("https://cdn.example.com/lesson_01.mp4"),
        customCacheKey = "lesson_01",
        sentences = listOf(
            SentenceBoundary(0, 0L, 4200L, "Listen and repeat."),
            SentenceBoundary(1, 4200L, 9000L, "Swipe to jump by sentence."),
        ),
    )
)

Online Streaming And Cache

The SDK uses AndroidX Media3 SimpleCache through CacheDataSource.Factory. It supports regular MP4 streams plus HLS and DASH through Media3. Configure the cache location and maximum size at initialization:

OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(maxCacheBytes = 1024L * 1024L * 1024L)
)

Sentence Boundary API

The SDK defaults to https://video_service.d1kt.cn and requests:

GET /api/v1/videos/{sha256}/sentence-boundaries

For a known hash:

sdk.sentenceBoundaryApi.fetch(videoHash, callback)

For a local content:// video, the SDK can hash the file in streaming chunks before querying the API:

sdk.sentenceBoundaryApi.fetchForUri(videoUri, contentResolver, callback)

Override the service only when a staging or private deployment is required:

OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(
        sentenceBoundaryApiBaseUrl = "https://video_service.d1kt.cn",
        assessmentApiKey = BuildConfig.ORAL_TRAINER_CLIENT_KEY,
    )
)

Video Catalog

The SDK loads ready videos from GET /api/v1/videos and resolves both absolute and relative stream URLs:

sdk.videoCatalogApi.fetch(object : VideoCatalogCallback {
    override fun onSuccess(videos: List<TrainingVideoSummary>) {
        val video = videos.first()
        sdk.sentenceBoundaryApi.fetch(video.videoHash, object : SentenceBoundaryApiCallback {
            override fun onSuccess(result: SentenceBoundaryApiResult) {
                controller.loadItem(video.toTrainingMediaItem(result.sentences))
            }

            override fun onError(error: Throwable) {
                controller.loadItem(video.toTrainingMediaItem())
            }
        })
    }

    override fun onError(error: Throwable) = Unit
})

Local Video Testing

The computer path /Users/... is not visible to an Android device. For a quick test, copy the video to the device or use the 选择本地视频 button in the sample app. The Android document picker returns a content:// URI, which the SDK supports without requesting broad storage permissions.

For production courses, keep the video on an HTTPS CDN or object-storage service and pass its URL as TrainingMediaItem.uri. The SDK streams it and caches downloaded ranges locally. Copying large 4K files to each device is better reserved for explicitly offline courses.

Remote Imitation Scoring

Create the controller with the built-in remote assessor:

val remoteAssessor = sdk.createRemoteImitationQualityAssessor()
val controller = sdk.createController(
    imitationAssessor = remoteAssessor
)

Then call assessCurrentSentence(recordingUri, callback) after the student records a sentence. The current TrainingMediaItem.id must be the server video SHA-256, which is already true for items created by TrainingVideoSummary:

controller.assessCurrentSentence(
    recordingUri = recordingUri,
    locale = "en",
    callback = object : ImitationAssessmentCallback {
        override fun onResult(result: ImitationAssessmentResult) {
            val passed = result.passed == true
            val score = result.overallScore
            val durationRatio = result.durationRatio
        }

        override fun onError(error: Throwable) = Unit
    }
)

The server returns content, completeness, fluency, duration, pause, and speech rate scores. Phoneme pronunciation and prosody are nullable until the dedicated models are enabled.

Build

Prerequisites: JDK 17 or newer and Android SDK Platform 36.1. Open the android/ directory in Android Studio, or run:

ANDROID_HOME="$HOME/Library/Android/sdk" ./gradlew :oral-trainer-sdk:assembleDebug

If the Gradle ZIP has already been downloaded, extract it and select the extracted directory in Android Studio under Settings > Build, Execution, Deployment > Build Tools > Gradle > Gradle distribution > Local installation. For example, the local installation directory on the development machine is /Users/liushuming/Downloads/ssss/gradle-8.13.

5.7 KiB Raw Blame History