mediaplayer/android/README.md

# Oral Trainer Android SDK

Android SDK for online oral-training video playback. It provides an embeddable
Media3/ExoPlayer player, streaming cache, tablet gestures, sentence navigation,
the server video catalog, and remote imitation-quality assessment.

## Modules

- `oral-trainer-sdk`: Android library module to publish as an AAR.
- `sample-app`: Minimal Android app showing SDK integration.

## Default Gesture Mapping

- Single tap on the playback area: play / pause.
- Swipe left: previous sentence; if there is no sentence data, rewind by the
  configured seek step.
- Swipe right: next sentence; if there is no sentence data, fast-forward by the
  configured seek step.

The sample app's previous/next sentence buttons use the same fallback behavior.
Actual sentence navigation requires `SentenceBoundary` timestamps supplied by
the course service or extracted from subtitle/speech analysis data.

These defaults match the desktop player's arrow-key workflow while fitting a
tablet touch screen.

## Basic Integration

```kotlin
val sdk = OralTrainerSdk.init(context)
val controller = sdk.createController(
    playerConfig = PlayerConfig(
        sentenceMode = true,
        defaultSeekStepMs = 10_000L,
        autoPlay = false,
    )
)

val playerView = OralTrainerPlayerView(context)
playerView.bind(controller)

controller.loadItem(
    TrainingMediaItem(
        id = "lesson_01",
        title = "Lesson 01",
        uri = Uri.parse("https://cdn.example.com/lesson_01.mp4"),
        customCacheKey = "lesson_01",
        sentences = listOf(
            SentenceBoundary(0, 0L, 4200L, "Listen and repeat."),
            SentenceBoundary(1, 4200L, 9000L, "Swipe to jump by sentence."),
        ),
    )
)
```

## Online Streaming And Cache

The SDK uses AndroidX Media3 `SimpleCache` through `CacheDataSource.Factory`.
It supports regular MP4 streams plus HLS and DASH through Media3. Configure the
cache location and maximum size at initialization:

```kotlin
OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(maxCacheBytes = 1024L * 1024L * 1024L)
)
```

## Sentence Boundary API

The SDK defaults to `https://video_service.d1kt.cn` and requests:

```text
GET /api/v1/videos/{sha256}/sentence-boundaries
```

For a known hash:

```kotlin
sdk.sentenceBoundaryApi.fetch(videoHash, callback)
```

For a local `content://` video, the SDK can hash the file in streaming chunks
before querying the API:

```kotlin
sdk.sentenceBoundaryApi.fetchForUri(videoUri, contentResolver, callback)
```

Override the service only when a staging or private deployment is required:

```kotlin
OralTrainerSdk.init(
    context,
    OralTrainerSdkConfig(
        sentenceBoundaryApiBaseUrl = "https://video_service.d1kt.cn",
        assessmentApiKey = BuildConfig.ORAL_TRAINER_CLIENT_KEY,
    )
)
```

## Video Catalog

The SDK loads ready videos from `GET /api/v1/videos` and resolves both absolute
and relative stream URLs:

```kotlin
sdk.videoCatalogApi.fetch(object : VideoCatalogCallback {
    override fun onSuccess(videos: List<TrainingVideoSummary>) {
        val video = videos.first()
        sdk.sentenceBoundaryApi.fetch(video.videoHash, object : SentenceBoundaryApiCallback {
            override fun onSuccess(result: SentenceBoundaryApiResult) {
                controller.loadItem(video.toTrainingMediaItem(result.sentences))
            }

            override fun onError(error: Throwable) {
                controller.loadItem(video.toTrainingMediaItem())
            }
        })
    }

    override fun onError(error: Throwable) = Unit
})
```

## Local Video Testing

The computer path `/Users/...` is not visible to an Android device. For a
quick test, copy the video to the device or use the `选择本地视频` button in the
sample app. The Android document picker returns a `content://` URI, which the
SDK supports without requesting broad storage permissions.

For production courses, keep the video on an HTTPS CDN or object-storage
service and pass its URL as `TrainingMediaItem.uri`. The SDK streams it and
caches downloaded ranges locally. Copying large 4K files to each device is
better reserved for explicitly offline courses.

## Remote Imitation Scoring

Create the controller with the built-in remote assessor:

```kotlin
val remoteAssessor = sdk.createRemoteImitationQualityAssessor()
val controller = sdk.createController(
    imitationAssessor = remoteAssessor
)
```

Then call `assessCurrentSentence(recordingUri, callback)` after the student
records a sentence. The current `TrainingMediaItem.id` must be the server video
SHA-256, which is already true for items created by `TrainingVideoSummary`:

```kotlin
controller.assessCurrentSentence(
    recordingUri = recordingUri,
    locale = "en",
    callback = object : ImitationAssessmentCallback {
        override fun onResult(result: ImitationAssessmentResult) {
            val passed = result.passed == true
            val score = result.overallScore
            val durationRatio = result.durationRatio
        }

        override fun onError(error: Throwable) = Unit
    }
)
```

The server returns content, completeness, fluency, duration, pause, and speech
rate scores. Phoneme pronunciation and prosody are nullable until the dedicated
models are enabled.

## Build

Prerequisites: JDK 17 or newer and Android SDK Platform 36.1. Open the
`android/` directory in Android Studio, or run:

```bash
ANDROID_HOME="$HOME/Library/Android/sdk" ./gradlew :oral-trainer-sdk:assembleDebug
```

If the Gradle ZIP has already been downloaded, extract it and select the
extracted directory in Android Studio under `Settings > Build, Execution,
Deployment > Build Tools > Gradle > Gradle distribution > Local installation`.
For example, the local installation directory on the development machine is
`/Users/liushuming/Downloads/ssss/gradle-8.13`.