add android support

This commit is contained in:
2026-08-14 18:08:29 +08:00
parent 3b0fd719d9
commit f8a6bf24e4
26 changed files with 1538 additions and 0 deletions

View File

@@ -113,3 +113,11 @@ pip install -r requirements.txt
- **Linux 缺少 tkinter**`sudo apt install python3-tk`Debian/Ubuntu
- **Linux 下视频没有嵌入主窗口**:多见于 Wayland 环境,程序会自动改用独立视频窗口。
- **macOS 视频显示在独立窗口**:这是平台限制,属正常行为,`F` 键仍可全屏。
## Android SDK
仓库新增了 `android/` 子工程,用于把播放器和口语训练操作做成 Android SDK。
SDK 基于 AndroidX Media3 / ExoPlayer支持在线视频流、本地缓存、播放区单击
暂停/继续、左划/右划按句或按步长跳转,并预留了学生模仿质量评分算法接口。
详见 [android/README.md](android/README.md)。

6
android/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
.gradle/
.kotlin/
**/build/
local.properties
.idea/
*.iml

92
android/README.md Normal file
View File

@@ -0,0 +1,92 @@
# Oral Trainer Android SDK
Android SDK for online oral-training video playback. It provides an embeddable
Media3/ExoPlayer player, streaming cache, tablet gestures, sentence navigation,
and a placeholder interface for future imitation-quality scoring.
## Modules
- `oral-trainer-sdk`: Android library module to publish as an AAR.
- `sample-app`: Minimal Android app showing SDK integration.
## Default Gesture Mapping
- Single tap on the playback area: play / pause.
- Swipe left: previous sentence; if there is no sentence data, rewind by the
configured seek step.
- Swipe right: next sentence; if there is no sentence data, fast-forward by the
configured seek step.
These defaults match the desktop player's arrow-key workflow while fitting a
tablet touch screen.
## Basic Integration
```kotlin
val sdk = OralTrainerSdk.init(context)
val controller = sdk.createController(
playerConfig = PlayerConfig(
sentenceMode = true,
defaultSeekStepMs = 10_000L,
autoPlay = false,
)
)
val playerView = OralTrainerPlayerView(context)
playerView.bind(controller)
controller.loadItem(
TrainingMediaItem(
id = "lesson_01",
title = "Lesson 01",
uri = Uri.parse("https://cdn.example.com/lesson_01.mp4"),
customCacheKey = "lesson_01",
sentences = listOf(
SentenceBoundary(0, 0L, 4200L, "Listen and repeat."),
SentenceBoundary(1, 4200L, 9000L, "Swipe to jump by sentence."),
),
)
)
```
## Online Streaming And Cache
The SDK uses AndroidX Media3 `SimpleCache` through `CacheDataSource.Factory`.
It supports regular MP4 streams plus HLS and DASH through Media3. Configure the
cache location and maximum size at initialization:
```kotlin
OralTrainerSdk.init(
context,
OralTrainerSdkConfig(maxCacheBytes = 1024L * 1024L * 1024L)
)
```
## Future Imitation Scoring
Provide an implementation of `ImitationQualityAssessor` when the speech
assessment algorithm is ready:
```kotlin
val controller = sdk.createController(
imitationAssessor = MyImitationQualityAssessor()
)
```
Then call `assessCurrentSentence(recordingUri, callback)` after the student
records a sentence.
## Build
Prerequisites: JDK 17 or newer and Android SDK Platform 36.1. Open the
`android/` directory in Android Studio, or run:
```bash
ANDROID_HOME="$HOME/Library/Android/sdk" ./gradlew :oral-trainer-sdk:assembleDebug
```
If the Gradle ZIP has already been downloaded, extract it and select the
extracted directory in Android Studio under `Settings > Build, Execution,
Deployment > Build Tools > Gradle > Gradle distribution > Local installation`.
For example, the local installation directory on the development machine is
`/Users/liushuming/Downloads/ssss/gradle-8.13`.

5
android/build.gradle.kts Normal file
View File

@@ -0,0 +1,5 @@
plugins {
id("com.android.application") version "8.13.2" apply false
id("com.android.library") version "8.13.2" apply false
id("org.jetbrains.kotlin.android") version "2.2.10" apply false
}

View File

@@ -0,0 +1,4 @@
android.useAndroidX=true
android.nonTransitiveRClass=true
kotlin.code.style=official
org.gradle.jvmargs=-Xmx3g -Dfile.encoding=UTF-8

Binary file not shown.

View File

@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.13-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists

252
android/gradlew vendored Executable file
View File

@@ -0,0 +1,252 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
#
##############################################################################
#
# Gradle start up script for POSIX generated by Gradle.
#
# Important for running:
#
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
# noncompliant, but you have some other compliant shell such as ksh or
# bash, then to run this script, type that shell name before the whole
# command line, like:
#
# ksh Gradle
#
# Busybox and similar reduced shells will NOT work, because this script
# requires all of these POSIX shell features:
# * functions;
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
# * compound commands having a testable exit status, especially «case»;
# * various built-in commands including «command», «set», and «ulimit».
#
# Important for patching:
#
# (2) This script targets any POSIX shell, so it avoids extensions provided
# by Bash, Ksh, etc; in particular arrays are avoided.
#
# The "traditional" practice of packing multiple parameters into a
# space-separated string is a well documented source of bugs and security
# problems, so this is (mostly) avoided, by progressively accumulating
# options in "$@", and eventually passing that to Java.
#
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
# see the in-line comments for details.
#
# There are tweaks for specific operating systems such as AIX, CygWin,
# Darwin, MinGW, and NonStop.
#
# (3) This script is generated from the Groovy template
# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
# within the Gradle project.
#
# You can find Gradle at https://github.com/gradle/gradle/.
#
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
app_path=$0
# Need this for daisy-chained symlinks.
while
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
[ -h "$app_path" ]
do
ls=$( ls -ld "$app_path" )
link=${ls#*' -> '}
case $link in #(
/*) app_path=$link ;; #(
*) app_path=$APP_HOME$link ;;
esac
done
# This is normally unused
# shellcheck disable=SC2034
APP_BASE_NAME=${0##*/}
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s
' "$PWD" ) || exit
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD=maximum
warn () {
echo "$*"
} >&2
die () {
echo
echo "$*"
echo
exit 1
} >&2
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "$( uname )" in #(
CYGWIN* ) cygwin=true ;; #(
Darwin* ) darwin=true ;; #(
MSYS* | MINGW* ) msys=true ;; #(
NONSTOP* ) nonstop=true ;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD=$JAVA_HOME/jre/sh/java
else
JAVACMD=$JAVA_HOME/bin/java
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD=java
if ! command -v java >/dev/null 2>&1
then
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
fi
# Increase the maximum file descriptors if we can.
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
case $MAX_FD in #(
max*)
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
MAX_FD=$( ulimit -H -n ) ||
warn "Could not query maximum file descriptor limit"
esac
case $MAX_FD in #(
'' | soft) :;; #(
*)
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
# shellcheck disable=SC2039,SC3045
ulimit -n "$MAX_FD" ||
warn "Could not set maximum file descriptor limit to $MAX_FD"
esac
fi
# Collect all arguments for the java command, stacking in reverse order:
# * args from the command line
# * the main class name
# * -classpath
# * -D...appname settings
# * --module-path (only if needed)
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
# For Cygwin or MSYS, switch paths to Windows format before running java
if "$cygwin" || "$msys" ; then
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" )
JAVACMD=$( cygpath --unix "$JAVACMD" )
# Now convert the arguments - kludge to limit ourselves to /bin/sh
for arg do
if
case $arg in #(
-*) false ;; # don't mess with options #(
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
[ -e "$t" ] ;; #(
*) false ;;
esac
then
arg=$( cygpath --path --ignore --mixed "$arg" )
fi
# Roll the args list around exactly as many times as the number of
# args, so each arg winds up back in the position where it started, but
# possibly modified.
#
# NB: a `for` loop captures its iteration list before it begins, so
# changing the positional parameters here affects neither the number of
# iterations, nor the values presented in `arg`.
shift # remove old arg
set -- "$@" "$arg" # push replacement arg
done
fi
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Collect all arguments for the java command:
# * DEFAULT_JVM_OPTS, JAVA_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
# and any embedded shellness will be escaped.
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
# treated as '${Hostname}' itself on the command line.
set -- \
"-Dorg.gradle.appname=$APP_BASE_NAME" \
-classpath "$CLASSPATH" \
org.gradle.wrapper.GradleWrapperMain \
"$@"
# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi
# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
#
# In Bash we could simply go:
#
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
# set -- "${ARGS[@]}" "$@"
#
# but POSIX shell has neither arrays nor command substitution, so instead we
# post-process each arg (as a line of input to sed) to backslash-escape any
# character that might be a shell metacharacter, then use eval to reverse
# that process (while maintaining the separation between arguments), and wrap
# the whole thing up as a single "set" statement.
#
# This will of course break if any of these variables contains a newline or
# an unmatched quote.
#
eval "set -- $(
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
xargs -n1 |
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
tr '\n' ' '
)" '"$@"'
exec "$JAVACMD" "$@"

94
android/gradlew.bat vendored Normal file
View File

@@ -0,0 +1,94 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@rem SPDX-License-Identifier: Apache-2.0
@rem
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%"=="" set DIRNAME=.
@rem This is normally unused
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if %ERRORLEVEL% equ 0 goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo. 1>&2
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
echo. 1>&2
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
echo location of your Java installation. 1>&2
goto fail
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
:end
@rem End local scope for the variables with windows NT shell
if %ERRORLEVEL% equ 0 goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View File

@@ -0,0 +1,44 @@
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
plugins {
id("com.android.library")
id("org.jetbrains.kotlin.android")
}
android {
namespace = "cn.learningpad.oraltrainer.sdk"
compileSdk {
version = release(36) {
minorApiLevel = 1
}
}
buildToolsVersion = "36.1.0"
defaultConfig {
minSdk = 26
consumerProguardFiles("consumer-rules.pro")
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
}
kotlin {
compilerOptions {
jvmTarget.set(JvmTarget.JVM_17)
}
}
dependencies {
val media3Version = "1.11.0"
implementation("androidx.media3:media3-common:$media3Version")
implementation("androidx.media3:media3-database:$media3Version")
implementation("androidx.media3:media3-datasource:$media3Version")
implementation("androidx.media3:media3-exoplayer:$media3Version")
implementation("androidx.media3:media3-exoplayer-dash:$media3Version")
implementation("androidx.media3:media3-exoplayer-hls:$media3Version")
implementation("androidx.media3:media3-ui:$media3Version")
}

View File

@@ -0,0 +1 @@
-keep class cn.learningpad.oraltrainer.sdk.** { *; }

View File

@@ -0,0 +1,3 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
</manifest>

View File

@@ -0,0 +1,48 @@
package cn.learningpad.oraltrainer.sdk
import android.net.Uri
data class ImitationAssessmentRequest @JvmOverloads constructor(
val mediaId: String,
val sentence: SentenceBoundary,
val recordingUri: Uri,
val referenceAudioUri: Uri? = null,
val locale: String? = null,
val metadata: Map<String, String> = emptyMap(),
)
data class ImitationAssessmentResult @JvmOverloads constructor(
val overallScore: Float,
val pronunciationScore: Float? = null,
val fluencyScore: Float? = null,
val completenessScore: Float? = null,
val feedback: String? = null,
val details: Map<String, String> = emptyMap(),
)
fun interface CancellableAssessment {
fun cancel()
}
interface ImitationAssessmentCallback {
fun onResult(result: ImitationAssessmentResult)
fun onError(error: Throwable)
}
interface ImitationQualityAssessor {
fun assess(
request: ImitationAssessmentRequest,
callback: ImitationAssessmentCallback,
): CancellableAssessment
}
object NoopImitationQualityAssessor : ImitationQualityAssessor {
override fun assess(
request: ImitationAssessmentRequest,
callback: ImitationAssessmentCallback,
): CancellableAssessment {
callback.onError(UnsupportedOperationException("No imitation quality assessor is configured."))
return CancellableAssessment {}
}
}

View File

@@ -0,0 +1,372 @@
@file:androidx.media3.common.util.UnstableApi
package cn.learningpad.oraltrainer.sdk
import android.content.Context
import android.net.Uri
import android.os.Handler
import android.os.Looper
import androidx.media3.common.C
import androidx.media3.common.MediaItem
import androidx.media3.common.PlaybackException
import androidx.media3.common.Player
import androidx.media3.exoplayer.ExoPlayer
import androidx.media3.exoplayer.source.DefaultMediaSourceFactory
import java.util.concurrent.CopyOnWriteArraySet
import kotlin.math.max
import kotlin.math.min
class OralTrainerController internal constructor(
context: Context,
private val sdkConfig: OralTrainerSdkConfig,
initialConfig: PlayerConfig,
private val imitationAssessor: ImitationQualityAssessor,
) {
private val appContext = context.applicationContext
private val listeners = CopyOnWriteArraySet<OralTrainerListener>()
private val mediaItems = mutableListOf<TrainingMediaItem>()
private val mainHandler = Handler(Looper.getMainLooper())
private var released = false
private var lastSentenceIndex: Int? = null
var config: PlayerConfig = initialConfig
private set
var loopMode: LoopMode = LoopMode.ALL
private set
val player: ExoPlayer
private val ticker = object : Runnable {
override fun run() {
if (released) {
return
}
notifySentenceIfChanged()
notifySnapshot()
mainHandler.postDelayed(this, 250L)
}
}
init {
val mediaSourceFactory = DefaultMediaSourceFactory(
StreamingCache.dataSourceFactory(appContext, sdkConfig)
)
player = ExoPlayer.Builder(appContext)
.setMediaSourceFactory(mediaSourceFactory)
.build()
.also { exoPlayer ->
exoPlayer.repeatMode = Player.REPEAT_MODE_ALL
exoPlayer.playWhenReady = initialConfig.autoPlay
exoPlayer.addListener(object : Player.Listener {
override fun onPlaybackStateChanged(playbackState: Int) {
notifySnapshot()
}
override fun onIsPlayingChanged(isPlaying: Boolean) {
notifySnapshot()
}
override fun onMediaItemTransition(mediaItem: MediaItem?, reason: Int) {
lastSentenceIndex = null
listeners.forEach { it.onMediaChanged(currentTrainingItem()) }
notifySentenceIfChanged(force = true)
notifySnapshot()
}
override fun onPositionDiscontinuity(
oldPosition: Player.PositionInfo,
newPosition: Player.PositionInfo,
reason: Int,
) {
notifySentenceIfChanged(force = true)
notifySnapshot()
}
override fun onPlayerError(error: PlaybackException) {
listeners.forEach { it.onPlayerError(error) }
}
})
}
mainHandler.post(ticker)
}
fun addListener(listener: OralTrainerListener) {
listeners.add(listener)
}
fun removeListener(listener: OralTrainerListener) {
listeners.remove(listener)
}
fun updateConfig(config: PlayerConfig) {
this.config = config
}
@JvmOverloads
fun loadCourse(
course: TrainingCourse,
startIndex: Int = 0,
startPositionMs: Long = 0L,
) {
require(course.items.isNotEmpty()) { "TrainingCourse must contain at least one media item." }
loadItems(course.items, startIndex, startPositionMs)
}
@JvmOverloads
fun loadItems(
items: List<TrainingMediaItem>,
startIndex: Int = 0,
startPositionMs: Long = 0L,
) {
require(items.isNotEmpty()) { "At least one TrainingMediaItem is required." }
val safeIndex = startIndex.coerceIn(0, items.lastIndex)
mediaItems.clear()
mediaItems.addAll(items)
lastSentenceIndex = null
player.setMediaItems(items.map { it.toMedia3Item() }, safeIndex, max(0L, startPositionMs))
player.prepare()
player.playWhenReady = config.autoPlay
listeners.forEach { it.onMediaChanged(currentTrainingItem()) }
notifySentenceIfChanged(force = true)
notifySnapshot()
}
fun loadItem(item: TrainingMediaItem) {
loadItems(listOf(item))
}
fun play() {
player.play()
}
fun pause() {
player.pause()
}
fun togglePlayPause() {
if (player.isPlaying) {
pause()
} else {
play()
}
}
fun stop() {
player.stop()
}
fun seekTo(positionMs: Long) {
player.seekTo(max(0L, positionMs))
}
fun seekRelative(deltaMs: Long) {
seekTo(player.currentPosition + deltaMs)
}
fun rewind(stepMs: Long = config.defaultSeekStepMs) {
seekRelative(-max(0L, stepMs))
}
fun fastForward(stepMs: Long = config.defaultSeekStepMs) {
seekRelative(max(0L, stepMs))
}
fun previousItem() {
player.seekToPreviousMediaItem()
}
fun nextItem() {
player.seekToNextMediaItem()
}
fun setPlaybackSpeed(speed: Float) {
val clamped = min(config.maxPlaybackSpeed, max(config.minPlaybackSpeed, speed))
player.setPlaybackSpeed(clamped)
}
fun setVolume(volume: Float) {
player.volume = volume.coerceIn(0f, 1f)
}
fun setLoopMode(loopMode: LoopMode) {
this.loopMode = loopMode
player.repeatMode = when (loopMode) {
LoopMode.OFF -> Player.REPEAT_MODE_OFF
LoopMode.ONE -> Player.REPEAT_MODE_ONE
LoopMode.ALL -> Player.REPEAT_MODE_ALL
}
}
fun setSentenceMode(enabled: Boolean) {
config = config.copy(sentenceMode = enabled)
}
fun currentTrainingItem(): TrainingMediaItem? {
val index = player.currentMediaItemIndex
return mediaItems.getOrNull(index)
}
fun currentSentence(): SentenceBoundary? {
val item = currentTrainingItem() ?: return null
val position = player.currentPosition
return item.sentences.lastOrNull { sentence ->
position >= sentence.startMs && position < sentence.endMs
}
}
fun seekToPreviousSentence(): Boolean {
val item = currentTrainingItem() ?: return false
if (item.sentences.isEmpty()) {
return false
}
val position = player.currentPosition
val target = item.sentences
.asReversed()
.firstOrNull { it.startMs < position - PREVIOUS_SENTENCE_TOLERANCE_MS }
?: item.sentences.first()
seekTo(target.startMs)
return true
}
fun seekToNextSentence(): Boolean {
val item = currentTrainingItem() ?: return false
if (item.sentences.isEmpty()) {
return false
}
val position = player.currentPosition
val target = item.sentences.firstOrNull {
it.startMs > position + NEXT_SENTENCE_TOLERANCE_MS
} ?: return false
seekTo(target.startMs)
return true
}
fun performSwipeAction(action: SwipeAction) {
when (action) {
SwipeAction.NONE -> Unit
SwipeAction.REWIND -> rewind()
SwipeAction.FORWARD -> fastForward()
SwipeAction.PREVIOUS_SENTENCE -> seekToPreviousSentence()
SwipeAction.NEXT_SENTENCE -> seekToNextSentence()
SwipeAction.PREVIOUS_SENTENCE_OR_REWIND -> {
if (!config.sentenceMode || !seekToPreviousSentence()) {
rewind()
}
}
SwipeAction.NEXT_SENTENCE_OR_FORWARD -> {
if (!config.sentenceMode || !seekToNextSentence()) {
fastForward()
}
}
}
}
internal fun dispatchGesture(event: GestureEvent) {
listeners.forEach { it.onGesture(event) }
}
fun assessCurrentSentence(
recordingUri: Uri,
callback: ImitationAssessmentCallback,
referenceAudioUri: Uri? = null,
locale: String? = null,
metadata: Map<String, String> = emptyMap(),
): CancellableAssessment {
val item = currentTrainingItem()
?: return callback.unsupported("No media item is currently loaded.")
val sentence = currentSentence()
?: return callback.unsupported("No active sentence is available at the current position.")
val request = ImitationAssessmentRequest(
mediaId = item.id,
sentence = sentence,
recordingUri = recordingUri,
referenceAudioUri = referenceAudioUri,
locale = locale,
metadata = metadata,
)
return imitationAssessor.assess(request, callback)
}
fun snapshot(): PlaybackSnapshot {
val item = currentTrainingItem()
val duration = player.duration.takeUnless { it == C.TIME_UNSET } ?: -1L
return PlaybackSnapshot(
mediaId = item?.id,
positionMs = max(0L, player.currentPosition),
durationMs = duration,
bufferedPositionMs = max(0L, player.bufferedPosition),
isPlaying = player.isPlaying,
playbackState = player.playbackState.toOralTrainerState(),
playbackSpeed = player.playbackParameters.speed,
sentenceIndex = currentSentence()?.index,
)
}
fun release() {
if (released) {
return
}
released = true
mainHandler.removeCallbacks(ticker)
listeners.clear()
player.release()
}
private fun notifySnapshot() {
if (released) {
return
}
val snapshot = snapshot()
listeners.forEach { it.onPlaybackSnapshot(snapshot) }
}
private fun notifySentenceIfChanged(force: Boolean = false) {
val sentence = currentSentence()
val index = sentence?.index
if (!force && index == lastSentenceIndex) {
return
}
lastSentenceIndex = index
listeners.forEach { it.onSentenceChanged(sentence) }
}
private fun TrainingMediaItem.toMedia3Item(): MediaItem {
val builder = MediaItem.Builder()
.setMediaId(id)
.setUri(uri)
mimeType?.let(builder::setMimeType)
customCacheKey?.let(builder::setCustomCacheKey)
subtitleUri?.let { uri ->
builder.setSubtitleConfigurations(
listOf(
MediaItem.SubtitleConfiguration.Builder(uri)
.setMimeType(subtitleMimeType)
.setLanguage(subtitleLanguage)
.setSelectionFlags(C.SELECTION_FLAG_DEFAULT)
.build()
)
)
}
return builder.build()
}
private fun Int.toOralTrainerState(): OralTrainerPlaybackState {
return when (this) {
Player.STATE_BUFFERING -> OralTrainerPlaybackState.BUFFERING
Player.STATE_READY -> OralTrainerPlaybackState.READY
Player.STATE_ENDED -> OralTrainerPlaybackState.ENDED
else -> OralTrainerPlaybackState.IDLE
}
}
private fun ImitationAssessmentCallback.unsupported(message: String): CancellableAssessment {
onError(IllegalStateException(message))
return CancellableAssessment {}
}
private companion object {
const val PREVIOUS_SENTENCE_TOLERANCE_MS = 250L
const val NEXT_SENTENCE_TOLERANCE_MS = 150L
}
}

View File

@@ -0,0 +1,104 @@
package cn.learningpad.oraltrainer.sdk
import android.content.Context
import android.util.AttributeSet
import android.view.GestureDetector
import android.view.MotionEvent
import android.widget.FrameLayout
import androidx.media3.ui.PlayerView
import kotlin.math.abs
class OralTrainerPlayerView @JvmOverloads constructor(
context: Context,
attrs: AttributeSet? = null,
defStyleAttr: Int = 0,
) : FrameLayout(context, attrs, defStyleAttr) {
val playerView: PlayerView = PlayerView(context).apply {
useController = false
layoutParams = LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT)
}
private var controller: OralTrainerController? = null
private var gestureControls = GestureControlsConfig()
private val density = resources.displayMetrics.density
private val gestureDetector = GestureDetector(
context,
object : GestureDetector.SimpleOnGestureListener() {
override fun onDown(e: MotionEvent): Boolean = true
override fun onSingleTapConfirmed(e: MotionEvent): Boolean {
val activeController = controller ?: return false
if (!gestureControls.enabled || !gestureControls.tapTogglesPlayPause) {
return false
}
activeController.togglePlayPause()
activeController.dispatchGesture(GestureEvent(GestureKind.SINGLE_TAP))
return true
}
override fun onFling(
e1: MotionEvent?,
e2: MotionEvent,
velocityX: Float,
velocityY: Float,
): Boolean {
val activeController = controller ?: return false
val start = e1 ?: return false
if (!gestureControls.enabled) {
return false
}
val dx = e2.x - start.x
val dy = e2.y - start.y
val minDistancePx = gestureControls.minSwipeDistanceDp * density
val minVelocityPx = gestureControls.minSwipeVelocityDpPerSecond * density
if (abs(dx) < abs(dy) || abs(dx) < minDistancePx || abs(velocityX) < minVelocityPx) {
return false
}
val isRight = dx > 0
val action = if (isRight) {
gestureControls.rightSwipeAction
} else {
gestureControls.leftSwipeAction
}
activeController.performSwipeAction(action)
activeController.dispatchGesture(
GestureEvent(
kind = if (isRight) GestureKind.SWIPE_RIGHT else GestureKind.SWIPE_LEFT,
action = action,
)
)
return true
}
}
)
init {
addView(playerView)
isClickable = true
isFocusable = true
}
fun bind(controller: OralTrainerController) {
this.controller = controller
this.gestureControls = controller.config.gestureControls
playerView.player = controller.player
}
fun unbind() {
playerView.player = null
controller = null
}
fun updateGestureControls(config: GestureControlsConfig) {
gestureControls = config
}
override fun onTouchEvent(event: MotionEvent): Boolean {
return gestureDetector.onTouchEvent(event) || super.onTouchEvent(event)
}
override fun dispatchTouchEvent(ev: MotionEvent): Boolean {
return gestureDetector.onTouchEvent(ev) || super.dispatchTouchEvent(ev)
}
}

View File

@@ -0,0 +1,52 @@
package cn.learningpad.oraltrainer.sdk
import android.content.Context
class OralTrainerSdk private constructor(
context: Context,
val config: OralTrainerSdkConfig,
) {
private val appContext = context.applicationContext
val cache: OralTrainerCache = OralTrainerCache(appContext, config)
@JvmOverloads
fun createController(
playerConfig: PlayerConfig = PlayerConfig(),
imitationAssessor: ImitationQualityAssessor = NoopImitationQualityAssessor,
): OralTrainerController {
return OralTrainerController(
context = appContext,
sdkConfig = config,
initialConfig = playerConfig,
imitationAssessor = imitationAssessor,
)
}
fun release() {
StreamingCache.release()
}
companion object {
@Volatile
private var instance: OralTrainerSdk? = null
@JvmStatic
@JvmOverloads
fun init(
context: Context,
config: OralTrainerSdkConfig = OralTrainerSdkConfig(),
): OralTrainerSdk {
return synchronized(this) {
instance ?: OralTrainerSdk(context.applicationContext, config).also {
instance = it
}
}
}
@JvmStatic
fun get(): OralTrainerSdk {
return instance ?: error("Call OralTrainerSdk.init(context) before using the SDK.")
}
}
}

View File

@@ -0,0 +1,11 @@
package cn.learningpad.oraltrainer.sdk
import java.io.File
data class OralTrainerSdkConfig @JvmOverloads constructor(
val cacheDirectory: File? = null,
val maxCacheBytes: Long = 512L * 1024L * 1024L,
val userAgent: String = "OralTrainerSdk/0.1.0",
val connectTimeoutMs: Int = 15_000,
val readTimeoutMs: Int = 30_000,
)

View File

@@ -0,0 +1,42 @@
package cn.learningpad.oraltrainer.sdk
data class PlaybackSnapshot(
val mediaId: String?,
val positionMs: Long,
val durationMs: Long,
val bufferedPositionMs: Long,
val isPlaying: Boolean,
val playbackState: OralTrainerPlaybackState,
val playbackSpeed: Float,
val sentenceIndex: Int?,
)
enum class OralTrainerPlaybackState {
IDLE,
BUFFERING,
READY,
ENDED,
}
data class GestureEvent(
val kind: GestureKind,
val action: SwipeAction? = null,
)
enum class GestureKind {
SINGLE_TAP,
SWIPE_LEFT,
SWIPE_RIGHT,
}
interface OralTrainerListener {
fun onPlaybackSnapshot(snapshot: PlaybackSnapshot) = Unit
fun onMediaChanged(item: TrainingMediaItem?) = Unit
fun onSentenceChanged(sentence: SentenceBoundary?) = Unit
fun onGesture(event: GestureEvent) = Unit
fun onPlayerError(error: Throwable) = Unit
}

View File

@@ -0,0 +1,35 @@
package cn.learningpad.oraltrainer.sdk
data class PlayerConfig @JvmOverloads constructor(
val sentenceMode: Boolean = true,
val defaultSeekStepMs: Long = 10_000L,
val autoPlay: Boolean = false,
val minPlaybackSpeed: Float = 0.5f,
val maxPlaybackSpeed: Float = 2.0f,
val gestureControls: GestureControlsConfig = GestureControlsConfig(),
)
data class GestureControlsConfig @JvmOverloads constructor(
val enabled: Boolean = true,
val tapTogglesPlayPause: Boolean = true,
val leftSwipeAction: SwipeAction = SwipeAction.PREVIOUS_SENTENCE_OR_REWIND,
val rightSwipeAction: SwipeAction = SwipeAction.NEXT_SENTENCE_OR_FORWARD,
val minSwipeDistanceDp: Float = 48f,
val minSwipeVelocityDpPerSecond: Float = 160f,
)
enum class SwipeAction {
NONE,
REWIND,
FORWARD,
PREVIOUS_SENTENCE,
NEXT_SENTENCE,
PREVIOUS_SENTENCE_OR_REWIND,
NEXT_SENTENCE_OR_FORWARD,
}
enum class LoopMode {
OFF,
ONE,
ALL,
}

View File

@@ -0,0 +1,75 @@
@file:androidx.media3.common.util.UnstableApi
package cn.learningpad.oraltrainer.sdk
import android.content.Context
import androidx.media3.database.StandaloneDatabaseProvider
import androidx.media3.datasource.DataSource
import androidx.media3.datasource.cache.CacheDataSource
import androidx.media3.datasource.cache.LeastRecentlyUsedCacheEvictor
import androidx.media3.datasource.cache.SimpleCache
import androidx.media3.datasource.DefaultHttpDataSource
import java.io.File
internal object StreamingCache {
private var databaseProvider: StandaloneDatabaseProvider? = null
private var cache: SimpleCache? = null
@Synchronized
fun get(context: Context, config: OralTrainerSdkConfig): SimpleCache {
cache?.let { return it }
val appContext = context.applicationContext
val provider = StandaloneDatabaseProvider(appContext)
val directory = config.cacheDirectory ?: File(appContext.cacheDir, "oral_trainer_media_cache")
val evictor = LeastRecentlyUsedCacheEvictor(config.maxCacheBytes)
return SimpleCache(directory, evictor, provider).also {
databaseProvider = provider
cache = it
}
}
fun dataSourceFactory(
context: Context,
config: OralTrainerSdkConfig,
): DataSource.Factory {
val upstreamFactory = DefaultHttpDataSource.Factory()
.setUserAgent(config.userAgent)
.setConnectTimeoutMs(config.connectTimeoutMs)
.setReadTimeoutMs(config.readTimeoutMs)
.setAllowCrossProtocolRedirects(true)
return CacheDataSource.Factory()
.setCache(get(context, config))
.setUpstreamDataSourceFactory(upstreamFactory)
.setFlags(CacheDataSource.FLAG_IGNORE_CACHE_ON_ERROR)
}
@Synchronized
fun release() {
cache?.release()
cache = null
databaseProvider = null
}
@Synchronized
fun clear(context: Context, config: OralTrainerSdkConfig) {
val appContext = context.applicationContext
val directory = config.cacheDirectory ?: File(appContext.cacheDir, "oral_trainer_media_cache")
val provider = databaseProvider ?: StandaloneDatabaseProvider(appContext)
cache?.release()
cache = null
SimpleCache.delete(directory, provider)
databaseProvider = null
}
}
class OralTrainerCache internal constructor(
private val context: Context,
private val config: OralTrainerSdkConfig,
) {
fun sizeBytes(): Long = StreamingCache.get(context, config).cacheSpace
fun clear() {
StreamingCache.clear(context, config)
}
}

View File

@@ -0,0 +1,35 @@
package cn.learningpad.oraltrainer.sdk
import android.net.Uri
import androidx.media3.common.MimeTypes
data class SentenceBoundary @JvmOverloads constructor(
val index: Int,
val startMs: Long,
val endMs: Long,
val text: String? = null,
) {
init {
require(index >= 0) { "Sentence index must be non-negative." }
require(startMs >= 0) { "Sentence startMs must be non-negative." }
require(endMs > startMs) { "Sentence endMs must be greater than startMs." }
}
}
data class TrainingMediaItem @JvmOverloads constructor(
val id: String,
val title: String,
val uri: Uri,
val sentences: List<SentenceBoundary> = emptyList(),
val subtitleUri: Uri? = null,
val subtitleMimeType: String = MimeTypes.APPLICATION_SUBRIP,
val subtitleLanguage: String? = null,
val mimeType: String? = null,
val customCacheKey: String? = null,
)
data class TrainingCourse(
val id: String,
val title: String,
val items: List<TrainingMediaItem>,
)

View File

@@ -0,0 +1,40 @@
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
plugins {
id("com.android.application")
id("org.jetbrains.kotlin.android")
}
android {
namespace = "cn.learningpad.oraltrainer.sample"
compileSdk {
version = release(36) {
minorApiLevel = 1
}
}
buildToolsVersion = "36.1.0"
defaultConfig {
applicationId = "cn.learningpad.oraltrainer.sample"
minSdk = 26
targetSdk = 35
versionCode = 1
versionName = "0.1.0"
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
}
kotlin {
compilerOptions {
jvmTarget.set(JvmTarget.JVM_17)
}
}
dependencies {
implementation(project(":oral-trainer-sdk"))
implementation("androidx.media3:media3-common:1.11.0")
}

View File

@@ -0,0 +1,19 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<application
android:allowBackup="true"
android:label="Oral Trainer Sample"
android:supportsRtl="true"
android:theme="@style/AppTheme">
<activity
android:name=".MainActivity"
android:configChanges="keyboardHidden|orientation|screenSize"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>

View File

@@ -0,0 +1,156 @@
package cn.learningpad.oraltrainer.sample
import android.app.Activity
import android.graphics.Color
import android.net.Uri
import android.os.Bundle
import android.view.Gravity
import android.view.ViewGroup
import android.widget.Button
import android.widget.LinearLayout
import android.widget.TextView
import androidx.media3.common.MimeTypes
import cn.learningpad.oraltrainer.sdk.GestureEvent
import cn.learningpad.oraltrainer.sdk.LoopMode
import cn.learningpad.oraltrainer.sdk.OralTrainerController
import cn.learningpad.oraltrainer.sdk.OralTrainerListener
import cn.learningpad.oraltrainer.sdk.OralTrainerPlayerView
import cn.learningpad.oraltrainer.sdk.OralTrainerSdk
import cn.learningpad.oraltrainer.sdk.PlaybackSnapshot
import cn.learningpad.oraltrainer.sdk.PlayerConfig
import cn.learningpad.oraltrainer.sdk.SentenceBoundary
import cn.learningpad.oraltrainer.sdk.TrainingMediaItem
class MainActivity : Activity() {
private lateinit var controller: OralTrainerController
private lateinit var statusText: TextView
private lateinit var sentenceText: TextView
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
val sdk = OralTrainerSdk.init(this)
controller = sdk.createController(
playerConfig = PlayerConfig(
sentenceMode = true,
defaultSeekStepMs = 10_000L,
autoPlay = false,
)
)
controller.setLoopMode(LoopMode.ALL)
val playerView = OralTrainerPlayerView(this).apply {
bind(controller)
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
0,
1f,
)
}
statusText = TextView(this).apply {
setTextColor(Color.WHITE)
textSize = 14f
setPadding(24, 16, 24, 8)
}
sentenceText = TextView(this).apply {
setTextColor(Color.WHITE)
textSize = 18f
setPadding(24, 4, 24, 16)
}
val controls = LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL
gravity = Gravity.CENTER
setPadding(16, 8, 16, 20)
addView(commandButton("快退") { controller.rewind() })
addView(commandButton("播放/暂停") { controller.togglePlayPause() })
addView(commandButton("快进") { controller.fastForward() })
addView(commandButton("上一句") { controller.seekToPreviousSentence() })
addView(commandButton("下一句") { controller.seekToNextSentence() })
}
val root = LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
setBackgroundColor(Color.rgb(18, 18, 18))
addView(playerView)
addView(statusText)
addView(sentenceText)
addView(controls)
}
setContentView(root)
controller.addListener(object : OralTrainerListener {
override fun onPlaybackSnapshot(snapshot: PlaybackSnapshot) {
statusText.text = buildString {
append(if (snapshot.isPlaying) "播放中" else "已暂停")
append(" ")
append(format(snapshot.positionMs))
append(" / ")
append(format(snapshot.durationMs))
append(" 速度 ")
append(snapshot.playbackSpeed)
append("x")
}
}
override fun onSentenceChanged(sentence: SentenceBoundary?) {
sentenceText.text = sentence?.let {
"${it.index + 1} 句:${it.text.orEmpty()}"
} ?: "当前没有句子边界"
}
override fun onGesture(event: GestureEvent) {
statusText.text = "手势:${event.kind} ${event.action ?: ""}"
}
})
controller.loadItem(sampleOnlineLesson())
}
override fun onDestroy() {
controller.release()
super.onDestroy()
}
private fun commandButton(label: String, action: () -> Unit): Button {
return Button(this).apply {
text = label
setOnClickListener { action() }
layoutParams = LinearLayout.LayoutParams(
0,
ViewGroup.LayoutParams.WRAP_CONTENT,
1f,
).apply {
marginStart = 4
marginEnd = 4
}
}
}
private fun sampleOnlineLesson(): TrainingMediaItem {
return TrainingMediaItem(
id = "online_sample_01",
title = "Online Sample Lesson",
uri = Uri.parse("https://storage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"),
mimeType = MimeTypes.VIDEO_MP4,
customCacheKey = "online_sample_01",
sentences = listOf(
SentenceBoundary(0, 0L, 3_000L, "Listen once, then imitate."),
SentenceBoundary(1, 3_000L, 7_000L, "Swipe left to go back."),
SentenceBoundary(2, 7_000L, 11_000L, "Swipe right to move forward."),
SentenceBoundary(3, 11_000L, 16_000L, "Tap the video area to pause or play."),
),
)
}
private fun format(ms: Long): String {
if (ms < 0) {
return "--:--"
}
val totalSeconds = ms / 1000
val minutes = totalSeconds / 60
val seconds = totalSeconds % 60
return "%02d:%02d".format(minutes, seconds)
}
}

View File

@@ -0,0 +1,9 @@
<resources>
<style name="AppTheme" parent="android:style/Theme.Material.NoActionBar">
<item name="android:windowActionBar">false</item>
<item name="android:windowNoTitle">true</item>
<item name="android:windowFullscreen">false</item>
<item name="android:fontFamily">sans</item>
<item name="android:colorAccent">#2563EB</item>
</style>
</resources>

View File

@@ -0,0 +1,24 @@
pluginManagement {
repositories {
maven("https://maven.aliyun.com/repository/google")
maven("https://maven.aliyun.com/repository/central")
maven("https://maven.aliyun.com/repository/gradle-plugin")
google()
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
maven("https://maven.aliyun.com/repository/google")
maven("https://maven.aliyun.com/repository/central")
google()
mavenCentral()
}
}
rootProject.name = "OralTrainerAndroid"
include(":oral-trainer-sdk")
include(":sample-app")