beautified some ui

This commit is contained in:
2026-08-17 14:28:37 +08:00
parent 624b834632
commit d27c2e229e
15 changed files with 1023 additions and 200 deletions

View File

@@ -24,6 +24,22 @@ the course service or extracted from subtitle/speech analysis data.
These defaults match the desktop player's arrow-key workflow while fitting a These defaults match the desktop player's arrow-key workflow while fitting a
tablet touch screen. tablet touch screen.
## Sentence Playback Modes
`PlayerConfig.continuousPlayback` controls how playback crosses sentence
boundaries:
- `true` (default): the video plays continuously through all sentences.
- `false`: playback stays on the current sentence; when its boundary is
reached, playback loops back to the start of the same sentence until the
student swipes to the previous/next sentence.
Toggle it at runtime with `controller.setContinuousPlayback(enabled)`.
`controller.playCurrentSentenceAndStop()` plays only the current sentence and
automatically pauses once it finishes — useful for a "play the reference
sentence" button in a recording test flow.
## Basic Integration ## Basic Integration
```kotlin ```kotlin
@@ -123,6 +139,63 @@ sdk.videoCatalogApi.fetch(object : VideoCatalogCallback {
}) })
``` ```
## Sample App Configuration
The sample app points at the cloud service through the `SERVER_BASE_URL`
constant at the top of `MainActivity.kt`:
```kotlin
const val SERVER_BASE_URL = "https://videoservice.d1kt.cn"
```
Change it to your real deployment address before building the APK. For a LAN
or development server, an `http://<server-ip>:<port>` address works because the
sample app permits cleartext HTTP (see
`res/xml/network_security_config.xml`); production should keep HTTPS.
The sample app also enables `allowHttpFallback`: if an `https://` request fails
with a TLS error (for example a misconfigured or unstable gateway/certificate),
the SDK automatically retries the same request over plain HTTP. This is a
convenience for private deployments only — fix the TLS edge in production.
The app has two modules, switched by the `训练` / `测试` tabs at the top:
- `训练`: browse the cloud course list and do sentence-by-sentence playback
practice. The old built-in "示例课程" card was removed; only courses served
by `GET /api/v1/videos` appear here.
- `测试`: record your reading of the current sentence and submit it to the
server's assessment endpoint for AI scoring. Only cloud courses are
supported (the server must know the video hash and reference sentences).
The video's bottom-right corner has a small rotation icon that forces landscape
or portrait orientation, so rotation works even when the device's auto-rotate
switch is turned off. The progress bar below the sentence panel is draggable,
so the student can seek to any position by dragging it.
For the test module, fill in the client key that the server expects in the
`X-Client-Key` header (the `CLIENT_API_KEY` value from the server's
`sentence_api/.env`):
```kotlin
const val ASSESSMENT_API_KEY = "replace-with-your-client-api-key"
```
The app requests microphone permission at runtime on first use.
### Troubleshooting "云端暂不可用"
That label means `GET /api/v1/videos` failed. Since the fix, the sample app
shows the underlying error next to the label (for example DNS resolution,
timeout, TLS, or an unexpected HTTP status). Common causes:
- The APK was built with the wrong `SERVER_BASE_URL` for your deployment.
- The phone cannot reach the domain/IP (check with a browser on the phone, and
verify the server with `curl https://<server>/api/v1/videos`).
- The gateway only exposes plain HTTP, so the app must use `http://...` instead
of `https://...`.
- The server certificate is invalid/self-signed on the phone; use HTTP for
internal testing or install a valid certificate at the gateway.
## Local Video Testing ## Local Video Testing
The computer path `/Users/...` is not visible to an Android device. For a The computer path `/Users/...` is not visible to an Android device. For a

View File

@@ -0,0 +1,48 @@
package cn.learningpad.oraltrainer.sdk
import android.net.Uri
import javax.net.ssl.SSLException
/**
* Runs one blocking request against [baseUrl]. When [OralTrainerSdkConfig.allowHttpFallback]
* is enabled and the HTTPS attempt fails with a TLS error (broken certificates or
* misconfigured TLS termination at the gateway), retries the same request over plain HTTP.
* This is intended for private deployments; production should fix the TLS edge instead.
*/
internal class HttpFallback private constructor(
private val allowFallback: Boolean,
) {
fun <T> execute(baseUrl: String, block: (String) -> T): T {
try {
return block(baseUrl)
} catch (error: Throwable) {
if (!allowFallback || !isTlsFailure(error)) {
throw error
}
val uri = Uri.parse(baseUrl)
if (uri.scheme != "https") {
throw error
}
val httpUrl = uri.buildUpon().scheme("http").build().toString()
return block(httpUrl)
}
}
private fun isTlsFailure(error: Throwable): Boolean {
var current: Throwable? = error
while (current != null) {
val message = current.message.orEmpty()
if (current is SSLException || message.contains("SSL", ignoreCase = true)) {
return true
}
current = current.cause
}
return false
}
companion object {
fun forConfig(config: OralTrainerSdkConfig): HttpFallback {
return HttpFallback(config.allowHttpFallback)
}
}
}

View File

@@ -28,6 +28,8 @@ class OralTrainerController internal constructor(
private val mainHandler = Handler(Looper.getMainLooper()) private val mainHandler = Handler(Looper.getMainLooper())
private var released = false private var released = false
private var lastSentenceIndex: Int? = null private var lastSentenceIndex: Int? = null
private var stopAtSentenceEnd = false
private var stopAtSentenceEndIndex: Int? = null
var config: PlayerConfig = initialConfig var config: PlayerConfig = initialConfig
private set private set
@@ -42,6 +44,7 @@ class OralTrainerController internal constructor(
if (released) { if (released) {
return return
} }
enforceSentenceBoundary()
notifySentenceIfChanged() notifySentenceIfChanged()
notifySnapshot() notifySnapshot()
mainHandler.postDelayed(this, 250L) mainHandler.postDelayed(this, 250L)
@@ -124,6 +127,8 @@ class OralTrainerController internal constructor(
mediaItems.clear() mediaItems.clear()
mediaItems.addAll(items) mediaItems.addAll(items)
lastSentenceIndex = null lastSentenceIndex = null
stopAtSentenceEnd = false
stopAtSentenceEndIndex = null
player.setMediaItems(items.map { it.toMedia3Item() }, safeIndex, max(0L, startPositionMs)) player.setMediaItems(items.map { it.toMedia3Item() }, safeIndex, max(0L, startPositionMs))
player.prepare() player.prepare()
player.playWhenReady = config.autoPlay player.playWhenReady = config.autoPlay
@@ -141,6 +146,8 @@ class OralTrainerController internal constructor(
} }
fun pause() { fun pause() {
stopAtSentenceEnd = false
stopAtSentenceEndIndex = null
player.pause() player.pause()
} }
@@ -153,10 +160,14 @@ class OralTrainerController internal constructor(
} }
fun stop() { fun stop() {
stopAtSentenceEnd = false
stopAtSentenceEndIndex = null
player.stop() player.stop()
} }
fun seekTo(positionMs: Long) { fun seekTo(positionMs: Long) {
stopAtSentenceEnd = false
stopAtSentenceEndIndex = null
player.seekTo(max(0L, positionMs)) player.seekTo(max(0L, positionMs))
} }
@@ -202,6 +213,18 @@ class OralTrainerController internal constructor(
config = config.copy(sentenceMode = enabled) config = config.copy(sentenceMode = enabled)
} }
fun setContinuousPlayback(enabled: Boolean) {
config = config.copy(continuousPlayback = enabled)
}
fun playCurrentSentenceAndStop() {
val sentence = currentSentence() ?: return
player.seekTo(sentence.startMs)
stopAtSentenceEnd = true
stopAtSentenceEndIndex = sentence.index
player.play()
}
fun currentTrainingItem(): TrainingMediaItem? { fun currentTrainingItem(): TrainingMediaItem? {
val index = player.currentMediaItemIndex val index = player.currentMediaItemIndex
return mediaItems.getOrNull(index) return mediaItems.getOrNull(index)
@@ -335,6 +358,36 @@ class OralTrainerController internal constructor(
listeners.forEach { it.onSentenceChanged(sentence) } listeners.forEach { it.onSentenceChanged(sentence) }
} }
private fun enforceSentenceBoundary() {
if (!config.sentenceMode || !player.isPlaying) {
return
}
val item = currentTrainingItem() ?: return
if (item.sentences.isEmpty()) {
return
}
val position = player.currentPosition
if (stopAtSentenceEnd) {
val index = stopAtSentenceEndIndex ?: return
val sentence = item.sentences.getOrNull(index) ?: return
if (position >= sentence.endMs) {
stopAtSentenceEnd = false
stopAtSentenceEndIndex = null
player.pause()
player.seekTo(sentence.startMs)
}
return
}
if (config.continuousPlayback) {
return
}
val anchorIndex = lastSentenceIndex ?: return
val anchor = item.sentences.getOrNull(anchorIndex) ?: return
if (position >= anchor.endMs) {
player.seekTo(anchor.startMs)
}
}
private fun TrainingMediaItem.toMedia3Item(): MediaItem { private fun TrainingMediaItem.toMedia3Item(): MediaItem {
val builder = MediaItem.Builder() val builder = MediaItem.Builder()
.setMediaId(id) .setMediaId(id)

View File

@@ -10,4 +10,5 @@ data class OralTrainerSdkConfig @JvmOverloads constructor(
val readTimeoutMs: Int = 30_000, val readTimeoutMs: Int = 30_000,
val sentenceBoundaryApiBaseUrl: String = "https://videoservice.d1kt.cn", val sentenceBoundaryApiBaseUrl: String = "https://videoservice.d1kt.cn",
val assessmentApiKey: String? = null, val assessmentApiKey: String? = null,
val allowHttpFallback: Boolean = false,
) )

View File

@@ -4,6 +4,7 @@ data class PlayerConfig @JvmOverloads constructor(
val sentenceMode: Boolean = true, val sentenceMode: Boolean = true,
val defaultSeekStepMs: Long = 10_000L, val defaultSeekStepMs: Long = 10_000L,
val autoPlay: Boolean = false, val autoPlay: Boolean = false,
val continuousPlayback: Boolean = true,
val minPlaybackSpeed: Float = 0.5f, val minPlaybackSpeed: Float = 0.5f,
val maxPlaybackSpeed: Float = 2.0f, val maxPlaybackSpeed: Float = 2.0f,
val gestureControls: GestureControlsConfig = GestureControlsConfig(), val gestureControls: GestureControlsConfig = GestureControlsConfig(),

View File

@@ -65,7 +65,8 @@ class RemoteImitationQualityAssessor internal constructor(
require(SHA256_PATTERN.matches(videoHash)) { require(SHA256_PATTERN.matches(videoHash)) {
"ImitationAssessmentRequest.mediaId or metadata[video_hash] must be a SHA-256 hash." "ImitationAssessmentRequest.mediaId or metadata[video_hash] must be a SHA-256 hash."
} }
val endpoint = baseUri.buildUpon() return HttpFallback.forConfig(config).execute(config.sentenceBoundaryApiBaseUrl) { effectiveBaseUrl ->
val endpoint = Uri.parse(effectiveBaseUrl).buildUpon()
.appendPath("api") .appendPath("api")
.appendPath("v1") .appendPath("v1")
.appendPath("videos") .appendPath("videos")
@@ -76,7 +77,7 @@ class RemoteImitationQualityAssessor internal constructor(
.build() .build()
val connection = URL(endpoint.toString()).openConnection() as HttpURLConnection val connection = URL(endpoint.toString()).openConnection() as HttpURLConnection
val boundary = "----OralTrainer-${UUID.randomUUID()}" val boundary = "----OralTrainer-${UUID.randomUUID()}"
return try { try {
connection.requestMethod = "POST" connection.requestMethod = "POST"
connection.doOutput = true connection.doOutput = true
connection.connectTimeout = config.connectTimeoutMs connection.connectTimeout = config.connectTimeoutMs
@@ -104,6 +105,7 @@ class RemoteImitationQualityAssessor internal constructor(
connection.disconnect() connection.disconnect()
} }
} }
}
private fun writeFormField(output: BufferedOutputStream, boundary: String, name: String, value: String) { private fun writeFormField(output: BufferedOutputStream, boundary: String, name: String, value: String) {
output.write("--$boundary\r\n".toByteArray(Charsets.UTF_8)) output.write("--$boundary\r\n".toByteArray(Charsets.UTF_8))

View File

@@ -98,7 +98,8 @@ class SentenceBoundaryApi internal constructor(
} }
private fun fetchBlocking(videoHash: String): SentenceBoundaryApiResult { private fun fetchBlocking(videoHash: String): SentenceBoundaryApiResult {
val endpoint = baseUri.buildUpon() return HttpFallback.forConfig(config).execute(config.sentenceBoundaryApiBaseUrl) { effectiveBaseUrl ->
val endpoint = Uri.parse(effectiveBaseUrl).buildUpon()
.appendPath("api") .appendPath("api")
.appendPath("v1") .appendPath("v1")
.appendPath("videos") .appendPath("videos")
@@ -106,7 +107,7 @@ class SentenceBoundaryApi internal constructor(
.appendPath("sentence-boundaries") .appendPath("sentence-boundaries")
.build() .build()
val connection = URL(endpoint.toString()).openConnection() as HttpURLConnection val connection = URL(endpoint.toString()).openConnection() as HttpURLConnection
return try { try {
connection.requestMethod = "GET" connection.requestMethod = "GET"
connection.connectTimeout = config.connectTimeoutMs connection.connectTimeout = config.connectTimeoutMs
connection.readTimeout = config.readTimeoutMs connection.readTimeout = config.readTimeoutMs
@@ -127,6 +128,7 @@ class SentenceBoundaryApi internal constructor(
connection.disconnect() connection.disconnect()
} }
} }
}
private fun parseResponse( private fun parseResponse(
body: String, body: String,

View File

@@ -44,7 +44,6 @@ class VideoCatalogApi internal constructor(
) { ) {
private val mainHandler = Handler(Looper.getMainLooper()) private val mainHandler = Handler(Looper.getMainLooper())
private val executor: ExecutorService = Executors.newCachedThreadPool() private val executor: ExecutorService = Executors.newCachedThreadPool()
private val baseUrl = config.sentenceBoundaryApiBaseUrl.trimEnd('/') + "/"
fun fetch(callback: VideoCatalogCallback): CancellableRequest { fun fetch(callback: VideoCatalogCallback): CancellableRequest {
val future = executor.submit { val future = executor.submit {
@@ -67,9 +66,10 @@ class VideoCatalogApi internal constructor(
} }
private fun fetchBlocking(): List<TrainingVideoSummary> { private fun fetchBlocking(): List<TrainingVideoSummary> {
val endpoint = URL(URL(baseUrl), "api/v1/videos") return HttpFallback.forConfig(config).execute(config.sentenceBoundaryApiBaseUrl) { effectiveBaseUrl ->
val endpoint = URL(URL(effectiveBaseUrl), "api/v1/videos")
val connection = endpoint.openConnection() as HttpURLConnection val connection = endpoint.openConnection() as HttpURLConnection
return try { try {
connection.requestMethod = "GET" connection.requestMethod = "GET"
connection.connectTimeout = config.connectTimeoutMs connection.connectTimeout = config.connectTimeoutMs
connection.readTimeout = config.readTimeoutMs connection.readTimeout = config.readTimeoutMs
@@ -79,25 +79,32 @@ class VideoCatalogApi internal constructor(
val stream = if (statusCode in 200..299) connection.inputStream else connection.errorStream val stream = if (statusCode in 200..299) connection.inputStream else connection.errorStream
val body = stream?.bufferedReader(Charsets.UTF_8)?.use { it.readText() }.orEmpty() val body = stream?.bufferedReader(Charsets.UTF_8)?.use { it.readText() }.orEmpty()
if (statusCode !in 200..299) throw VideoCatalogApiException(statusCode, body) if (statusCode !in 200..299) throw VideoCatalogApiException(statusCode, body)
parseResponse(body) parseResponse(body, effectiveBaseUrl)
} finally { } finally {
connection.disconnect() connection.disconnect()
} }
} }
}
private fun parseResponse(body: String): List<TrainingVideoSummary> { private fun parseResponse(body: String, effectiveBaseUrl: String): List<TrainingVideoSummary> {
val videos = JSONObject(body).getJSONArray("videos") val videos = JSONObject(body).getJSONArray("videos")
val effectiveScheme = Uri.parse(effectiveBaseUrl).scheme
return buildList(videos.length()) { return buildList(videos.length()) {
for (index in 0 until videos.length()) { for (index in 0 until videos.length()) {
val item = videos.getJSONObject(index) val item = videos.getJSONObject(index)
val status = item.getString("status") val status = item.getString("status")
if (status != "ready") continue if (status != "ready") continue
val streamUrl = URL(URL(baseUrl), item.getString("stream_url")) val resolvedStreamUrl = URL(URL(effectiveBaseUrl), item.getString("stream_url")).toString()
val streamUri = if (effectiveScheme == "http") {
Uri.parse(resolvedStreamUrl).buildUpon().scheme("http").build().toString()
} else {
resolvedStreamUrl
}
add( add(
TrainingVideoSummary( TrainingVideoSummary(
videoHash = item.getString("video_hash"), videoHash = item.getString("video_hash"),
title = item.getString("title"), title = item.getString("title"),
streamUri = Uri.parse(streamUrl.toString()), streamUri = Uri.parse(streamUri),
durationMs = if (item.isNull("duration_ms")) null else item.getLong("duration_ms"), durationMs = if (item.isNull("duration_ms")) null else item.getLong("duration_ms"),
sizeBytes = item.getLong("size_bytes"), sizeBytes = item.getLong("size_bytes"),
language = if (item.isNull("language")) null else item.getString("language"), language = if (item.isNull("language")) null else item.getString("language"),

View File

@@ -1,9 +1,11 @@
<manifest xmlns:android="http://schemas.android.com/apk/res/android"> <manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<application <application
android:allowBackup="true" android:allowBackup="true"
android:label="@string/app_name" android:label="@string/app_name"
android:networkSecurityConfig="@xml/network_security_config"
android:supportsRtl="true" android:supportsRtl="true"
android:theme="@style/AppTheme"> android:theme="@style/AppTheme">
<activity <activity

View File

@@ -1,12 +1,18 @@
package cn.learningpad.oraltrainer.sample package cn.learningpad.oraltrainer.sample
import android.Manifest
import android.app.Activity import android.app.Activity
import android.content.Intent import android.content.Intent
import android.content.pm.ActivityInfo
import android.content.pm.PackageManager
import android.content.res.ColorStateList import android.content.res.ColorStateList
import android.content.res.Configuration
import android.graphics.Color import android.graphics.Color
import android.graphics.Typeface import android.graphics.Typeface
import android.graphics.drawable.GradientDrawable import android.graphics.drawable.GradientDrawable
import android.media.MediaRecorder
import android.net.Uri import android.net.Uri
import android.os.Build
import android.os.Bundle import android.os.Bundle
import android.view.Gravity import android.view.Gravity
import android.view.View import android.view.View
@@ -14,17 +20,22 @@ import android.view.ViewGroup
import android.widget.Button import android.widget.Button
import android.widget.FrameLayout import android.widget.FrameLayout
import android.widget.HorizontalScrollView import android.widget.HorizontalScrollView
import android.widget.ImageButton
import android.widget.ImageView
import android.widget.LinearLayout import android.widget.LinearLayout
import android.widget.ProgressBar import android.widget.SeekBar
import android.widget.Switch
import android.widget.TextView import android.widget.TextView
import androidx.media3.common.MimeTypes
import cn.learningpad.oraltrainer.sdk.GestureEvent import cn.learningpad.oraltrainer.sdk.GestureEvent
import cn.learningpad.oraltrainer.sdk.GestureKind import cn.learningpad.oraltrainer.sdk.GestureKind
import cn.learningpad.oraltrainer.sdk.ImitationAssessmentCallback
import cn.learningpad.oraltrainer.sdk.ImitationAssessmentResult
import cn.learningpad.oraltrainer.sdk.LoopMode import cn.learningpad.oraltrainer.sdk.LoopMode
import cn.learningpad.oraltrainer.sdk.OralTrainerController import cn.learningpad.oraltrainer.sdk.OralTrainerController
import cn.learningpad.oraltrainer.sdk.OralTrainerListener import cn.learningpad.oraltrainer.sdk.OralTrainerListener
import cn.learningpad.oraltrainer.sdk.OralTrainerPlayerView import cn.learningpad.oraltrainer.sdk.OralTrainerPlayerView
import cn.learningpad.oraltrainer.sdk.OralTrainerSdk import cn.learningpad.oraltrainer.sdk.OralTrainerSdk
import cn.learningpad.oraltrainer.sdk.OralTrainerSdkConfig
import cn.learningpad.oraltrainer.sdk.PlaybackSnapshot import cn.learningpad.oraltrainer.sdk.PlaybackSnapshot
import cn.learningpad.oraltrainer.sdk.PlayerConfig import cn.learningpad.oraltrainer.sdk.PlayerConfig
import cn.learningpad.oraltrainer.sdk.SentenceBoundary import cn.learningpad.oraltrainer.sdk.SentenceBoundary
@@ -33,11 +44,13 @@ import cn.learningpad.oraltrainer.sdk.SentenceBoundaryApiResult
import cn.learningpad.oraltrainer.sdk.TrainingMediaItem import cn.learningpad.oraltrainer.sdk.TrainingMediaItem
import cn.learningpad.oraltrainer.sdk.TrainingVideoSummary import cn.learningpad.oraltrainer.sdk.TrainingVideoSummary
import cn.learningpad.oraltrainer.sdk.VideoCatalogCallback import cn.learningpad.oraltrainer.sdk.VideoCatalogCallback
import java.io.File
import java.util.Locale import java.util.Locale
class MainActivity : Activity() { class MainActivity : Activity() {
private lateinit var sdk: OralTrainerSdk private lateinit var sdk: OralTrainerSdk
private lateinit var controller: OralTrainerController private lateinit var controller: OralTrainerController
private lateinit var controllerListener: OralTrainerListener
private lateinit var playerView: OralTrainerPlayerView private lateinit var playerView: OralTrainerPlayerView
private lateinit var lessonTitleText: TextView private lateinit var lessonTitleText: TextView
private lateinit var statusText: TextView private lateinit var statusText: TextView
@@ -45,32 +58,78 @@ class MainActivity : Activity() {
private lateinit var sentenceMetaText: TextView private lateinit var sentenceMetaText: TextView
private lateinit var timeText: TextView private lateinit var timeText: TextView
private lateinit var speedText: TextView private lateinit var speedText: TextView
private lateinit var progressBar: ProgressBar private lateinit var progressBar: SeekBar
private var progressBarDragging = false
private lateinit var catalogList: LinearLayout private lateinit var catalogList: LinearLayout
private lateinit var catalogStatusText: TextView private lateinit var catalogStatusText: TextView
private lateinit var testStatusText: TextView
private lateinit var scoreSummaryText: TextView
private lateinit var scoreDetailText: TextView
private lateinit var recordButton: Button
private var mediaRecorder: MediaRecorder? = null
private var recordingFile: File? = null
private var activeItemId: String = SAMPLE_ID private var activeItemId: String = ""
private var currentSentenceCount = 0 private var currentSentenceCount = 0
private var catalogVideos: List<TrainingVideoSummary> = emptyList() private var catalogVideos: List<TrainingVideoSummary> = emptyList()
private var catalogStatusTextValue = "正在同步"
private var activeModule = Module.TRAIN
private var continuousPlaybackEnabled = false
override fun onCreate(savedInstanceState: Bundle?) { override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
configureWindow() configureWindow()
sdk = OralTrainerSdk.init(this) sdk = OralTrainerSdk.init(
this,
OralTrainerSdkConfig(
sentenceBoundaryApiBaseUrl = SERVER_BASE_URL,
assessmentApiKey = ASSESSMENT_API_KEY,
allowHttpFallback = true,
)
)
controller = sdk.createController( controller = sdk.createController(
playerConfig = PlayerConfig( playerConfig = PlayerConfig(
sentenceMode = true, sentenceMode = true,
defaultSeekStepMs = 10_000L, defaultSeekStepMs = 10_000L,
autoPlay = false, autoPlay = false,
) continuousPlayback = false,
),
imitationAssessor = sdk.createRemoteImitationQualityAssessor(),
) )
controller.setLoopMode(LoopMode.ALL) controller.setLoopMode(LoopMode.ALL)
controllerListener = createControllerListener()
controller.addListener(controllerListener)
setContentView(createContentView()) setContentView(createContentView())
bindPlayerEvents()
loadSampleLesson()
loadCatalog() loadCatalog()
refreshCurrentUi()
}
override fun onConfigurationChanged(newConfig: Configuration) {
super.onConfigurationChanged(newConfig)
setContentView(createContentView())
renderCatalog()
refreshCurrentUi()
if (activeModule == Module.TEST) {
refreshTestUi()
}
}
override fun onRequestPermissionsResult(
requestCode: Int,
permissions: Array<out String>,
grantResults: IntArray,
) {
super.onRequestPermissionsResult(requestCode, permissions, grantResults)
if (requestCode != RECORD_AUDIO_REQUEST) {
return
}
if (grantResults.firstOrNull() == PackageManager.PERMISSION_GRANTED) {
startRecording()
} else if (::testStatusText.isInitialized) {
testStatusText.text = "需要麦克风权限才能进行录音评测"
}
} }
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) { override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
@@ -122,6 +181,7 @@ class MainActivity : Activity() {
} }
override fun onDestroy() { override fun onDestroy() {
discardRecording()
controller.release() controller.release()
super.onDestroy() super.onDestroy()
} }
@@ -133,15 +193,119 @@ class MainActivity : Activity() {
} }
private fun createContentView(): View { private fun createContentView(): View {
val isLandscape = resources.configuration.orientation == Configuration.ORIENTATION_LANDSCAPE
return if (isLandscape) {
createLandscapeContentView()
} else {
createPortraitContentView()
}
}
private fun createPortraitContentView(): View {
return LinearLayout(this).apply { return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL orientation = LinearLayout.VERTICAL
setBackgroundColor(COLOR_BACKGROUND) setBackgroundColor(COLOR_BACKGROUND)
addView(createModuleTabBar())
if (activeModule == Module.TEST) {
addView(createTestContentView())
} else {
addView(createHeader()) addView(createHeader())
addView(createPlayerSection()) addView(createPlayerSection())
addView(createSentenceSection()) addView(createSentenceSection())
addView(createCatalogSection()) addView(createCatalogSection())
} }
} }
}
private fun createTestContentView(): View {
return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
setBackgroundColor(COLOR_BACKGROUND)
addView(createPlayerSection())
addView(createSentenceSection())
addView(createTestSection())
}
}
private fun createModuleTabBar(): View {
return LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL
setPadding(16.dp, 40.dp, 16.dp, 0)
addView(moduleTab("训练", Module.TRAIN))
addView(moduleTab("测试", Module.TEST))
}
}
private fun moduleTab(label: String, module: Module): TextView {
val selected = activeModule == module
return TextView(this).apply {
text = label
textSize = 15f
typeface = Typeface.DEFAULT_BOLD
gravity = Gravity.CENTER
isClickable = true
setTextColor(if (selected) Color.WHITE else COLOR_TEXT_MUTED)
background = if (selected) {
rounded(COLOR_BUTTON, 8f)
} else {
rounded(COLOR_SURFACE, 8f, COLOR_BORDER)
}
layoutParams = LinearLayout.LayoutParams(0, 44.dp, 1f).withMargins(0, 0, 8.dp, 0)
setOnClickListener {
if (activeModule == module) {
return@setOnClickListener
}
if (activeModule == Module.TEST) {
discardRecording()
}
activeModule = module
controller.setContinuousPlayback(
if (activeModule == Module.TRAIN) continuousPlaybackEnabled else true
)
setContentView(createContentView())
renderCatalog()
refreshCurrentUi()
if (activeModule == Module.TEST) {
refreshTestUi()
}
}
}
}
private fun createLandscapeContentView(): View {
return FrameLayout(this).apply {
setBackgroundColor(Color.BLACK)
addView(createPlayerView())
addView(createPlayerOverlay())
addView(
LinearLayout(this@MainActivity).apply {
orientation = LinearLayout.VERTICAL
layoutParams = FrameLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.WRAP_CONTENT,
Gravity.BOTTOM,
)
addView(createLandscapeSentencePanel().apply {
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.WRAP_CONTENT,
)
})
addView(createRotationButtonRow())
}
)
}
}
private fun createLandscapeSentencePanel(): View {
return createSentenceContent(translucent = true).apply {
layoutParams = FrameLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.WRAP_CONTENT,
Gravity.BOTTOM,
)
}
}
private fun createHeader(): View { private fun createHeader(): View {
val titleBlock = LinearLayout(this).apply { val titleBlock = LinearLayout(this).apply {
@@ -149,16 +313,16 @@ class MainActivity : Activity() {
gravity = Gravity.CENTER_VERTICAL gravity = Gravity.CENTER_VERTICAL
layoutParams = LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f) layoutParams = LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f)
addView(TextView(this@MainActivity).apply { addView(TextView(this@MainActivity).apply {
text = "口语宝" text = "跟读虫"
setTextColor(Color.WHITE) setTextColor(Color.WHITE)
textSize = 26f textSize = 32f
typeface = Typeface.DEFAULT_BOLD typeface = Typeface.DEFAULT_BOLD
includeFontPadding = false includeFontPadding = false
}) })
addView(TextView(this@MainActivity).apply { addView(TextView(this@MainActivity).apply {
text = "中英跟读训练" text = "外语跟读训练神器"
setTextColor(COLOR_TEXT_MUTED) setTextColor(COLOR_TEXT_MUTED)
textSize = 13f textSize = 20f
setPadding(1.dp, 4.dp, 0, 0) setPadding(1.dp, 4.dp, 0, 0)
}) })
} }
@@ -174,14 +338,40 @@ class MainActivity : Activity() {
} }
private fun createPlayerSection(): View { private fun createPlayerSection(): View {
playerView = OralTrainerPlayerView(this).apply { return FrameLayout(this).apply {
background = rounded(COLOR_SURFACE, 8f, COLOR_BORDER)
clipToOutline = true
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
0,
1f,
).withMargins(16.dp, 0, 16.dp, 10.dp)
addView(createPlayerView())
addView(createPlayerOverlay())
addView(createRotationButton().apply {
layoutParams = FrameLayout.LayoutParams(
44.dp,
44.dp,
Gravity.BOTTOM or Gravity.END,
).apply {
marginEnd = 12.dp
bottomMargin = 12.dp
}
})
}
}
private fun createPlayerView(): OralTrainerPlayerView {
return OralTrainerPlayerView(this).apply {
bind(controller) bind(controller)
layoutParams = FrameLayout.LayoutParams( layoutParams = FrameLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT,
) )
}.also { playerView = it }
} }
private fun createPlayerOverlay(): View {
val overlay = LinearLayout(this).apply { val overlay = LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL orientation = LinearLayout.HORIZONTAL
gravity = Gravity.CENTER_VERTICAL gravity = Gravity.CENTER_VERTICAL
@@ -192,71 +382,123 @@ class MainActivity : Activity() {
Gravity.TOP, Gravity.TOP,
) )
} }
timeText = overlayPill("--:-- / --:--").apply { timeText = overlayPill("--:-- / --:--").apply {
layoutParams = LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f) layoutParams = LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f)
} }
speedText = overlayPill("1x") speedText = overlayPill("1x")
overlay.addView(timeText) overlay.addView(timeText)
overlay.addView(speedText) overlay.addView(speedText)
return overlay
}
return FrameLayout(this).apply { private fun createRotationButtonRow(): View {
background = rounded(COLOR_SURFACE, 8f, COLOR_BORDER) return LinearLayout(this).apply {
clipToOutline = true orientation = LinearLayout.HORIZONTAL
layoutParams = LinearLayout.LayoutParams( gravity = Gravity.END
ViewGroup.LayoutParams.MATCH_PARENT, setPadding(0, 0, 12.dp, 12.dp)
0, addView(createRotationButton())
1f, }
).withMargins(16.dp, 0, 16.dp, 10.dp) }
addView(playerView)
addView(overlay) private fun createRotationButton(): ImageButton {
val isLandscape = resources.configuration.orientation == Configuration.ORIENTATION_LANDSCAPE
return ImageButton(this).apply {
setImageResource(R.drawable.ic_screen_rotation)
imageTintList = ColorStateList.valueOf(Color.WHITE)
background = rounded(Color.argb(178, 9, 12, 16), 22.dp.toFloat())
scaleType = ImageView.ScaleType.CENTER_INSIDE
setPadding(10.dp, 10.dp, 10.dp, 10.dp)
contentDescription = if (isLandscape) "竖屏" else "横屏"
layoutParams = LinearLayout.LayoutParams(44.dp, 44.dp)
setOnClickListener { toggleOrientation() }
}
}
private fun toggleOrientation() {
val isLandscape = resources.configuration.orientation == Configuration.ORIENTATION_LANDSCAPE
requestedOrientation = if (isLandscape) {
ActivityInfo.SCREEN_ORIENTATION_PORTRAIT
} else {
ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE
} }
} }
private fun createSentenceSection(): View { private fun createSentenceSection(): View {
lessonTitleText = TextView(this).apply { return createSentenceContent(translucent = false).apply {
setTextColor(COLOR_TEXT_DARK)
textSize = 17f
typeface = Typeface.DEFAULT_BOLD
maxLines = 1
}
sentenceMetaText = TextView(this).apply {
setTextColor(COLOR_ACCENT_DEEP)
textSize = 13f
setPadding(0, 8.dp, 0, 0)
}
sentenceText = TextView(this).apply {
setTextColor(COLOR_TEXT_DARK)
textSize = 22f
typeface = Typeface.DEFAULT_BOLD
setLineSpacing(2.dp.toFloat(), 1.05f)
setPadding(0, 10.dp, 0, 10.dp)
}
progressBar = ProgressBar(this, null, android.R.attr.progressBarStyleHorizontal).apply {
max = PROGRESS_MAX
progress = 0
progressTintList = ColorStateList.valueOf(COLOR_ACCENT)
progressBackgroundTintList = ColorStateList.valueOf(COLOR_PROGRESS_TRACK)
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
6.dp,
)
}
statusText = TextView(this).apply {
setTextColor(COLOR_TEXT_SUBTLE)
textSize = 13f
setPadding(0, 10.dp, 0, 0)
}
return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
background = rounded(Color.WHITE, 8f, COLOR_LIGHT_BORDER) background = rounded(Color.WHITE, 8f, COLOR_LIGHT_BORDER)
setPadding(18.dp, 16.dp, 18.dp, 16.dp) setPadding(18.dp, 16.dp, 18.dp, 16.dp)
layoutParams = LinearLayout.LayoutParams( layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT,
).withMargins(16.dp, 0, 16.dp, 12.dp) ).withMargins(16.dp, 0, 16.dp, 12.dp)
}
}
private fun createSentenceContent(translucent: Boolean): LinearLayout {
lessonTitleText = TextView(this).apply {
setTextColor(if (translucent) Color.WHITE else COLOR_TEXT_DARK)
textSize = 17f
typeface = Typeface.DEFAULT_BOLD
maxLines = 1
}
sentenceMetaText = TextView(this).apply {
setTextColor(if (translucent) COLOR_ACCENT_LIGHT else COLOR_ACCENT_DEEP)
textSize = 13f
setPadding(0, 8.dp, 0, 0)
}
sentenceText = TextView(this).apply {
setTextColor(if (translucent) Color.WHITE else COLOR_TEXT_DARK)
textSize = 22f
typeface = Typeface.DEFAULT_BOLD
setLineSpacing(2.dp.toFloat(), 1.05f)
setPadding(0, 10.dp, 0, 10.dp)
}
progressBar = SeekBar(this).apply {
max = PROGRESS_MAX
progress = 0
progressTintList = ColorStateList.valueOf(COLOR_ACCENT)
progressBackgroundTintList = ColorStateList.valueOf(
if (translucent) COLOR_PROGRESS_TRACK_DARK else COLOR_PROGRESS_TRACK
)
thumbTintList = ColorStateList.valueOf(COLOR_ACCENT)
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
28.dp,
)
setOnSeekBarChangeListener(object : SeekBar.OnSeekBarChangeListener {
override fun onProgressChanged(seekBar: SeekBar, progress: Int, fromUser: Boolean) {
if (fromUser) {
val duration = controller.snapshot().durationMs
timeText.text = "${formatTime(seekPositionForProgress(progress))} / ${formatTime(duration)}"
}
}
override fun onStartTrackingTouch(seekBar: SeekBar) {
progressBarDragging = true
}
override fun onStopTrackingTouch(seekBar: SeekBar) {
progressBarDragging = false
controller.seekTo(seekPositionForProgress(seekBar.progress))
}
})
}
statusText = TextView(this).apply {
setTextColor(if (translucent) COLOR_TEXT_OVERLAY else COLOR_TEXT_SUBTLE)
textSize = 13f
setPadding(0, 10.dp, 0, 0)
}
return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
if (translucent) {
background = rounded(COLOR_OVERLAY, 0f, null)
setPadding(18.dp, 14.dp, 18.dp, 16.dp)
}
addView(lessonTitleText) addView(lessonTitleText)
if (activeModule == Module.TRAIN) {
addView(createContinuousPlaybackRow(translucent))
}
addView(sentenceMetaText) addView(sentenceMetaText)
addView(sentenceText) addView(sentenceText)
addView(progressBar) addView(progressBar)
@@ -264,6 +506,36 @@ class MainActivity : Activity() {
} }
} }
private fun createContinuousPlaybackRow(translucent: Boolean): View {
return LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL
gravity = Gravity.CENTER_VERTICAL
setPadding(0, 6.dp, 0, 0)
addView(TextView(this@MainActivity).apply {
text = "连续播放"
setTextColor(if (translucent) COLOR_TEXT_OVERLAY else COLOR_TEXT_SUBTLE)
textSize = 13f
typeface = Typeface.DEFAULT_BOLD
layoutParams = LinearLayout.LayoutParams(0, ViewGroup.LayoutParams.WRAP_CONTENT, 1f)
})
addView(Switch(this@MainActivity).apply {
isChecked = continuousPlaybackEnabled
buttonTintList = ColorStateList.valueOf(
if (translucent) COLOR_ACCENT_LIGHT else COLOR_ACCENT
)
setOnCheckedChangeListener { _, checked ->
continuousPlaybackEnabled = checked
controller.setContinuousPlayback(checked)
statusText.text = if (checked) {
"已开启连续播放,将按句子顺序连续播放"
} else {
"已关闭连续播放,将循环播放当前句子"
}
}
})
}
}
private fun createCatalogSection(): View { private fun createCatalogSection(): View {
catalogStatusText = TextView(this).apply { catalogStatusText = TextView(this).apply {
setTextColor(COLOR_TEXT_MUTED) setTextColor(COLOR_TEXT_MUTED)
@@ -302,13 +574,243 @@ class MainActivity : Activity() {
} }
} }
private fun bindPlayerEvents() { private fun createTestSection(): View {
controller.addListener(object : OralTrainerListener { testStatusText = TextView(this).apply {
setTextColor(COLOR_ACCENT_DEEP)
textSize = 13f
setPadding(0, 0, 0, 8.dp)
}
scoreSummaryText = TextView(this).apply {
setTextColor(COLOR_TEXT_DARK)
textSize = 22f
typeface = Typeface.DEFAULT_BOLD
setPadding(0, 8.dp, 0, 0)
text = "尚未评测"
}
scoreDetailText = TextView(this).apply {
setTextColor(COLOR_TEXT_SUBTLE)
textSize = 13f
setLineSpacing(4.dp.toFloat(), 1f)
setPadding(0, 8.dp, 0, 0)
}
val sentenceButton = { label: String, action: () -> Unit ->
Button(this).apply {
text = label
isAllCaps = false
setTextColor(Color.WHITE)
textSize = 13f
typeface = Typeface.DEFAULT_BOLD
minWidth = 0
minHeight = 0
minimumWidth = 0
minimumHeight = 0
setPadding(0, 0, 0, 0)
background = rounded(COLOR_BUTTON, 8f)
layoutParams = LinearLayout.LayoutParams(0, 40.dp, 1f).withMargins(0, 0, 8.dp, 0)
setOnClickListener { action() }
}
}
recordButton = Button(this).apply {
text = "开始录音"
isAllCaps = false
setTextColor(Color.WHITE)
textSize = 14f
typeface = Typeface.DEFAULT_BOLD
minWidth = 0
minHeight = 0
minimumWidth = 0
minimumHeight = 0
background = rounded(COLOR_ACCENT, 8f)
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
44.dp,
)
setOnClickListener {
if (mediaRecorder == null) {
startRecording()
} else {
stopRecordingAndAssess()
}
}
}
val controlRow = LinearLayout(this).apply {
orientation = LinearLayout.HORIZONTAL
addView(sentenceButton("上一句") { controller.seekToPreviousSentence() })
addView(sentenceButton("播放原句") {
controller.playCurrentSentenceAndStop()
statusText.text = "正在播放当前句子,播放完自动停止"
})
addView(sentenceButton("下一句") { controller.seekToNextSentence() })
}
return LinearLayout(this).apply {
orientation = LinearLayout.VERTICAL
background = rounded(Color.WHITE, 8f, COLOR_LIGHT_BORDER)
setPadding(18.dp, 14.dp, 18.dp, 16.dp)
layoutParams = LinearLayout.LayoutParams(
ViewGroup.LayoutParams.MATCH_PARENT,
ViewGroup.LayoutParams.WRAP_CONTENT,
).withMargins(16.dp, 0, 16.dp, 12.dp)
addView(TextView(this@MainActivity).apply {
text = "朗读评测(测试)"
setTextColor(COLOR_TEXT_DARK)
textSize = 16f
typeface = Typeface.DEFAULT_BOLD
})
addView(testStatusText)
addView(controlRow)
addView(recordButton)
addView(scoreSummaryText)
addView(scoreDetailText)
}
}
private fun refreshTestUi() {
if (!::testStatusText.isInitialized) {
return
}
recordButton.text = if (mediaRecorder == null) "开始录音" else "停止并评测"
val item = controller.currentTrainingItem()
val sentence = controller.currentSentence()
testStatusText.text = if (item == null || sentence == null) {
"请先在训练模块选择一个云端课程"
} else {
"评测对象:第 ${sentence.index + 1} 句(共 ${item.sentences.size} 句)"
}
}
private fun startRecording() {
controller.pause()
if (!::testStatusText.isInitialized) {
return
}
if (checkSelfPermission(Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
requestPermissions(arrayOf(Manifest.permission.RECORD_AUDIO), RECORD_AUDIO_REQUEST)
return
}
val file = File(cacheDir, "attempt-${System.currentTimeMillis()}.m4a")
val recorder = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
MediaRecorder(this)
} else {
@Suppress("DEPRECATION")
MediaRecorder()
}
try {
recorder.setAudioSource(MediaRecorder.AudioSource.MIC)
recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
recorder.setAudioSamplingRate(16_000)
recorder.setAudioEncodingBitRate(96_000)
recorder.setOutputFile(file.absolutePath)
recorder.prepare()
recorder.start()
mediaRecorder = recorder
recordingFile = file
recordButton.text = "停止并评测"
testStatusText.text = "正在录音…读完当前句子后点击“停止并评测”"
} catch (error: Throwable) {
testStatusText.text = "录音启动失败:${error.message.orEmpty()}"
runCatching { recorder.release() }
}
}
private fun stopRecordingAndAssess() {
val recorder = mediaRecorder ?: return
val file = recordingFile ?: return
mediaRecorder = null
recordButton.text = "开始录音"
try {
recorder.stop()
} catch (error: Throwable) {
testStatusText.text = "录音太短或无法保存:${error.message.orEmpty()}"
file.delete()
recordingFile = null
return
} finally {
runCatching { recorder.release() }
}
recordingFile = null
submitAssessment(Uri.fromFile(file))
}
private fun discardRecording() {
val recorder = mediaRecorder ?: return
mediaRecorder = null
runCatching { recorder.stop() }
runCatching { recorder.release() }
recordingFile?.delete()
recordingFile = null
}
private fun submitAssessment(recordingUri: Uri) {
val item = controller.currentTrainingItem()
val sentence = controller.currentSentence()
if (item == null || sentence == null) {
testStatusText.text = "没有正在学习的句子,无法评测"
return
}
if (!SHA256_PATTERN.matches(item.id)) {
testStatusText.text = "仅云端课程支持朗读评测(本地视频请先上传到服务器)"
return
}
if (ASSESSMENT_API_KEY.isBlank() || ASSESSMENT_API_KEY.startsWith("replace-")) {
testStatusText.text = "未配置评分密钥:请在 MainActivity 填入服务器 .env 的 CLIENT_API_KEY"
return
}
testStatusText.text = "正在评测第 ${sentence.index + 1} 句,请稍候…"
scoreSummaryText.text = "评测中…"
scoreDetailText.text = ""
controller.assessCurrentSentence(
recordingUri = recordingUri,
locale = sentence.language
?.takeIf { it.isNotBlank() }
?: item.sentences.getOrNull(sentence.index)?.language,
metadata = mapOf("video_hash" to item.id),
callback = object : ImitationAssessmentCallback {
override fun onResult(result: ImitationAssessmentResult) {
if (::testStatusText.isInitialized) {
applyAssessmentResult(result)
}
}
override fun onError(error: Throwable) {
if (::testStatusText.isInitialized) {
testStatusText.text = "评测失败:${error.message.orEmpty()}"
scoreSummaryText.text = "评测失败"
}
}
},
)
}
private fun applyAssessmentResult(result: ImitationAssessmentResult) {
testStatusText.text = if (result.passed == true) "已通过" else "未通过"
scoreSummaryText.text = String.format(Locale.US, "总分 %.1f", result.overallScore)
scoreDetailText.text = buildString {
result.contentScore?.let { appendLine(String.format(Locale.US, "内容分 %.1f", it)) }
result.fluencyScore?.let { appendLine(String.format(Locale.US, "流畅度 %.1f", it)) }
result.durationScore?.let { appendLine(String.format(Locale.US, "时长分 %.1f", it)) }
result.pauseScore?.let { appendLine(String.format(Locale.US, "停顿分 %.1f", it)) }
result.speechRateScore?.let { appendLine(String.format(Locale.US, "语速分 %.1f", it)) }
result.referenceText?.let { appendLine("参考:$it") }
result.recognizedText?.let { appendLine("识别:$it") }
if (result.missingTokens.isNotEmpty()) {
appendLine("漏读:${result.missingTokens.joinToString("、")}")
}
if (result.extraTokens.isNotEmpty()) {
appendLine("多读:${result.extraTokens.joinToString("、")}")
}
result.feedback?.let { appendLine(it) }
}.trimEnd()
}
private fun createControllerListener(): OralTrainerListener {
return object : OralTrainerListener {
override fun onPlaybackSnapshot(snapshot: PlaybackSnapshot) { override fun onPlaybackSnapshot(snapshot: PlaybackSnapshot) {
timeText.text = "${formatTime(snapshot.positionMs)} / ${formatTime(snapshot.durationMs)}" applyPlaybackSnapshot(snapshot)
speedText.text = formatSpeed(snapshot.playbackSpeed)
statusText.text = playbackStatus(snapshot)
progressBar.progress = playbackProgress(snapshot)
} }
override fun onMediaChanged(item: TrainingMediaItem?) { override fun onMediaChanged(item: TrainingMediaItem?) {
@@ -317,18 +819,7 @@ class MainActivity : Activity() {
} }
override fun onSentenceChanged(sentence: SentenceBoundary?) { override fun onSentenceChanged(sentence: SentenceBoundary?) {
if (sentence == null) { applySentence(sentence)
sentenceMetaText.text = "暂无句子边界"
sentenceText.text = lessonTitleText.text
return
}
val countText = if (currentSentenceCount > 0) {
" / $currentSentenceCount"
} else {
""
}
sentenceMetaText.text = "${sentence.index + 1}$countText${formatTime(sentence.startMs)}-${formatTime(sentence.endMs)}"
sentenceText.text = sentence.text?.takeIf { it.isNotBlank() } ?: "当前句子"
} }
override fun onGesture(event: GestureEvent) { override fun onGesture(event: GestureEvent) {
@@ -343,33 +834,76 @@ class MainActivity : Activity() {
override fun onPlayerError(error: Throwable) { override fun onPlayerError(error: Throwable) {
statusText.text = "播放失败:${error.message.orEmpty()}" statusText.text = "播放失败:${error.message.orEmpty()}"
} }
}) }
} }
private fun loadSampleLesson() { private fun applyPlaybackSnapshot(snapshot: PlaybackSnapshot) {
val item = sampleOnlineLesson() if (!progressBarDragging) {
activeItemId = item.id timeText.text = "${formatTime(snapshot.positionMs)} / ${formatTime(snapshot.durationMs)}"
currentSentenceCount = item.sentences.size progressBar.progress = playbackProgress(snapshot)
controller.loadItem(item) }
lessonTitleText.text = item.title speedText.text = formatSpeed(snapshot.playbackSpeed)
renderCatalog() statusText.text = playbackStatus(snapshot)
}
private fun seekPositionForProgress(progress: Int): Long {
val duration = controller.snapshot().durationMs
if (duration <= 0L) {
return 0L
}
return progress.toLong() * duration / PROGRESS_MAX
}
private fun applySentence(sentence: SentenceBoundary?) {
if (sentence == null) {
sentenceMetaText.text = "暂无句子边界"
sentenceText.text = lessonTitleText.text
return
}
val countText = if (currentSentenceCount > 0) {
" / $currentSentenceCount"
} else {
""
}
sentenceMetaText.text = "${sentence.index + 1}$countText${formatTime(sentence.startMs)}-${formatTime(sentence.endMs)}"
sentenceText.text = sentence.text?.takeIf { it.isNotBlank() } ?: "当前句子"
}
private fun refreshCurrentUi() {
val item = controller.currentTrainingItem()
lessonTitleText.text = item?.title ?: "未选择课程"
currentSentenceCount = item?.sentences?.size ?: 0
applyPlaybackSnapshot(controller.snapshot())
applySentence(controller.currentSentence())
if (::catalogStatusText.isInitialized) {
catalogStatusText.text = catalogStatusTextValue
}
} }
private fun loadCatalog() { private fun loadCatalog() {
catalogStatusText.text = "正在同步" catalogStatusTextValue = "正在同步"
if (::catalogStatusText.isInitialized) {
catalogStatusText.text = catalogStatusTextValue
}
sdk.videoCatalogApi.fetch(object : VideoCatalogCallback { sdk.videoCatalogApi.fetch(object : VideoCatalogCallback {
override fun onSuccess(videos: List<TrainingVideoSummary>) { override fun onSuccess(videos: List<TrainingVideoSummary>) {
catalogVideos = videos catalogVideos = videos
catalogStatusText.text = if (videos.isEmpty()) { catalogStatusTextValue = if (videos.isEmpty()) {
"暂无云端课程" "暂无云端课程"
} else { } else {
"${videos.size} 个云端课程" "${videos.size} 个云端课程"
} }
if (::catalogStatusText.isInitialized) {
catalogStatusText.text = catalogStatusTextValue
}
renderCatalog() renderCatalog()
} }
override fun onError(error: Throwable) { override fun onError(error: Throwable) {
catalogStatusText.text = "云端暂不可用" catalogStatusTextValue = "云端暂不可用${error.message.orEmpty()}"
if (::catalogStatusText.isInitialized) {
catalogStatusText.text = catalogStatusTextValue
}
renderCatalog() renderCatalog()
} }
}) })
@@ -380,15 +914,6 @@ class MainActivity : Activity() {
return return
} }
catalogList.removeAllViews() catalogList.removeAllViews()
catalogList.addView(
videoCard(
id = SAMPLE_ID,
title = "示例课程",
meta = "4 句 · ${formatTime(16_000L)}",
) {
loadSampleLesson()
}
)
catalogVideos.forEach { video -> catalogVideos.forEach { video ->
catalogList.addView( catalogList.addView(
videoCard( videoCard(
@@ -516,22 +1041,6 @@ class MainActivity : Activity() {
) )
} }
private fun sampleOnlineLesson(): TrainingMediaItem {
return TrainingMediaItem(
id = SAMPLE_ID,
title = "口语宝示例课",
uri = Uri.parse("https://storage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"),
mimeType = MimeTypes.VIDEO_MP4,
customCacheKey = SAMPLE_ID,
sentences = listOf(
SentenceBoundary(0, 0L, 3_000L, "Good morning, everyone."),
SentenceBoundary(1, 3_000L, 7_000L, "Today we will practice listening carefully."),
SentenceBoundary(2, 7_000L, 11_000L, "Please repeat each sentence clearly."),
SentenceBoundary(3, 11_000L, 16_000L, "Small daily practice makes progress visible."),
),
)
}
private fun playbackStatus(snapshot: PlaybackSnapshot): String { private fun playbackStatus(snapshot: PlaybackSnapshot): String {
val state = when { val state = when {
snapshot.isPlaying -> "播放中" snapshot.isPlaying -> "播放中"
@@ -599,22 +1108,38 @@ class MainActivity : Activity() {
get() = (this * resources.displayMetrics.density).toInt() get() = (this * resources.displayMetrics.density).toInt()
private companion object { private companion object {
// 服务端地址:改成你实际部署的域名或 IP。
// 内网/开发环境可填 http://<服务器IP>:<端口>(例如 http://192.168.1.100:80
const val SERVER_BASE_URL = "https://videoservice.d1kt.cn"
// 朗读评测密钥:填服务器 sentence_api/.env 里的 CLIENT_API_KEY。
const val ASSESSMENT_API_KEY = "fcf60fa10bc1c49e5ddb93d570bc54df5648e187b64096962498d57661c14220"
const val PICK_VIDEO_REQUEST = 1001 const val PICK_VIDEO_REQUEST = 1001
const val SAMPLE_ID = "online_sample_01" const val RECORD_AUDIO_REQUEST = 1002
const val PROGRESS_MAX = 1000 const val PROGRESS_MAX = 1000
private val SHA256_PATTERN = Regex("^[0-9a-fA-F]{64}$")
enum class Module {
TRAIN,
TEST,
}
val COLOR_BACKGROUND: Int = Color.rgb(12, 15, 18) val COLOR_BACKGROUND: Int = Color.rgb(12, 15, 18)
val COLOR_SURFACE: Int = Color.rgb(28, 34, 40) val COLOR_SURFACE: Int = Color.rgb(28, 34, 40)
val COLOR_BORDER: Int = Color.rgb(50, 59, 67) val COLOR_BORDER: Int = Color.rgb(50, 59, 67)
val COLOR_BUTTON: Int = Color.rgb(37, 99, 235) val COLOR_BUTTON: Int = Color.rgb(37, 99, 235)
val COLOR_ACCENT: Int = Color.rgb(21, 184, 132) val COLOR_ACCENT: Int = Color.rgb(21, 184, 132)
val COLOR_ACCENT_DEEP: Int = Color.rgb(7, 118, 86) val COLOR_ACCENT_DEEP: Int = Color.rgb(7, 118, 86)
val COLOR_ACCENT_LIGHT: Int = Color.rgb(94, 234, 182)
val COLOR_SELECTED: Int = Color.rgb(229, 248, 240) val COLOR_SELECTED: Int = Color.rgb(229, 248, 240)
val COLOR_PROGRESS_TRACK: Int = Color.rgb(224, 231, 235) val COLOR_PROGRESS_TRACK: Int = Color.rgb(224, 231, 235)
val COLOR_PROGRESS_TRACK_DARK: Int = Color.rgb(55, 62, 70)
val COLOR_LIGHT_BORDER: Int = Color.rgb(218, 226, 232) val COLOR_LIGHT_BORDER: Int = Color.rgb(218, 226, 232)
val COLOR_OVERLAY: Int = Color.argb(178, 9, 12, 16) val COLOR_OVERLAY: Int = Color.argb(178, 9, 12, 16)
val COLOR_TEXT_DARK: Int = Color.rgb(18, 24, 31) val COLOR_TEXT_DARK: Int = Color.rgb(18, 24, 31)
val COLOR_TEXT_MUTED: Int = Color.rgb(151, 162, 174) val COLOR_TEXT_MUTED: Int = Color.rgb(151, 162, 174)
val COLOR_TEXT_SUBTLE: Int = Color.rgb(88, 98, 108) val COLOR_TEXT_SUBTLE: Int = Color.rgb(88, 98, 108)
val COLOR_TEXT_OVERLAY: Int = Color.rgb(163, 172, 182)
} }
} }

View File

@@ -0,0 +1,9 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="24dp"
android:height="24dp"
android:viewportWidth="24.0"
android:viewportHeight="24.0">
<path
android:fillColor="#FFFFFFFF"
android:pathData="M16.48,2.52c3,2.02 5.02,5.27 5.02,9.48 0,1.77 -0.38,3.45 -1.06,4.97L10.02,2.94C11.39,2.51 12.86,2.28 14.4,2.28c0.74,0 1.47,0.08 2.08,0.24zM10.02,21.06L1.56,7.43C0.56,8.82 0,10.36 0,12c0,4.97 4.03,9 9,9h0.51C9.73,21.06 9.87,21.06 10.02,21.06zM9,22.06C3.96,22.06 -0.03,18.08 -0.03,13.04c0,-1.6 0.39,-3.11 1.07,-4.44l8.57,13.12C9.63,21.72 9.6,21.88 9.6,22.06 9.6,22.06 9.6,22.06 9,22.06zM20.93,5.44c0.68,1.33 1.07,2.84 1.07,4.44 0,5.04 -3.99,9.02 -9.03,9.02 -0.2,0 -0.4,-0.01 -0.6,-0.02l8.56,-13.44L20.93,5.44z"/>
</vector>

View File

@@ -1,3 +1,3 @@
<resources> <resources>
<string name="app_name">口语宝</string> <string name="app_name">跟读虫</string>
</resources> </resources>

View File

@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
允许 HTTP 明文访问,便于直连内网/开发环境部署的服务器(如 http://192.168.x.x
同时信任系统证书和用户手动安装的证书(自签/私有 CA 时需先把证书装到手机)。
-->
<network-security-config>
<base-config cleartextTrafficPermitted="true">
<trust-anchors>
<certificates src="system" />
<certificates src="user" />
</trust-anchors>
</base-config>
</network-security-config>

View File

@@ -286,6 +286,84 @@ https://videoservice.d1kt.cn/admin
不要硬编码进公开代码仓库。SDK 会在评分请求中发送 `X-Client-Key`。公开视频列表和播放 不要硬编码进公开代码仓库。SDK 会在评分请求中发送 `X-Client-Key`。公开视频列表和播放
接口仍可交给 CDN 缓存GPU 评分接口则受到密钥保护。 接口仍可交给 CDN 缓存GPU 评分接口则受到密钥保护。
### 4A. 如果要在服务器本机终结 HTTPS可选
若总出口不做 TLS 终结,而是由服务器本机 nginx 提供 HTTPS443 的 `server` 块必须
包含与 80 块相同的上传相关指令,否则大文件上传会被掐断:
```nginx
server {
listen 443 ssl;
server_name videoservice.d1kt.cn;
ssl_certificate /etc/letsencrypt/live/videoservice.d1kt.cn/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/videoservice.d1kt.cn/privkey.pem;
client_max_body_size 12g;
client_body_timeout 7200s;
send_timeout 7200s;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_request_buffering off;
proxy_read_timeout 7200s;
proxy_send_timeout 7200s;
}
location ~ "^/api/v1/videos/[0-9a-fA-F]{64}/content$" {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Range $http_range;
proxy_set_header If-Range $http_if_range;
proxy_force_ranges on;
proxy_buffering off;
}
}
```
签发证书并应用:
```bash
sudo apt install -y certbot python3-certbot-nginx
sudo certbot --nginx -d videoservice.d1kt.cn
```
### 4B. 上传中断ClientDisconnect排查
后台日志出现 `starlette.requests.ClientDisconnect` 表示浏览器到后端之间某处的连接在
请求体传完之前被断开。先确认实际生效的 nginx 配置:
```bash
sudo nginx -T | grep -E "client_max_body_size|client_body_timeout|proxy_request_buffering"
```
最常见的原因是新加的 HTTPS `server` 块漏掉了上传指令nginx 默认
`client_max_body_size` 只有 1 MB超过会直接返回 413 并断开连接,后端表现为
`ClientDisconnect`。其他常见原因:
- 总出口(网关/CDN有更小的请求体大小或更短的上传超时需在网关侧放行。
- `client_body_timeout`(默认 60s过短网络慢时上传暂停过久会被掐断按上文设
为 7200s。
- 浏览器/客户端主动取消或网络中断。
绕过网关在本机回源验证(能通则问题在网关):
```bash
curl -X PUT -H 'Host: videoservice.d1kt.cn' -H "X-Admin-Key: $ADMIN_KEY" \
--data-binary @large.mp4 \
'http://127.0.0.1/api/v1/admin/videos/raw?filename=large.mp4' \
-o /dev/null -w '%{http_code}\n'
```
API 已对 `ClientDisconnect` 做优雅处理:连接中断时返回 400 并记录一条 WARNING
不再产生 500 堆栈日志。
## 5. 上传与处理流程 ## 5. 上传与处理流程
后台上传后,服务会: 后台上传后,服务会:

View File

@@ -22,6 +22,7 @@ from fastapi import (
from fastapi.responses import FileResponse, Response from fastapi.responses import FileResponse, Response
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from starlette.concurrency import run_in_threadpool from starlette.concurrency import run_in_threadpool
from starlette.requests import ClientDisconnect
from .assessment import AssessmentService from .assessment import AssessmentService
from .audio_metrics import AudioAnalysisError from .audio_metrics import AudioAnalysisError
@@ -96,6 +97,14 @@ def create_app(
if static_dir.is_dir(): if static_dir.is_dir():
application.mount("/static", StaticFiles(directory=static_dir), name="static") application.mount("/static", StaticFiles(directory=static_dir), name="static")
@application.exception_handler(ClientDisconnect)
async def handle_client_disconnect(request: Request, exc: ClientDisconnect) -> Response:
logger.warning(
"Upload interrupted: the client or an intermediate proxy closed the "
"connection before the request body completed."
)
return Response(status_code=400, content="Upload connection was interrupted.")
def require_admin(x_admin_key: Optional[str] = Header(default=None)) -> None: def require_admin(x_admin_key: Optional[str] = Header(default=None)) -> None:
expected = service_settings.admin_api_key expected = service_settings.admin_api_key
if expected and not hmac.compare_digest(x_admin_key or "", expected): if expected and not hmac.compare_digest(x_admin_key or "", expected):