Success Criterion 1.2.4 - Level AA
Captions (Live)
Ensure real-time captions are available for all live videos with audio. This allows people who need subtitles to directly access the spoken information.
Impact
People who are deaf or hard of hearing want to be able to understand what is being said during live videos.
Check
“Are real-time captions available for live videos?”
This can be tested visually, no assistive technologies are needed.
Solution
The challenge with live captions is both organizational and technical. A captioner must be present who can provide real-time captions for the video by using appropriate software.
Add live captions
- Android
- Jetpack Compose
- iOS
- SwiftUI
- Flutter
- React Native
- .NET MAUI
- Xamarin
Live captions - Android
On Android, we recommend using a library such as Media3, also known as ExoPlayer, to support live captions. Media3 is developed by Google and is an open-source alternative to Android's MediaPlayer for audio and video playback. Many code examples can be found in the Camera & Media dev center. You can use DefaultTrackSelector in combination with DefaultHttpDataSource.Factory to show subtitles.
val trackSelector = DefaultTrackSelector(baseContext).apply {
setParameters(buildUponParameters().setPreferredTextLanguage("nl"))
}
val player = ExoPlayer.Builder(context)
.setTrackSelector(trackSelector)
.build()
val dataSourceFactory = DefaultHttpDataSource.Factory()
.setUserAgent(Util.getUserAgent(context, "Appt"))
.setAllowCrossProtocolRedirects(true)
val mediaUri = Uri.parse("https://appt.org/live-video")
val mediaSource = HlsMediaSource.Factory(dataSourceFactory)
.createMediaSource(MediaItem.fromUri(mediaUri))
player.setMediaSource(mediaSource)
player.prepare()
player.playWhenReady = true
Live captions - Jetpack Compose
In Jetpack Compose, we recommend using a library such as Media3, also known as ExoPlayer, to support live captions. Media3 is developed by Google and is an open-source alternative to Android's MediaPlayer for audio and video playback. Many code examples can be found in the Camera & Media dev center. You can use DefaultTrackSelector in combination with DefaultHttpDataSource.Factory to show subtitles.
val context = LocalContext.current
var exoPlayer by remember { mutableStateOf<ExoPlayer?>(null) }
DisposableEffect(Unit) {
// Create a track selector with preferred text language
val trackSelector = DefaultTrackSelector(context).apply {
setParameters(buildUponParameters().setPreferredTextLanguage("nl"))
}
val player = ExoPlayer.Builder(context)
.setTrackSelector(trackSelector)
.build()
val dataSourceFactory = DefaultHttpDataSource.Factory()
.setUserAgent(Util.getUserAgent(context, "Appt"))
.setAllowCrossProtocolRedirects(true)
// Prepare the media source
val mediaUri = Uri.parse("https://appt.org/live-video")
val mediaSource = HlsMediaSource.Factory(dataSourceFactory)
.createMediaSource(MediaItem.fromUri(mediaUri))
player.setMediaSource(mediaSource)
player.prepare()
player.playWhenReady = true
exoPlayer = player
onDispose {
player.release()
exoPlayer = null
}
}
Live captions - iOS
On iOS, AVPlayer has built-in support for live video with captions. Users can automatically turn on captions via System Preferences. The easiest way to stream a live video is through AVPlayerViewController.
guard let url = URL(string: "https://appt.org/live-video") else {
return
}
let player = AVPlayer(url: url)
let playerViewController = AVPlayerViewController()
playerViewController.player = player
present(playerViewController, animated: true) {
player.play()
}
Live captions - SwiftUI
In SwiftUI, AVPlayer has built-in support for live video with captions. Users can automatically turn on captions via System Preferences. The easiest way to stream a live video is through VideoPlayer view.
guard let url = URL(string: "https://appt.org/live-video") else {
return
}
let player = AVPlayer(url: url)
VideoPlayer(player: player)
// for a player on fullscreen
.edgesIgnoringSafeArea(.all)
.onAppear {
player.play()
}
Live captions - Flutter
On Flutter, the video_player package does not have support for captions in live videos. Issue #50595 has been opened to request support.
Other packages, like better_player, do have support for using captions on live video. The better_player documentation contains detailed information on how to do so.
BetterPlayerController controller = BetterPlayerController(
const BetterPlayerConfiguration(
controlsConfiguration: BetterPlayerControlsConfiguration(
enableAudioTracks: true,
enableSubtitles: true,
),
),
betterPlayerDataSource: BetterPlayerDataSource.network(
'https://appt.org/live-video',
liveStream: true,
useAsmsSubtitles: true,
),
);
Widget build(BuildContext context) {
return BetterPlayer(controller: controller);
}
Live captions - React Native
React Native does not have any out-of-the box packages which enable you to provide real-time captions for live audio or video streams.
You can however embed a YouTube stream in your app and enable Live Stream Captioning.
import YoutubePlayer from "react-native-youtube-iframe";
<YoutubePlayer
height={300}
play={playing}
videoId="dQw4w9WgXcQ"
onChangeState={onStateChange}
/>
Live captions - .NET MAUI
In MAUI, you can use MediaElement to embed videos. Unfortunately, there is no built-in support for live captions. In this case, a custom control will be required to support it. You can consider the following options:
Use a
Custom Handlerto create a control from scratch.Use a
Platform Behaviorto extend theMediaElementcomponent.Create a native binding to expose any native third-party library that supports this feature via Native Library Interop for .NET MAUI.
Not available, contribute!
Live captions - Xamarin
On Xamarin, you can use MediaElement to embed videos. Unfortunately, there is no built-in support for live captions.
Not available, contribute!