Navi is an app that adds live captions and real-time translations to your FaceTime calls.
Real-time subtitles for any video, audio, live stream on the web or PC (Google Chrome Live Caption)
The app uses SharePlay and built-in speech recognition to add subtitles and translations in 20 languages to your FaceTime calls. It’s an incredible use of SharePlay, which most of us see as a gimmicky way to watch dubbed movies with people in other places. You probably don’t need to fire your translator just yet, but an app that gets this right can be insanely useful.
“I don't get the audio from the FaceTime call,” Navi developer Jordi Bruin wrote on Twitter, “but I use SharePlay to share it with the participants in the call.”
SharePlay is a new feature in iOS 15 and macOS 12.1 that lets you share and sync things in FaceTime calls. Using the movie watching example above, any participant can pause or play the movie, for example, while you’re all chatting in the FaceTime call. The FaceTime video remains open in a small, floating picture-in-picture panel, and each participant runs the app locally on their device. The trick with SharePlay is to sync whatever’s happening in those local apps so that everyone is sharing the experience, whether it’s a movie, a Fitness+ workout, or a spreadsheet.
