DEVHIDE
Login Or Sign up

Playing a video with captions in Jupyter notebook

834 Views Asked by At

How to play a video with captions in Jupyter notebook?

With code snippets from these post, I've tried to play a video inside jupyter notebook:

from IPython.display import HTML

# Show video
compressed_path = 'team-rocket.video-compressed.mp4'
mp4 = open(compressed_path,'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()


HTML("""
<video width=400 controls>
      <source src="%s" type="video/mp4">
</video>
""" % data_url)

[out]:

enter image description here

When I tried to add a .vtt file as caption, the option appears

from IPython.display import HTML

# Show video
compressed_path = 'team-rocket.video-compressed.mp4'
mp4 = open(compressed_path,'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()


HTML("""
<video width=400 controls>
      <source src="%s" type="video/mp4">
      <track src="team-rocket.vtt" label="English" kind="captions" srclang="en" default >
</video>
""" % data_url)

[out]:

enter image description here

But the subtitles/captions are not present when playing the video. How to play a video with captions in Jupyter notebook?

The files used in the examples above are on:

python video jupyter-notebook html5-video webvtt
Original Q&A
2

There are 2 best solutions below

2
Mazhar On BEST ANSWER

Try this:

from IPython.display import HTML
from base64 import b64encode

video_path = 'team-rocket.video-compressed.mp4'
captions_path = 'team-rocket.vtt'

with open(video_path, 'rb') as f:
    video_data = f.read()
    video_base64 = b64encode(video_data).decode()

with open(captions_path, 'r') as f:
    captions_data = f.read()
    captions_base64 = b64encode(captions_data.encode('utf-8')).decode()

video_html = f"""
<video width="640" height="360" controls>
    <source src="data:video/mp4;base64,{video_base64}" type="video/mp4">
    <track src="data:text/vtt;base64,{captions_base64}" kind="captions" srclang="en" label="English" default>
</video>
"""

HTML(video_html)

For some reason, explicitly streaming in the video + captions/subtitle while specifying the encoding would bypass the security issues that @igrinis' answer pointed out.

  • "data:video/mp4;base64,{video_base64}"
  • "data:text/vtt;base64,{captions_base64}"
1
igrinis On

It is a combination of security issues. Modern browsers forbid using local data (local subtitles file in this case) and Colab prevents you rendering embedded videos through "insecure" http protocol. So in order to display a video with subtitles you will need to create an iframe and embed there a video that is served through https (both video and subtitles).

You can test this by creating a cell with the following content:

%%html
<iframe width="800" height="500" src="https://iandevlin.com/html5/webvtt-example.html"
frameborder="0" allow="encrypted-media;" allowfullscreen />

So as I see it you will need to create a self signed certificate, add it to your browser for it to become legit, run a TLS web server in a thread to not block your notebook, create a small HTML file with the <video> tag which (the HTML file) your web server will serve over HTTPS and its URL will be the source of the <iframe> in the notebook. Whoa, this is really complicated.

If we try to load an full HTML into an iframe that has the video + track assets stored non-locally, these works:

%%html
<iframe width="800" height="500" src="https://iandevlin.com/html5/webvtt-example.html"
frameborder="0" allow="encrypted-media;" allowfullscreen />

or

HTML("""<iframe width="800" height="500" src="https://iandevlin.com/html5/webvtt-example.html"
frameborder="0" allow="encrypted-media;" allowfullscreen /> """)

[out]:

enter image description here

But when you try to take the HTML from https://iandevlin.com/html5/webvtt-example.html and load it into the iframe through srcdoc as follows, it fails to load the captions/subtitles:

%%html
<iframe width="800" height="500" 
srcdoc='
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset=utf-8>
    <title>WebVTT Video Subtitles Example for Chrome</title>
    <meta name="keywords" content="webvtt,webvtt example,chrome,html5" />
    <meta name="description" content="Sample HTML5 code that shows how Chrome supports WebVTT subtitles" />
    <style>
        body { 
            padding:10px;
            font-size:80%;
            font-family:verdana, serif;
        }
        figure { margin:30px 0; }
    </style>
</head>
<body>
    <header>
        <h1>WebVTT Video Subtitles Example for Chrome</h1>
        <p><a href="http://www.iandevlin.com/blog/2012/06/html5/google-chrome-supports-webvtt-subtitles">Read the full article</a></p>
        <p>To view this in Google Chrome, you must first enable the <code>track</code> element:</p>
        <ol>
            <li>Go to the configuration tools in Chrome by typing <code>chrome:flags</code> in the address bar</li>
            <li>Search for "Enable the &lt;track&gt; element" and activate it</li>
            <li>Restart the browser</li>
        </ol>
    </header>
    <figure>
        <video controls preload="metadata" poster="http://html5multimedia.com/media/elephants-dream.title.jpg" width="422" height="253">
            <source src="http://html5multimedia.com/media/elephants-dream-medium.webm" type="video/webm">
            <track label="English subtitles" kind="subtitles" srclang="en" src="https://iandevlin.com/html5/english-subtitles.vtt" default>
        </video>
        <figcaption>
            <small><a href="http://www.elephantsdream.org">Elephant's Dream</a> &ndash; &copy; copyright 2006, Blender Foundation / Netherlands Media Art Institute</small>
        </figcaption>
    </figure>
</body>
</html>' />

And taking a closer look at the console following @atalay-k's comment, you will see:

Unsafe attempt to load URL https://iandevlin.com/html5/english-subtitles.vtt from frame with URL about:srcdoc. Domains, protocols and ports must match.

which looks like a pesky issue from Security error loading subtitles on HTML video

Related Questions in PYTHON

Related Questions in VIDEO

Related Questions in JUPYTER-NOTEBOOK

Related Questions in HTML5-VIDEO

Related Questions in WEBVTT

Trending Questions

Popular # Hahtags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json

Popular Questions

.

Copyright © 2021 Jogjafile Inc.