Male describer: A MULTIMEDIA SLIDE PRESENTATION IS TITLED
HTML5 VIDEO ACCESSIBILITY AND THE WebVTT FILE FORMAT.
MARCH 23, 2011. SILVIA PFEIFFER.
IN AN AUDITORIUM, A WOMAN SPEAKS AT A PODIUM.
Black: So welcome all. Thank you for coming.
My name is Naomi Black,
and I'm a member of the Accessibility Engineering Group
here at Google.
And today, I'm very pleased to invite
our speaker, Dr. Silvia Pfeiffer.
Silvia is a member and an invited expert
on the W3C HTML Accessibility Task Force.
And she's also the author of
"The Definitive Guide to HTML5 Video."
So Silvia's here to talk to us--
come on up.
Silvia's here to talk to us today about WebVTT,
which is one of the standards for timed text
which is under consideration by the W3C.
Thank you, Silvia.
Pfeiffer: Thank you.
Describer: SILVIA TAKES THE PODIUM.
Pfeiffer: Thanks for inviting me to come and speak
about this important topic today.
We know there are a lot of discussions going on
about formats for captions,
and we want to standardize them for the web.
But standardizing it for the web
has a much larger impact these days
than just on web browsers,
just on the web itself.
It goes into many different devices.
So we're very interested and very keen
to give a broad coverage of available technology,
and this is what we're trying to do here today.
So I'll be talking mostly about WebVTT, the file format.
But I'll also be talking a little bit about
how to plug that into the web browser into HTML
so that in the future,
we have a very simple way of displaying captions
in web browsers on videos
without having to do much more than authoring a file
and giving a link to the web browser for that file.
So it will be very simple for people in the future
to create more captions.
Describer: SILVIA CUES A NEW SLIDE TITLED
"REQUIREMENTS OF A WEB TEXT FORMAT."
HEADINGS BELOW READ "CAPTIONS OR SUBTITLES"
"TEXT VIDEO DESCRIPTIONS" "NAVIGATION OR CHAPTERS"
AND "METADATA."
Pfeiffer: All right, let's dig right in.
As we were looking at requirements
of such a text format,
a web text format for video,
we looked at the different types of content
that can be time-aligned with video.
And captions and subtitles are the obvious ones,
but text video descriptions are also an important use.
These are for blind users
and can be read out by screen readers
in parallel to the playback of the video.
This may well not be the most usable way
of doing audio descriptions,
but it is a much easier way to publish
audio descriptions for blind users.
And, in fact, for a lot of blind users,
it may well be all they need,
because they already have their screen readers set up,
and it works really well for some people.
Further to that,
we're also talking about navigation or chapters,
which is also very important
for blind and, in fact, any user.
If you want to go through a video quickly
and find out what's in there,
you want to jump to what we now know as chapter markers.
We can call them navigation markers.
This can be also covered with the same kind of format.
And more generally, metadata.
This is something that archives
are particularly interested in,
to attach metadata to sections in the video.
It can also be done with such a time-aligned text format.
So what we have discussed for browsers
is a very simple format.
It's called WebVTT, Video Text Tracks.
WebVTT.
Describer: A NEW SLIDE TITLED "WebVTT OR VIDEO TEXT TRACKS."
A SAMPLE FILE IS HEADED "WEBVTT" WITH TWO ELEMENTS BELOW.
EACH ELEMENT CONTAINS THREE LINES OF INFORMATION:
CAPTION NUMBER,
TIME CODE START AND TIME CODE END,
AND CAPTION TEXT.
BELOW THE TEXT, A STILL PHOTO OF A MAN SEATED IN A WORKSHOP
IS CAPTIONED "I HEARD ABOUT THIS ARDUINO PROJECT,
AND I SAW IT ONLINE."
Pfeiffer: This is one of the very simple files
that we can think about.
Just a marker at the beginning of the file
that identifies the file format.
The captions or subtitles-- let's call them cues--
then have an individual identifier.
In this case, it's the number one and number two.
Could be any string, however.
And then we've got start times and end times
on each one of these cues,
and a piece of text in there.
It turns out in--
as we all know how captions are displayed on screen
in something like this
if it's automatically created by the browser.
Describer: THE CAPTION DISPLAYS AT THE BOTTOM OF THE SCREEN
IN WHITE LETTERING OVER A SHADED GRAY BOX.
Pfeiffer: That was the very simplest way of doing subtitles.
Now, we want to do more than just the simple captions.
In particular, if we want to achieve
all the functionality
of, for example, the CEA-608 captions,
then we need to do a bit more than just text.
We also want to have some formatting in there.
Describer: ANOTHER SLIDE, TITLED "WebVTT FORMATTED SUBTITLES,"
DISPLAYS THE SAME STILL IMAGE, WITH THE CAPTION IN GERMAN.
THE WORD "ARDUINO" IS WRITTEN IN ALL CAPS AND COLORED RED.
Pfeiffer: Here is an example on how to do bold.
I'll point to it.
There's a bold tag in here, so that will be bold text.
Here is some italic text.
Describer: IN THE SAMPLE FILE,
THE SECOND CAPTION CONTAINS THE WORD "WOW!"
SURROUNDED BY THE HTML TAG FOR BOLD,
THE LETTER "b" SURROUNDED BY ANGLED BRACKETS
BEFORE THE WORD "WOW,"
AND "/b" IN ANGLE BRACKETS AFTER THE WORD "WOW."
ANOTHER PHRASE IS ENCLOSED
IN THE TAG FOR ITALICS, "i" AND "/i".
Pfeiffer: And up here we've got a general way
to associate style or a class to a piece of text
and give it a meaning.
In this situation,
we've turned a piece of text into red text
and capitalized it.
Describer: THE WORD "ARDUINO"
IS PRECEDED BY THE TAG "c.red.caps"
WITH A "/c" TO CLOSE.
IN THE CAPTION, IT APPEARS RED AND CAPITALIZED.
Pfeiffer: Of course, if we're using this format
also for subtitles,
we need to be careful to cover internationalization issues.
WebVTT is very clear here.
It requires UTF-8 character encoding.
Describer: ANOTHER SLIDE, WebVTT FOR INTERNATIONALIZATION,
ABBREVIATED I18N.
A SAMPLE CAPTION FILE CONTAINS
TIME CODES FOR CAPTION START AND STOP
AND CAPTION TEXT IN ASIAN LANGUAGE CHARACTERS.
HEADINGS BELOW READ "UTF-8 CHARACTER ENCODING"
"RUBY TEXT"
AND "VERTICAL OR HORIZONTAL RENDERING AND ALIGNMENT."
Pfeiffer: It has a ruby tag,
which supports Asian languages in particular.
Describer: ANNOTATED ASIAN LANGUAGE CHARACTERS ARE SHOWN
BETWEEN AN OPEN AND CLOSE RUBY TAG,
WITH THE ANNOTATIONS BETWEEN "rt" TAGS
AND THE ANNOTATED TEXT BETWEEN "rb" TAGS.
TO THE RIGHT OF THE START AND STOP TIMES,
CODES READ "D:vertical" AND "A:start".
Pfeiffer: It also does vertical and horizontal
rendering of text.
Again, possibly one of the most important ones
are Asian languages,
and I think there are a few other languages
that are also rendered vertically.
And we need to make sure we get the alignment right.
Sometimes text is read from the right to the left,
so therefore it needs to be aligned on the right
rather than on the left.
Describer: ANOTHER SLIDE IS TITLED
"WebVTT CAPTION POSITIONING."
Pfeiffer: Now, positioning is another requirement
and, again, something that traditional captions,
TV captions, are able to do.
It's possible to position cues anywhere in WebVTT.
There are basically three important ways
to position text.
Describer: A STILL SHOT FROM "ANNIE HALL" FINDS DIANE KEATON
WEARING A DRESS SHIRT, NECKTIE, AND VEST.
CAPTIONS APPEAR IN THE CENTER OF THE VIDEO PANE
AT BOTH THE TOP AND BOTTOM OF THE SCREEN.
THE BOTTOM CAPTION TEXT WRAPS TO A SECOND LINE.
AFTER THE LINE BREAK, THE CAPTIONS ARE CENTER ALIGNED.
THE BOTTOM CAPTION READS
"YEAH, I SORT OF DABBLE AROUND, YOU KNOW."
THE TOP CAPTION: "I DABBLE? LISTEN TO ME. WHAT A JERK."
HEADINGS NEXT TO THE STILL READ "L, LINE POSITION,
T, TEXT POSITION A, ALIGNMENT."
Pfeiffer: There are line positions.
So the concept of display lines exists in WebVTT.
So the line position allows people
to directly address a specific line.
It can be done with a line number or a percentage.
Then we have the text position.
This means we're placing the text either on the left,
in the middle, or on the right.
No, hold on. That's the alignment, sorry.
Alignment is left, middle, and right.
And the text position is...
so when we have text like this,
it's in the middle, and it's centered.
So we can also do a centering,
and we can do a left alignment
and a right alignment.
But we can also move that whole text elsewhere.
So the text position is where we move the text
and the alignment is where we align it at.
Describer: A SAMPLE FILE FEATURES POSITION CODES
TO THE RIGHT OF THE TIME CODE MARKERS.
FOR THE CAPTION AT THE TOP, THE CODES READ
"A:middle" AND "L:10%".
FOR THE BOTTOM CAPTION, "A:middle" AND "L:60%".
A NEW SLIDE IS TITLED "WebVTT SPEAKER SEMANTICS."
TWO FRAMES OF VIDEO FROM THE ANIMATED PROGRAM "ARTHUR"
CAPTURE A MONKEY FROWNING AT A BUNNY.
THE FIRST FRAME IS CAPTIONED "AHEM..."
WITH THE CAPTION LEFT ALIGNED TO APPEAR BELOW THE MONKEY.
THE SECOND FRAME IS CAPTIONED "WHAT'S THE MATTER?"
AND IS RIGHT ALIGNED TO APPEAR BELOW THE BUNNY.
Pfeiffer: We also have speaker semantics
included into WebVTT,
which is interesting because it allows us
to put some semantic information into our markup.
Describer: IN THE SAMPLE FILE,
TAGS APPEAR BEFORE THE TEXT OF EACH CAPTION,
"v.Beatrix" IN THE FIRST CAPTION,
AND "v.Arthur" IN THE SECOND.
THE TAGS ARE NOT CLOSED.
Pfeiffer: Here, for example, we have two people speaking.
We know their position on the left and on the right.
And the speaker markup
can tell us where we want to position it
and can also, for example, help us always use
the same styling for the same speaker.
So, for example, we want to use the same font,
the same font color,
maybe a specific outline or something for a speaker.
We can define that
and then apply that always for that speaker.
Describer: ANOTHER SLIDE, WebVTT TEXT DESCRIPTIONS.
Pfeiffer: Now, so much for captions.
Now we move on to a little example
on text descriptions.
Here is one that I've used previously,
and we've got that as an example on the site.
Describer: A SAMPLE FILE CONTAINS THREE DESCRIPTIONS,
EACH WITH THREE LINES OF INFORMATION:
DESCRIPTION NUMBER, TIME CODE START AND STOP,
AND THE TEXT TO BE READ ALOUD BY A SCREEN READER.
THE SLIDE GIVES THREE SAMPLE DESCRIPTIONS.
SAMPLE ONE, "THE ORANGE OPEN MOVIE PROJECT PRESENTS."
SAMPLE TWO, "INTRODUCTORY TITLES ARE SHOWING ON THE BACKGROUND
"OF A WATER POOL WITH FISHES SWIMMING
AND MECHANICAL OBJECTS LYING ON A STONE FLOOR."
SAMPLE THREE, "TITLE: ELEPHANTS DREAM."
Pfeiffer: I'm not gonna go there;
I just want to mention it,
because we want to focus on captions today.
But what happens here is, we've got text that's aligned
with a start and end time as well.
And for a typical word rate of a screen reader,
it will fit into that space,
and it will be read back by the screen reader
during that time.
Describer: A NEW SLIDE, WebVTT CHAPTERS FOR NAVIGATION.
Pfeiffer: And here is the navigation example.
As I mentioned, WebVTT can also be used for navigation.
Here we have three chapters,
and we can directly jump from chapter to chapter.
There needs to be extra controls on videos to support this,
but this is something we're working towards.
Describer: A SAMPLE FILE CONTAINS THREE CHAPTERS,
EACH WITH THREE LINES OF INFORMATION:
CHAPTER NUMBER, TIME CODE START AND STOP, AND CHAPTER TITLE.
TEXT BELOW THE SAMPLE READS "NAVIGATION MARKERS."
A NEW SLIDE IS TITLED "TEXT TRACKS IN HTML5."
Pfeiffer: Now, of course, as I'm saying,
controls and input into web pages
and automatic rendering,
we need to know how we're going to do that.
And there is markup in HTML5
for associating captions and formats like this
with videos.
In this example, I've got all of the VTT files
that we've used before.
I've included them here.
And what we're using for it is called a track element,
and this track element is included
underneath the video element in the HTML5 markup.
It links through the VTT file.
And there is some description possible
for the type of file it is, so we have a label.
In this case, it's an English caption.
We have a kind, which gives us a means
to group all the tracks of the same type together.
Describer: A CODE SAMPLE LISTS A SOURCE VIDEO
AND A SERIES OF TRACKS WITH THE FOLLOWING LABELS:
ENGLISH CAPTIONS, GERMAN SUBTITLES, FRENCH SUBTITLES,
ENGLISH DESCRIPTIONS, AND CHAPTERS.
EACH TRACK HAS ATTRIBUTES "KIND" "SRC LANG" AND "SRC",
WITH THE "SRC" CONTAINING A URL TO THE WebVTT FILE.
THE SAMPLE CONTAINS THE KINDS
CAPTIONS, SUBTITLES, DESCRIPTIONS, AND CHAPTERS.
Pfeiffer: And we identify the language.
Because, of course, when we have user settings in browsers,
we want to automatically make certain tracks
available to the user
if the user has, for example,
said that they always want captions
or they always want subtitles in their language being shown.
So the browser can look through this markup
and identify which ones it has to turn on by default.
Describer: AFTER TEXT THAT READS "SRCLANG="
THE LETTERS "EN" ARE HIGHLIGHTED,
INDICATING ENGLISH LANGUAGE.
Pfeiffer: Now, in this case, I've used only WebVTT files.
The track layout, the way that we've defined track in HTML5,
is actually generic.
It can be used for other types of files as well,
TTML or SRT or any other formats
that will be implemented.
Describer: THE WORD "TRACK" IS HIGHLIGHTED
IN A TAG READING: TRACK LABEL="ENGLISH CAPTIONS".
Pfeiffer: But the generic way
that track works is in this way.
Describer: A NEW SLIDE IS TITLED
"USING CSS FOR RICHER STYLING."
A SAMPLE FILE FEATURES THE MAN IN HIS WORKSHOP
CAPTIONED "I HEARD ABOUT THIS ARDUINO PROJECT
AND SAW IT ONLINE."
THE WORD ARDUINO IS RED AND ALL CAPS.
THE REST OF THE CAPTION IS BLACK AND MIXED TYPECASE.
Pfeiffer: Now, once we've got it in the browser,
we can actually support more than what is directly possible
as markup in the WebVTT file,
because now we've got the text in the browser,
and we can make use
of all the functionality of the browser,
which has styling
and the concurrent style sheet functionality
available to it.
So this kind of styling is also available
if used in a browser, to these cues.
And the way in which this is being done
is that there's a pseudo-element in CSS
called ::cue.
And with that pseudo-element,
you can address, for example,
classes in the cue markup.
Describer: IN THE SAMPLE FILE, A TAG POINTING TO ".arduino"
SURROUNDS THE WORD ARDUINO.
Pfeiffer: And you can override the formatting
that by default would be given.
You can, for example--
well, in this case, it's been turned red,
uppercase, a different font family,
and a lighter weight.
Describer: IN A SEPARATE BOX, UNDER THE HEADING CSS,
THE ::CUE PSEUDO-ELEMENT ".arduino"
CONTAINS CODE SELECTING COLOR AS RED,
TEXT-TRANSFORM AS UPPERCASE,
FONT-FAMILY AS HELVETICA NEUE,
AND FONT-WEIGHT AS LIGHTER.
A NEW SLIDE IS TITLED "WebVTT DEFAULT RENDERING."
TWO HEADINGS READ
"POP-ON: BOTTOM THIRD OF VIDEO VIEWPORT, CENTERED"
AND "ROLL-UP: OVERLAPPING CAPTIONS ARE ADDED UNDERNEATH."
Pfeiffer: Now, we've spoken a lot.
We want to see a little bit of a demo here,
and I've made a little bit of a demo
which shows that we can do more than what's typically
being used for captions right now.
Most captions that are being used are pop-on captions,
which are captions that don't overlap in time.
There's one piece of caption,
one cue shown;
it disappears, and the next cue is brought up.
That's pop-on.
And that is the default way of rendering it.
But we may have a very different style
of providing captions as well,
which has traditionally been used
mostly in live captioning.
It's called roll-up.
So the cues will actually be added at the bottom
and roll up as the--
and the old ones will disappear.
So I've made a little example
that shows how that can be done as well.
Describer: A VIDEO PLAYS.
AS THE MAN IN HIS SHOP SPEAKS,
TWO LINES OF LEFT-ALIGNED CAPTIONS
We have a kind, which gives us a means
to group all the tracks of the same type together.
Describer: A CODE SAMPLE LISTS A SOURCE VIDEO
AND A SERIES OF TRACKS WITH THE FOLLOWING LABELS:
ENGLISH CAPTIONS, GERMAN SUBTITLES, FRENCH SUBTITLES,
ENGLISH DESCRIPTIONS, AND CHAPTERS.
EACH TRACK HAS ATTRIBUTES "KIND" "SRC LANG" AND "SRC",
WITH THE "SRC" CONTAINING A URL TO THE WebVTT FILE.
THE SAMPLE CONTAINS THE KINDS
CAPTIONS, SUBTITLES, DESCRIPTIONS, AND CHAPTERS.
Pfeiffer: And we identify the language.
Because, of course, when we have user settings in browsers,
we want to automatically make certain tracks
available to the user
if the user has, for example,
said that they always want captions
or they always want subtitles in their language being shown.
So the browser can look through this markup
and identify which ones it has to turn on by default.
Describer: AFTER TEXT THAT READS "SRCLANG="
THE LETTERS "EN" ARE HIGHLIGHTED,
INDICATING ENGLISH LANGUAGE.
Pfeiffer: Now, in this case, I've used only WebVTT files.
The track layout, the way that we've defined track in HTML5,
is actually generic.
It can be used for other types of files as well,
TTML or SRT or any other formats
that will be implemented.
Describer: THE WORD "TRACK" IS HIGHLIGHTED
IN A TAG READING: TRACK LABEL="ENGLISH CAPTIONS".
Pfeiffer: But the generic way
that track works is in this way.
Describer: A NEW SLIDE IS TITLED
"USING CSS FOR RICHER STYLING."
A SAMPLE FILE FEATURES THE MAN IN HIS WORKSHOP
CAPTIONED "I HEARD ABOUT THIS ARDUINO PROJECT
AND SAW IT ONLINE."
THE WORD ARDUINO IS RED AND ALL CAPS.
THE REST OF THE CAPTION IS BLACK AND MIXED TYPECASE.
Pfeiffer: Now, once we've got it in the browser,
we can actually support more than what is directly possible
as markup in the WebVTT file,
because now we've got the text in the browser,
and we can make use
of all the functionality of the browser,
which has styling
and the concurrent style sheet functionality
available to it.
So this kind of styling is also available
if used in a browser, to these cues.
And the way in which this is being done
is that there's a pseudo-element in CSS
called ::cue.
And with that pseudo-element,