Multimedia
Week 4
LBSC 690
Information Technology
Agenda
•	Questions
•	XML review
•	Images
•	Audio
•	Streaming
•	SMILe
What’s a Document?
•	Content

•	Structure

•	Appearance

•	Behavior
History of Structured Documents
•	Early standards were “typesetting languages”
–	NROFF, TeX, LaTeX, SGML
•	HTML was developed for the Web
–	Too specialized for other uses
•	Specialized standards met other needs
–	Change tracking in Word, annotating manuscripts, …
•	XML seeks to unify these threads
–	One standard format for printing, viewing, processing
Goals of XML
•	Metalanguage 
–	A toolkit for design markup languages
•	Unambiguous markup
–	Clear span of tags
•	Separate markup from presentation
–	Style info => stylesheet, so easy to change
•	Be simple
A Family of Standards
•	Definition: DTD
–	Names known types of entities with “labels”
–	Defines part-whole and is-a relationships
•	Markup: XML
–	“Tags” regions of text with labels
•	Markup: XLink
–	Defines “hypertext” (and other) link relationships
•	Presentation: XSL
–	Specifies how each type of entity should be “rendered”
Some XML Applications
•	Text Encoding Initiative
–	For adding annotation to historical manuscripts
–	http://www.tei-c.org/

•	Encoded Archival Description
–	To enhance automated processing of finding aids
–	http://www.loc.gov/ead/

•	Metadata Encoding and Transmission Standard
–	Bundles descriptive and administrative metadata
–	http://www.loc.gov/standards/mets/
The Gullibility of Human Senses
•	Three simple tricks for producing
–	Images
–	Video
–	Audio
•	But… how do you move the bits around fast enough?

Visual Perception
•	Closely spaced dots appear solid
–	But irregularities in diagonal lines can stand out
•	Any color can be produced from just three
–	Red, Blue and Green: “additive” primary colors
•	High frame rates produce apparent motion
–	Smooth motion requires about 24 frames/sec
•	Visual acuity varies markedly across features
–	Discontinuities easily seen, absolutes less crucial
Basic Image Coding
•	Raster of picture elements (pixels)
–	Each pixel has a “color”
•	Binary - black/white (1 bit)
•	Grayscale (8 bits)
•	Color  (3 colors, 8 bits each)
–	Red, green, blue
•	Screen
–	A 1024x768 image requires 2.4 MB
•	So a picture is worth 400,000 words!
Monitor Characteristics
•	Technology (CRT, Flat panel)
•	Size (15, 17, 19, 21 inch)
–	Measured diagonally
–	For CRT, key figure is “viewable area”
•	Resolution
–	640x480, 800x600, 1024x768, 1280x1024 pixels
•	Layout (three dot, lines)
•	Dot pitch (0.26, 0.28)
•	Refresh rate (60, 72, 80 Hz)
Some Questions
•	How many images can a 64 MB flash card store?
–	But mine holds 120.  How?

•	How long will it take to send an image at 64kb/s?
–	But my Web page loads faster than that.  How?

•	But in reality images don’t have these problems
–	How do we get around these problems?
Compression
•	Goal: reduce redundancy
–	Send the same information using fewer bits

•	Originally developed for fax transmission
–	Send high quality documents in short calls

•	Two basic strategies:
–	Lossless: can reconstruct exactly
–	Lossy: can’t reconstruct, but looks the same
Palette Selection
•	Opportunity:
–	No picture uses all 16 million colors
–	Human eye does not see small differences

•	Approach: 
–	Select a palette of 256 colors
–	Indicate which palette entry to use for each pixel
–	Look up each color in the palette
Run-Length Encoding
•	Opportunity:
–	Large regions of a single color are common

•	Approach:
–	Record # of consecutive pixels for each color

•	An example of lossless encoding
GIF
•	Palette selection, then lossless compression
•	Opportunity:
–	Common colors are sent more often
•	Approach:
–	Use fewer bits to represent common colors
•	1	Blue	75%	75x1=  75	75x2=150
•	01	White	20%	20x2=  40	20x2=  40
•	001	Red	  5%	  5x3=  15	  5x2=  10
                              	           130	           200
JPEG
•	Opportunity:
–	Eye sees sharp lines better than subtle shading

•	Approach:
–	Retain detail only for the most important parts
–	Accomplished with Discrete Cosine Transform
•	Allows user-selectable fidelity

•	Results:
–	Typical compression 20:1
Variable Compression in JPEG
Discussion Point:
JPEG vs. GIF in Web images
Hands-On Exercise: 
Convert Between Formats
•	Download and save two images
–	http://www.umiacs.umd.edu/~daqingd/image1.jpg
–	http://www.umiacs.umd.edu/~daqingd/image2.gif

•	Use Microsoft Paint to convert each to the other format, and compare quality and the size

•	Why the difference?

Discussion Point: When is Lossless Compression Important?
•	For images?

•	For text?

•	For sound?

•	For video?
Basic Video Coding
•	Display a sequence of images
–	Fast enough for smooth motion and no flicker

•	NTSC Video
–	60 “interlaced” half-frames/sec, 512x486

•	HDTV
–	30 “progressive” full-frames/sec, 1280x720
Video Compression
•	Opportunity: 
–	One frame looks very much like the next

•	Approach: 
–	Record only the pixels that change

•	Standards:
–	MPEG-1: Web video (file download)
–	MPEG-2: HDTV and DVD
–	MPEG-4: Web video (streaming)
Basic Audio Coding
•	Sample at twice the highest frequency
–	One or two bytes per sample

•	Speech (0-4 kHz) requires 8 kB/s
–	Standard telephone channel (1-byte samples)

•	Music (0-22kHz) requires 88 kB/s
–	Standard for CD-quality audio (2-byte samples)
Speech Compression
•	Opportunity:
–	Human voices vary in predictable ways

•	Approach:
–	Predict what’s next, then send only any corrections

•	Standards:
–	Real audio can code speech in 6.5 kb/sec

•	Demo at http://www.data-compression.com/speech.html
–	Scroll down to near the bottom
How do MP3s work?
•	Opportunity:
–	The human ear cannot hear all frequencies at once, all the time
•	Approach:
–	Don’t represent things that the human ear cannot hear
Human Hearing Response
Frequency Masking
Temporal Masking
Putting it all together…
•	Psychoacoustic compression:
–	Eliminate sounds below threshold of hearing 
–	Eliminate sounds that are frequency masked 
–	Eliminate sounds that are temporally masked 
–	Eliminate stereo information for low frequencies 

Transmission
•	Download
–	Transfer the whole file, then start replay
–	Can be very slow for large files

•	Streaming
–	Play the file as it is received
•	Also suitable for live broadcasts
–	Requires a sufficiently fast connection
The “Last Mile”
•	Traditional modems
–	“56” kb/sec modems really move ~3 kB/sec
•	Digital Subscriber Lines
–	384 kb/sec downloads (~38 kB/sec)
–	128 kb/sec uploads (~12 kB/sec)
•	Cable modems
–	10 Mb/sec downloads (~1 MB/sec)
–	256 kb/sec uploads (~25kB/sec)
Streaming Audio and Video
•	Begins replay after only a portion received 
•	Buffer provides time to recover lost packets
•	Interrupts replay when “rebuffering”
Hands On: RealPlayer
•	View streaming real video
–	http://www.c-span.org

•	Select “Tools/Playback statistics”

•	Pay attention to bandwidth and lost packets
Narrated PowerPoint
•	Create your slides

•	Slide Show -> Record Narration
–	Set microphone level

•	Record the narration
–	Slide transitions are automatically captured

•	Narration plays automatically when displayed
Adding Video to PowerPoint
•	Insert->Movies and Sounds
–	Movies from file (a .mpg file)

•	Decide whether you want autostart
–	If not, it starts when you click on it
Illustrating RealAudio
•	Create a .ram file
–	URL for the RealAudio
–	Dimensions of the picture
–	URL for the picture

http://www.umiacs.umd.edu/~oard/teaching/690/fall05/notes/4/media.html
Synchronizing Multiple Media
•	Scripting Languages
–	Synchronized Multimedia Integration Language (SMIL)

•	Custom applications
–	Macromedia Flash

•	Content representation standards
–	MPEG 4
SMILe
•	W3C standard
–	Player-specific extensions are common

•	XML, with a structure similar to HTML
<smil>
<head> … </head>
<body> … </body>
</smil>
Elements in SMIL
•	Window controls (in <head>)
–	Controlling layout: <region>, <root-layout>

•	Timeline controls (in <body>)
–	Sequence control: <seq>, <excl>, <par>
–	Timing control: <begin>, <end>, <dur>
 
•	Content types (in <body>)
–	<audio>, <video>, <img>, <ref>
SMIL Examples
•	Implemented in RealOne Player
•	Example:
http://www.umiacs.umd.edu/~oard/teaching/690/fall05/notes/4/media.html
–	First, run the executable
–	Then, view .smil file

Before You Go!
•	On a sheet of paper (no names), answer the following question:

	What was the muddiest point in today’s class?