10-Jul-84 16:10:31,32194;000000000000 
Return-path: <MILLS@USC-ISID.ARPA>
Received: from USC-ISID.ARPA by DCN6.ARPA ; 10-Jul-84 16:07:34-UT
Date: 10 Jul 1984 12:06:06 EDT
From: MILLS@USC-ISID.ARPA
Subject: MMM Format Document
To:   mills@DCN6.ARPA

Return-Path: <FINN@USC-ISIF.ARPA>
Received: FROM USC-ISIF.ARPA BY USC-ISID.ARPA WITH TCP ; 6 Jul 84 21:07:24 EDT
Date:  6 Jul 1984 17:34:40 PDT
Subject: Multimedia Format Document
From: Gregory G. Finn <FINN@USC-ISIF.ARPA>
To: mmm-people@USC-ISIF.ARPA

     I include a draft concerning the format of multimedia messages.
It follows the SRI Report syntax.  It defines a virtual display page,
how one can control defacto pagfination, and use of the SPATIAL
property for TEXT and IMAGE media.

     This file is also available in <MMM> on ISIF.ARPA as MMM-22.TXT.



MMM-22

















                     A FORMAT FOR THE CONSTRUCTION OF MULTIMEDIA
                                      MESSAGES

                                   Gregory G. Finn
                                     6 July 1984



         INTRODUCTION
             Multimedia  messages  are  distinguished  from  most electronic
         messages because they can carry media other than  text  within  the
         message.  The media with which this document is concerned are:

            - TEXT

            - IMAGE

            - GRAPHICS

            - VOICE

             The  message  is  transmitted  in  the  format described by the
         revised version  of  RFC759 [MPM  82a].    The  message  proper  is
         composed under the guidelines of the revised version of RFC767 [MPM
         82b]  and  the  SRI  Report [Garcia-Luna 84].  Copies of RFC759 and
         RFC767 may be obtained from the author.  The purpose of this  paper
         is  to  further  clarify those documents and to define what else is
         necessary to allow the successful exchange of multimedia messages.

         1. Display Characterization
             It is assumed that a display screen will be the default  output
         device  for  text,  graphics, and pictorial data sent in a message.
         We must provide the composer with a  method  which  guarantees  the
         reader can display a message in the manner intended.  This requires
         a  model for the construction and display of multimedia messages on
         a screen.

             In this discussion,  a  message  window  is  assumed  to  be  a
         rectangular  display  surface  with  a given width and height.  The
         message window contains the displayable portion of a  message  when
         presented  to  the  reader.    It  does not necessarily consume the
         entire screen surface.  In some systems, displayable  area  outside
         the message window is used for menus, control displays, etc.

             The  message  window represents a single A4 sheet of paper, 200
         mm wide 300 mm in length, with two centimeter margins on the  sides
         and  three centimeter margins top and bottom.  This is similar to a
         8.5 by 11 inch sheet with 3/4 inch margins.  It  is  the  intention
         that  displayable  data  be  confined  to  this area.  This enables
         conversion to hardcopy without clipping consideration.  The display
         of data in the margin area or outside it is  strongly  discouraged.
         A  message  system  need  make  no effort to display such data (for
         messages with content larger than a single message window  see  the
         discussion of pagination below).

             Certain  systems  may not be able to display the entire message
         window at once.  It is their responsibility to provide scrolling or
         sub-paging control so that the user  may  at  least  view  all  the
         message  window  area  in  slices.    For the reason stated above a
         message window size larger than A4 is not recommended.



         1.1. Coordinate System
             The message window has a coordinate system used to control  the
         positioning of all displayable message parts.  The measurement unit
         is  a  micrometer.  A 32-bit twos complement integer representation
         is used.  The upper-left corner is the origin of the message window
         and is given the coordinates (0,0).  Pictorially,

          --------------------------------------------------   Screen Window
         |                                                  |
         |                                                  |
         | (0,0)                                            |
         |   ==>>  +++++++++++++++++++++++++                |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +     Message Window      +               |
         |        +                         +               |
         |        + A4 with margins removed +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |        +                         +               |
         |         +++++++++++++++++++++++++  <== (160000,  |
         |                                          240000) |
          --------------------------------------------------



         1.1.1. Relation Between Coordinates and Pixels
             Window  systems on emerging workstations are usually defined in
         units of rasters or pixels (picture elements).  While a pixel is  a
         convenient unit of measure for a particular manufacturer, it is not
         appropriate  to  pick  a  particular  pixel  size  as standard.  As
         display technology improves pixel  sizes  will  shrink,  and  pixel
         sizes  vary  from  manufacturer  to  manufacturer.  For simplicity,
         pixels are assumed to have a 1-to-1 aspect ratio.


         1.1.2. Scaling the Message Window
             It may be the case that the A4 window will be  scaled  down  on
         the  display screen.  When scaled the aspect ratio of 3-to-2 should
         be preserved.  The displayed  default  font  sizes  would  then  be
         expected to scale similarly.


         1.1.3. Calculating Pixel Addresses
             Integer  arithmetic  is  used  exclusively  in determination of
         screen position.  Calculating pixel  position  within  the  message
         window involves determining how many micrometers/pixel there are on
         your  screen.   This is determined by the height chosen, in pixels,
         for the message window.  A message window of 600 pixels  in  height
         by 400 in width would have a 3-to-2 aspect ratio and a scale of:

                  300000/600 = 500 micrometers per pixel


             Many of the newer workstations have a screen size sufficient to
         display  the  message window unscaled.  The Xerox 1108 workstation,
         for example, has 72 pixels per inch and the message  window,  minus
         margins, occupies a 765 by 510 pixel window.



         1.2. Spatial Control
             Since a page of a multimedia message may contain several pieces
         of  possibly  differing  media,  the  model must allow the composer
         control over the space used by those pieces in the message  window.
         This  information  must  also  be  conveyed  by  the message to the
         reader's software.

                 Media such as voice are not  displayable  in  the  same
             sense  as textual or pictorial media.  In the discussion of
             spatial control we are discussing displayable media only.

             Each displayable portion or unit of  a  message  must  contain,
         either  explicitly  or by implication, information concerning where
         it is to be placed within the message window.   The  region  within
         the  message  window where a unit's data is displayed is called the
         unit area.  When explicit information is given for a unit's area it
         is conveyed by a SPATIAL control property  list  which  accompanies
         the unit:

              SPATIAL:  p{  LEFT: Integer
                             TOP: Integer
                           WIDTH: Integer
                          LENGTH: Integer  }

             The  first  two  components  are  coordinates  relative  to the
         message window's origin and are in micrometers.  The second pair of
         components are offsets from the first  pair.    They  are  used  to
         describe a rectangular area inside the message window, within which
         the  unit's  accompanying data is to be displayed.  The rectangular
         area described by the spatial control property is marginless.  Data
         may be placed within it so that it abuts the edges of the area.

             Any portion of the rectangular area  which  falls  outside  the
         message window is not necessarily displayable.  The display program
         should  make  an  effort  to display any 'reasonable' message unit.
         Negative values are allowed, but care should be taken to ensure the
         rectangular  area  described  falls  properly  within  the  message
         window.

             "Silly"  unit  areas  are  ignored  and  the  system default is
         supplied (see below).  Examples of silly areas are those  for  text
         less  than  a  line  in  height or less than one space in width.  A
         system implementor might wish to extend this to cover any unit area
         which extends outside the message window.


         1.2.1. Default Spatial Control
             Displayable units of a message must be placed somewhere  within
         the  message window.  Where no spatial control is present a default
         unit area must be supplied.  The default should be chosen  to  make
         sense  in  the  simplest  case.   The default unit area will be the
         entire message  window.    Although  this  works  properly  in  the
         simplest  cases,  it  may  cause overlaying of data in more complex
         messages.



         1.3. Pagination
             It will often be the case that a message will contain more data
         than can be displayed within a single message window.  In this case
         a pagination convention must  be  followed  when  constructing  the
         message.    Pagination  is  accomplished  by  use of the <doc-pair>
         proplist within a message.  When encountering a <doc-pair> the user
         program displaying the message should assume  that  a  new  message
         window  is  required.   The mechanics of achieving this are left to
         the particular user program.
             As an example, the graphical outline  of  a  two  page  message
         follows:

                                   BODY
                                    |
                                    |
                                SEQUENTIAL
                                  /   \
                                 /     \
                                /       \
                               DOC      DOC
                                |        |
                                |        |
                              TEXT     TEXT


         2. Text Message Units
             Following  the  notation  conventions  of [Garcia-Luna 84] this
         BNF-like grammer defines the protocol to  be  followed  by  a  TEXT
         message unit.

                  <tx-pair> ::= p{ TEXT <text structure> }

           <text structure> ::= p{ [ SPATIAL <spatial control> ]
                                    PROTOCOL <text protocol>
                                     VERSION <version number>
                                        DATA <text medium>       }

            <text protocol> ::= PARAGRAPH | VERBATIM | ENUMERATE | ITEMIZE

           <version number> ::= <index datum = 1>

              <text medium> ::= l{ <textstring datum>* }

             Within  the <text medium> list, there may be one or more blocks
         of text that will all be  formatted  according  to  the  associated
         <text protocol>.

             At  any  point  there is a default font in which characters are
         represented.  All character boxes in a font  have  the  same  fixed
         height  expressed  in  pixels.  Each character may have a different
         width which is the number  of  pixels  of  width  occupied  by  the
         character.   Although it is not desirable to rigidly specify fonts,
         some reasonable guidelines should be established.

             To ensure that a message's textual data is displayable  at  the
         destination  it  is  suggested  that  a 12 point font, with a fixed
         width, be assumed in  constructing  the  <text medium>  data  in  a
         message.    Upon  reception  the  use of a 12 point or smaller font
         would ensure all textual data would fall  within  their  respective
         unit areas.

             The  characters  within  the text element are defined to be the
         "printable ASCII subset" plus, in some  protocols,  several  format
         effectors.  The printable subset includes the characters space <SP>
         through  tilde,  codes  32  through  126 (decimal).  The acceptable
         format effectors are horizontal tab <HT> (decimal 9), and  the  two
         character  sequence  carriage  return-line  feed  <CR><LF> (decimal
         codes 11 and 13).  The exceptions to the  printable  ASCII  subset,
         <CR> and <LF>, should only appear in paragraphs formatted according
         to the VERBATIM protocol.

             A  standard tab grid is assumed.  Starting from the left border
         of the unit area a tab stop is set  every  five  spaces  until  the
         right  border  of  the unit area is reached or the placement of the
         next stop would exceed that border.

                 The  width  of  a  space  varies  with  both  font  and
             justification  styles.    To define a tab grid the width of
             some 'standard' space must be assumed.  Fixed  width  fonts
             have  a  default  space width usually equal to the width of
             the bounding box for a  character.    That  is  a  suitable
             definition.    Where  no  such  default is assumed, as in a
             variable width font, a space may be given the  width  of  a
             single  em  quad,  that  is, a square of width equal to the
             point size of the type font used.

             When encountered in text the horizontal  tab  character,  <HT>,
         has  the following behavior:  At least one space is output followed
         by the distance necessary to reach the next tab stop.

             A "line break" is the sequence of characters <CR><LF>.   It  is
         only used in the VERBATIM protocol.

             The  action  taken  when encountering any characters other than
         those legal for a  given  protocol  is  to  ignore  the  character.
         Underlining and overprinting are not currently defined.



         2.1. Appearance
             It  is  not the responsibility of the user interface program to
         typeset text.  A proper job of typesetting  requires  knowledge  of
         grammar  and  usage.   The program should be responsible for making
         the text meet standards of style assuming it is supplied with  text
         which makes sense as a paragraph.

             Multiple  spaces  are  not compressed to single spaces.  Spaces
         are not inserted where needed for punctuation.  Except for VERBATIM
         protocol, spaces at the beginning of a line are  only  allowed  for
         the  first  line  of  a  paragraph.  Indention of the first line is
         under control of the sender.  Setting of paragraphs in  block  mode
         is achieved by not passing any indention in the text.



         2.2. PARAGRAPH Protocol
             The  PARAGRAPH  protocol is used to display paragraphs of text.
         Each element within the <text medium> list is considered  to  be  a
         serarate  paragraph.  The manner of display will be flush left with
         indention, if any, provided in the text.    We  assume  the  cursor
         begins  at  the  upper  left-hand corner, down one line, within the
         unit area as the  display  of  a  paragraph  is  begun.    Text  is
         presented in word filled form in the unit area.

             Words  are  placed  onto a line until a word would break across
         the area's right border.  When this occurs  a  new  line  is  begun
         immediately  after  the  word  previous to the one that crossed the
         right border.  The cursor is placed  down  one  line  at  the  left
         border  and  the current word (the one which broke across the right
         border) begins the new line.  Either a fixed or variable width font
         may be used.  Justification is not assumed but may be used.

             A  blank line is placed on the display window immediately after
         the last line of any paragraph displayed.



         2.3. VERBATIM Protocol
             The VERBATIM protocol is used to display text precisely  as  it
         appears   within  a  block  of  text.    Each  element  within  the
         <text medium> list is considered to be a separate paragraph.    The
         cursor  begins at the upper left-hand corner, down one line, within
         the unit area.  Text  is  presented  as  is  (paying  attention  to
         <CR><LF>).

             Characters are placed onto a line until a character would break
         across  the  unit  area  right  border.    When  this  occurs, that
         character, and the  remaining  characters  from  the  current  line
         within  the  text  block  element  are  discarded.    The  line  is
         truncated.  The cursor is placed down one line at the  area's  left
         border  and  a new line is begun with the first character following
         the truncated line.  If the line is truncated it may  be  advisable
         to place some indication on the right border that more text existed
         than could be presented.  Justification may not be done on verbatim
         text.

             A  blank line is placed on the display window immediately after
         the last line of any paragraph displayed.



         2.4. ENUMERATE Protocol
             The ENUMERATE protocol is  used  to  format  text  as  numbered
         paragraphs.     Each  element  within  the  <text medium>  list  is
         considered to be a separate paragraph.  The layout  is  similar  to
         PARAGRAPH format except that it is set with a hanging indent.

             Each  paragraph  is labeled with its ordinal position, starting
         at 1) within the <text medium> list, and layed out with  a  hanging
         indent.    The  label  is placed toward the left border of the unit
         area and a single tab is issued, which defines the indention.

             A blank line is placed on the display window immediately  after
         the last line of any paragraph displayed.

             The  following  two  examples are both legal interpretations of
         this protocol.  The boxes are not part of  the  representation  but
         rather show the boundaries of the unit area.

         Example 1:

           +------------------------------------------------------------+
           |1    This is an example  of paragraphs  displayed  under the|
           |     ENUMERATE   protocol.   This  is  the  first  paragraph|
           |     displayed.                                             |
           |                                                            |
           |2    This is the second paragraph displayed.                |
           +------------------------------------------------------------+

         Example 2:

           +------------------------------------------------------------+
           |  1. This is an example of paragraphs displayed under the   |
           |     ENUMERATE protocol.  This is the first paragraph       |
           |     displayed.                                             |
           |                                                            |
           |  2. This is the second paragraph displayed.                |
           +------------------------------------------------------------+




         2.5. ITEMIZE Protocol
             The  ITEMIZE protocol is used to display paragraphs as seperate
         items.  It is identical  to  the  ENUMERATE  protocol,  except  the
         labels  on  the  text  blocks are all the same, typically an single
         ASCII character such as "*", "-", "+", ">" or "o".

             The following two examples are both  legal  interpretations  of
         this  protocol.    The boxes are not part of the representation but
         rather show the boundaries of the itemization.

         Example 1:

           +------------------------------------------------------------+
           |-    This is an example  of paragraphs  displayed  under the|
           |     ITEMIZE   protocol.   This  is  the   first   paragraph|
           |     displayed.                                             |
           |                                                            |
           |-    This is the second paragraph displayed.                |
           +------------------------------------------------------------+

         Example 2:

           +------------------------------------------------------------+
           |  *  This is an example of paragraphs displayed under the   |
           |     ITEMIZE protocol.  This is the first paragraph         |
           |     displayed.                                             |
           |                                                            |
           |  *  This is the second paragraph displayed.                |
           +------------------------------------------------------------+

         Example 3:

           +--------------------------------------------------------------+
           |Item:     This is an example of paragraphs displayed under the|
           |          ITEMIZE protocol.  This is the first paragraph      |
           |          displayed.                                          |
           |                                                              |
           |Item:     This is the second paragraph displayed.             |
           +--------------------------------------------------------------+


         3. Image Message Units
             The IMAGE medium is used to display pictorial data.  The syntax
         for an image message unit is:

                 <ix-pair> ::= p{ IMAGE <image structure> }

         <image structure> ::= p{ [ SPATIAL <spatial control> ]
                                  PROTOCOL <image protocol>
                                   VERSION <version number>
                                 PARAMETER <parameters>
                                      DATA <image medium>       }

          <image protocol> ::= BITMAP | RGB

          <version number> ::= <index datum = 1>
          <parameters> ::= p{     WIDTH : <index = # of pixels per line>
                                  LENGTH : <index = # of lines in image>
                                                    .
                                                    .
                                                    .
                            }

            <image medium> ::= l{ <bitstring datum>* }



         3.0.1. Data Consumption
             Pixel   data   is   consumed   from   the  <image medium>  data
         left-to-right for each line, and the lines are displayed  in  their
         unit  area  top-to-bottom.   Each <bitstring datum> must contain in
         its count field a number of bits evenly divisible by the  bits  per
         pixel.    The  number  of  bits  per  pixel  is  determined  by the
         particular <image protocol>.  A  pixel  is  not  allowed  to  break
         across a bitstring.


         3.0.2. Image Size
             The  image  size  (width  and  length  in pixels) is determined
         explicitly  by  the  WIDTH  and  LENGTH  values   passed   in   the
         <parameters>.    When an insufficient amount of data is present the
         image is zero-filled.  That is, enough zero pixels are  assumed  to
         exist to fulfill the image's data requirements.


         3.0.3. Conflict with SPATIAL Unit Area
             It  will  sometimes  be the case that SPATIAL data defining the
         unit area is too large or too small for the image size.  When  this
         is the case the following conventions are followed:

         Area too large  The  image  is  displayed  centered within the unit
                         area.

         Area too small  The center of the image  is  displayed.    This  is
                         achieved, as nearly as possible, by evenly trimming
                         the  top  and bottom of the image to fit vertically
                         within the unit  area,  and  performing  a  similar
                         operation  on the left and right sides of the image
                         if necessary.



         3.1. BITMAP Protocol
             The BITMAP protocol is used to define monochrome images.    The
         <parameters> field contains three name/value pairs.

          <parameters> ::= p{     WIDTH : <index = # of pixels per line>
                                 LENGTH : <index = # of lines in image>
                             BITS/PIXEL : <index = # of bits/pixel>
                            }

             BITS/PIXEL  tells  the displaying program how many bits of data
         to consume for each pixel.   The  bits  are  collected  to  form  a
         positive number which represents a binary grey level.  The bits are
         collected   left-to-right,   with  the  bit  of  largest  magnitude
         appearing in the data first.  The greater the magnitude of a  pixel
         the brighter when displayed.  In the degenerate case of one bit per
         pixel, a one represents white and a zero black.

                 For  monochrome one bit per pixel displays, it is often
             the case that software treats a pixel value of one as black
             and zero as white.  The user should be aware that for these
             systems the direct transmission of bits as received, to the
             screen,  may  result  in  an intensity inverted image.  The
             display program  should  provide  the  user  an  option  of
             inverting  the  pixel  values in this case.  The definition
             above is chosen to be  as  consistent  as  possible  across
             varying pixel sizes.

             Silly  values  for BITS/PIXEL are ignored and a value of one is
         assumed.  When a displaying program displays fewer bits  per  pixel
         than  is  supplied, the pixels must be rescaled so that they can be
         displayed.    The  particular  method  is  left   to   the   system
         implementor.



         3.2. RGB Protocol
             The  RGB  protocol  is  used  to  transmit  color  images.  The
         <parameters> field contains five name/value pairs.    The  required
         <parameters> are:

          <parameters> ::= p{     WIDTH : <index = # of pixels per line>
                                 LENGTH : <index = # of lines in image>
                               BITS/RED : <index = # of red bits/pixel>
                             BITS/GREEN : <index = # of green bits/pixel>
                              BITS/BLUE : <index = # of blue bits/pixel>
                            }

             The  WIDTH  and  LENGTH  were defined earlier.  An RGB protocol
         pixel may be thought of as comprised of three contiguous  component
         fields:    red, green, and blue.  These fields are read in from the
         data left-to-right.  The size of each field in bits  is  determined
         by  the  three  component  specifiers:  BITS/RED,  BITS/GREEN,  and
         BITS/BLUE.  The  first  field  contains  the  field  size  in  bits
         (magnitude)  of  the RED component for the pixel.  The second field
         contains data concerning  GREEN  component,  and  the  third  field
         contains  the BLUE component.  The values for BITS/RED, BITS/GREEN,
         and BITS/BLUE sum to the number of bits  per  pixel  which  is  why
         there is no specific BITS/PIXEL parameter.

                 There  is  great  variation  in  color  monitors.  Most
             utilize color lookup tables but of varying sizes.   Usually
             these  act  as  an index to a table of red, green, and blue
             magnitudes.  One may define the  transmission  standard  as
             close  to  the monitor as possible by specifying RGB values
             --- or --- as far away as possible by  defining  pixels  in
             terms  of  a  standard  such  as  a color cone or with hue,
             intensity, and saturation.

                 The RGB protocol  defined  here  makes  no  attempt  to
             ensure  color  fidelity.    Its chief virtue is simplicity.
             For differing color monitors, or  similar  but  differently
             adjusted  monitors,  equivalent  pixel  values  may produce
             differing colors.  For the RGB protocol, color fidelity  is
             ensured  only  when the sender and receiver have calibrated
             their  equipment  and  agreed  on  their   translation   of
             magnitude to color.

                 The  RGB protocol is chosen as the most straightforward
             method to transmit a color image.  It is by  no  means  the
             most   efficient.     Neither  is  it  the  most  faithful.
             Accompanying the image with a translation table  may  allow
             significant compression.  Other protocols may be defined to
             address these problems in detail.


                                     REFERENCES


         [Garcia-Luna 84]
                        Garcia-Luna, J.J., Poggio, A., Elliot, D.
                        Research Into Multimedia Message System
                           Architecture.
                        Technical Report SRI Project 5363, SRI
                           International, February, 1984.


         [MPM 82a]      Postel, J.
                        Internet Multimedia Mail Transfer Protocol [RFC:759]
                        Defense Advanced Research Projects Agency, 1982.


         [MPM 82b]      Postel, J.
                        Internet Multimedia Mail Document Format [RFC:767]
                        Defense Advanced Research Projects Agency, 1982.
-------
-------

                    