Contemporary display systems are notorious for their lack of flow across display pages. The capabilities to dynamically change the makeup and layout of a display that are resident in windowing and sensor fusion systems are beginning to present the display system designer with even more complex display transition problems. The motion picture industry, on the other hand, has for years successfully presented a series of dynamic visual displays, where changes from one scene (or display) to the next have been accomplished with no notice, or where the change has actually enhanced the information flow to the viewer. This paper discusses how some of the techniques and principles used in making editing decisions might be applied to the design of display systems.

During a recent NATO Advanced Research Workshop on information system failure, several participants emphasized the relationship between display system effectiveness and the failure of the entire information system (see Wise and Debons, 1987). If display systems are to be effective, designers need guidelines that will not only allow each new display page to be understood on its own merit, but will also assure that each display works effectively with the remainder of the display system.

A display system can most profitably be conceived of as a transmission component in some larger information system, and as such, the display system has as its purpose to assist in the production of change in the cognitive and/or affective state(s) of the user. For centuries the rules of data display, via the spoken or written word, were viewed as an art or as rhetoric. As the discipline of Human factors has used many sources for inspiration in developing hypotheses on display design. Among these have been rhetoric, experimental and perceptual psychology, and more recently cognitive psychology. As a result, there are a wide range of relevant human factors guidelines currently available for the design of individual or stand-alone display pages. But there is little available guidance addressing the transition between display pages and other intra-display system issues. For example, what makes it easy for an user to transition from one display or data set to another, and what makes it difficult? These later issues are especially important in the display of real time fused sensor data and in windowing systems, where artificially intelligent processors can independently switch the form and substance of displays.

Film Editing For several years the authors have privately discussed the applicability of the principles involved in motion picture editing as a possible source of inter-display transition guidelines. The concept has high intuitive appeal when one considers how a good editor (i.e., all of the individuals responsible for selecting and organizing the images presented) can present a series of images that the viewer quite naturally combines into a meaningful flowing whole, even though the individual images have been taken from different angles, contain different data, have different fields-of-view, etc. The editor can actually make the transition between these images appear very natural (i.e., the viewer does not realize the visual inconsistency), in spite of the fact that the human viewer could never have naturally experienced a series of glances which change so dramatically along these parameters (e.g., immediate changes in field-of-regard and angle-of-view). It is interesting to note that some early film directors felt that the camera had to remain in a fixed position throughout the entire film to avoid confusing the audience. The purpose of this paper, therefore, will be to consider how the theory and principles of film making can be used as a source of new hypotheses in display system design. The heart of that effort will involve the consideration of the editing function of film making.

Editing can be defined as the creative ordering and sequencing of graphic and symbolic forms to produce desired cognitive and affective responses in the viewer. Editing principles involve the effective selection and temporal sequencing of different data sets in a series of displays, rather than the parallel presentation of those same data on a single display page.

Hochberg and Brooks (1978) and Woods (1985) have previously discussed this concept. Hochberg and Brooks labeled the concept visual momentum, and described it as "the impetus to gather visual information" (p.312). They also identified two components of visual momentum: "a fast component that brings the eye to those peripherally visible regions that promise to be informative or act as landmarks, and a more sustained component that directs the eye to obtain more detailed information about the main features that have already been located" (p. 312). Woods addressed a number of classic film techniques that enhance the visual momentum of a display system.

Both Hochberg and Brooks, and Woods, for example, considered the implication of a wide angle or long shot concept. In editing, this is the shot that is inserted to provide the viewer with overview or big picture of the situation or to help the viewer determine the scene's location or the general ambiance prior to moving into a closeup of the action. In the typical display environment this would be the equivalent of having a system overview display that is viewed either before or in parallel with the data about a particular piece of equipment. This wide angle shot/overview display assists the viewers to establish the mental model that will allow them to correctly interpret the data on the subsequent displays.

Based on the above, it appears that editing can provide a reasonable basis for generating hypotheses about display system design, and particularly about the transition between different display pages. The hypotheses presented below were developed from editing principles and on the practical experience of a professional director.

Hypothesis 1: Minimum Change Threshold There is some level of change in terms of the content and/or format (as expressed in display objective), below which the change will be perceived as a nuisance.

In film it is recommended that there be a significant change in terms of angle of the shot and/or field-of-view when a cut is made (See Figure 1 below). Changes that fall below this threshold are considered to be visual noise and a nuisance. Subthreshold changes result in drawing the viewer's attention, but fail to provide the new or different visual information that the viewer expects with a cut. It is the visual equivalent of the boring speech, where after getting your attention by promising something new (i.e., the cut), the speaker repeats exactly the same argument.

In general display system design this rule is analogous to the situation where the individual display objectives have too much overlap in the problem domain. As a result there are a number of display pages that are very nearly identical in terms of purpose and content. Therefore, changes from one display to another result in little, if any, change in the level or types of data presented. In such cases the display system designer must use extreme care when establishing display objectives. It is just as easy to establish an unnecessary display objective (and consequently an unnecessary display), as it is to include a lot of unnecessary data that just clutters the display. If the cut, or new frame does not extend the user's awareness, it has no effectiveness.

In those display systems where a artificially intelligent processor can initiate format or content changes, this hypothesis would suggest that a heuristic be included to preclude subthesholdchanges. For example, in windowing systems careful consideration should be paid to the criteri that are used to determine when a window is added or dropped. Likewise in sensor fusion systems, general changes in display content and/or format should be based on well-reasoned criteria. Too many uninformative changes (e.g., false alarms) will increase the probability of the viewer ignoring important changes.

Hypothesis 2: Sense of Position Transitions will be smoother if the general positions of the data sets established in the overview display are maintained throughout the display system.

In film there is a principle that the positions established in the opening shot should be maintained in the subsequent shots, even though the angle and/or field-of-view changes. In Figure 2 below, different camera positions represent different display formats and the black and gray figures represent different data sets. If camera position A provides the overview or establishing shot, as shown in shot A, then a closeup of the gray figure (or data set) should use format C. A closeup of the black figure (or data set) would use format B, not shot D, which would make a transition difficult.

The sense of position hypothesis indicates that the general positional relationships established between the data sets by the system overview display should be maintained in the layout of all related displays, even for different display objectives or as different display components come tothe foreground.

In display systems that do follow this rule, the process is usually based on the assumption of two dimensionality of the display screen. Thus, the transitional relationships that are established are purely two dimensional (i.e., a fixed camera moving only in X and Y dimensions). In many cases such an approach is probably sufficient. However, sensor fusion systems, like the type proposed for Super Cockpit (see Furness, 1986), will demand that such considerations be addressed. (Sensor fusion is the process of integrating data from a number of different sensors into a more meaningful single image.) For example, if a pilot would ask the display to show a target not as it appears with the current sensor- arget geometric relationship, but how it will look when the aircraft is abeam of the target, the application of this hypothesis could prove critical to the speed and correctness of the interpretation of the display.

An important point here is that even in this display of a physical environment (i.e., the target as it will look), the composition of the image, in terms of the positions of the objects in the future position display to those in the current position display, could make the difference between the success and failure of the mission, if the pilot required too much time to interpret that display.

Hypothesis 3: Point of Fixation The smoothness of a transition will vary as a function of the location of the pre- and post-cut locations of the visual fixation point.

It is a common practice for the film editor to estimate the viewer's visual fixation point on the last frame before the cut (Ralston, 1986). (The estimation is generally made based on the location of the brightest spot on the frame (Ralston, 1986).) If a very smooth transition between frames is desired, an attempt is made to place the expected fixation point on the next frame be in the general same location as it was on the first. If on the other hand, shock is desired (as in a good horror movie), the expected fixation point for the second frame will be placed far from the fixation point on the first. The analogy for the display system designer seems clear, especially with regard to horror movies and bad displays.

In those cases where one page is followed immediately in time by another (e.g., a cycling display set), this film principle would suggest that it may be good practice to have the desired initial fixation point on each display to match the estimated last point of fixation on the previous display. Or in sensor fusion systems (especially for environments like Super Cockpit where eye position will be known), control over whether data is smoothly blended into the current display or brought in with some "shock" value might be possible based on this hypothesis.

On displays where menus (or any technique that requires the eye to move to some point before you can select the next display) are used, a fixed location of the menu items might be desirable, so that the eye can move to the position where the data will be located.

Hypothesis 4: Continuity The introduction of new data should only be made after its relationship to the remainder of the data has been established.

The entrance of a new characters into a scene requires the new character be shown entering the area and moving to the action. New characters are not just popped in the action without a visual introduction (unless there is a complete change in scene indicating a new environment).

Likewise, it can be hypothesized that the addition of data to a display needs some type of a similar introduction. A means of identifying what the data is, where it comes from (e.g., its source), and why it has appeared at this moment. This would be especially true in windowing systems, where a new data window could be inserted by the display system. It would also be very significant to sensor fusion systems, where some sophisticated processing algorithm might decide that a certain datum should now be included in the display. The instant appearance of a "strange" datum in either case, would probably be just as confusing as a film cut where a new person instantaneously appears in a group just previously shown.

Hypothesis 5: Dramatic Necessity Display system induced transitions from one display page to another, or the addition of a new window, can be used to emphasize actions that are taking place in the system being described by the display system.

In film, Reisz and Millar (1968) provide the following scene as an example of this concept. A woman is setting with cigarette in her hand, while searching for a match. She recognizessomething off screen, and just as she gets up a cut is made to a table where some matches lay. Thecut provides the viewer with the reasonfor the her movement. The new scene is immediatelyunderstood by the viewer in terms of the actions that have already taken place. Not only would thesecond scene be difficult to interpret without access to the first, the first scene would be incompletewithout the second.

This concept when applied to display system design would postulate that changes induced by display system can allow the new data set(s) to stand alone on a display page (e.g., like the matches on the table) if the previous page or action was such that the current data can be understood in terms of of that action. In displays of fused data, the application of this hypothesis could be particularly valuable. As it may allow the viewer to easily move from fused data , to single sensor data, back to fused data without any need for a transition between displays.

This design principle may be one of the most challenging to implement in artificially intelligent systems, because its application assumes that the processor understands enough of the content of display pages for it to make the transition rule work effectively. Basically, the series must be designed in a way that will allow the viewer to develop the mental model of the action, so that the new data can "automatically" be interpreted.

Hypothesis 6: Tone The degree of change in the overall tone from one display page to the next, presents the viewer with a estimate of the magnitude of the difference between them.

When filming sequences that are to spliced together, considerable effort is made to control the light and shade values of each. Significant changes in these variables draw the viewer's attention to the transition and in film are used to indicate a more significant change. If the goal is make a visually unobtrusive change the values are obviously maintained as closely as possible to the previous values. If change needs to be reinforced, then changes in the light and shadow values are accentuated.

The application to display system design is rather obvious. Significant changes in the objectives of displays should be visually enhanced through changes in tone. In alphanumeric displays this mightbe accomplished with changes in the hue of the background. Displays that have common objectives, but where significant data changes have taken place, might maintain a common background hue and vary the hue's brightness or saturation in proportion to the magnitude of the change that has taken place.

General In several of the hypotheses discussed above camera position and/or field-of-view were used to demonstrate the rationale for the hypothesis. It seems that this analogy may be valuable in many cases in display system design were no rules or hypotheses currently exist. The designer might be advised to think of the data sets as figures in a scene, and analyze how the various components might be visualized from different camera angles. This may allow the designer to better conceptualize the data and thus provide a basis for determining a good inter-display transition. It might also be interesting to consider how different types of cuts (e.g., wipes, fades, quick cuts) could be applied to enhance the display transition process. While displays systems have traditionally only used the straight cut to transition, the different cutting techniques might provide the designer with a set of tools that would will enhance the information transfer capability of the transition process.

Human participation in any complex system depends on the user being able to quickly acquire, process, and use data about different aspects of system performance available from different sources. Display systems basically provide the medium for transmitting that data to the user. However, data transmission alone is not sufficient to assure that the user will be able to respond efficiently and effectively. That demands that the display system be designed to stimulate the user cognitively. To the extent that editing is based on similar goals, then it can effectively provide the basis for generating hypotheses about display design and display theory in general.

Furness, T. A., III. (1986). The super cockpit and its human factors challenges. The Proceedings of the Human Factors Society - 30th Anual Meeting - 1986.Dayton, OH.

Hochberg, J. and V. Brooks. (1978). Film cutting and visual momentum. In J. W. Senders et al. Eye Movements and Higher Psychological Functions. Hillsdale, NJ: Lawrence Erlbaum.

Kracauer, S. (1979). Theory of Film: The Redemption of Physical Reality. Oxford: Oxford University Press.

Ralston, M. (1986). Personal Communication.

Reisz, K. and G. Millar. (1968). The Technique of Film Editing. London: Focal Press.

Wise, J. A. and A. Debons. (1987). Information Systems: Failure Analysis. Heidelberg: Springer-Verlag.

Woods, D. D. (1986). Visual momentum. International Journal of Man-Machine Communication.