Effect of a range ring and of intruder vertical rate on pilot perception of separation on a cockpit display of traffic information

This study was conducted to determine the effect of a range ring and intruder vertical rate on pilots' perception of aircraft separation as viewed on a cockpit display of traffic information. A group of 30 pilots monitored 80 unique scenarios in which they determined, as early as possible, what the vertical miss distance would be when a single intruder passed ownship. The pilots' decision time and perceived vertical miss distance were compiled for each scenario. Range ring did not have a significant effect on the perception of vertical miss with regards to time or error, while vertical rate had a significant effect on time and error.

Starting in the late 1970s and continuing through the 1980s, NASA's Ames and Langley Research Centers studied traffic display formats and pilot reactions. These CDTI studies used heading or track-up displays (with constantly changing orientation), so the displayed traffic information corresponded to ownship's heading. A great deal of study was done in looking at interface issues, including display size (e.g., Anderson, Curry, Weiss, Simpson, Connelly, and Imrich, 1971; Hart and Loomis, 1980; Abbott and Moen, 1981), display orientation (e.g., O'Conner, Jago, Baty, and Palmer, 1980; Anderson et al., 1971), update rate (e.g., Jago, Baty, O'Conner, and Palmer, 1981; Palmer, Jago, Baty, and O'Conner, 1980; Anderson et al., 1971), symbology (e.g., Hart and Loomis, 1980; O'Conner, Jago, Baty, and Palmer, 1980; Chappell and Palmer, 1983), and the use of perspective displays (e.g., Ellis, McGreevy, and Hitchcock, 1987).

While range rings have been used in several studies they have never been directly studied. When a ring was used there was no reason given as to its distance from ownship. Rooney (1992) stated that subjects thought the range ring was useful, but this was not experimentally examined.

Rooney (1992) examined pilots' perceptions and responses to information describing the vertical plane situation. He noted that there were few studies which included vertical rate in the encounter geometry and of those, no specific conclusions were drawn on the effect of vertical rate on pilot perception. While Rooney did find a significant relationship between vertical rate and error, a problem with the experiment and data analysis makes the results suspect.

Rooney's subjects noted that judging vertical separation was a more difficult task than judging horizontal separation. This is due to the inadequate vertical information provided by plan-view CDTI. Research will be needed to understand pilots' ability to use the available vertical information because the plan-view display will remain the primary format. A more thorough understanding of the effects of vertical rate information and symbology on pilots' perception of traffic geometries will lead to an effective and efficient presentation of the vertical plane on a plan-view display.

Subjects

The subjects participating in this study were 30 volunteers from Embry-Riddle Aeronautical University. All subjects held at least a private pilot license and satisfied FAA currency requirements. Subjects' ages ranged from 18 to 35 with a mean of 25 (s= 5.0). Total flight time ranged from 65 to 4000 hours with a mean of 433 hours (s = 756). Pilot certificates held included 19 private, seven commercial, and four certified flight instructors.

Apparatus

A Macintosh IIsi® personal computer and SuperCard® software was used for this study. Actual design of the CDTI display and images were accomplished using Canvas® graphics software and transferred to SuperCard. SuperCard was implemented to construct and then simulate a dynamic CDTI which sent the experimental data (time, error, & scenario number) to individual text files.

Procedure

The experiment employed a 2 x 4 x 5 x 2 within-subjects repeated measures design. The independent variables were whether the 3-mile range ring was displayed, the intruder vertical rate, the vertical miss distance, and the angle of approach for the intruder. The approach angles employed were 0 and 50 degrees from ownship heading. The vertical rates remained constant throughout each scenario, but were varied between scenarios. The four levels of intruder vertical rate were 1000, 1500, 2000, and 2500 feet per minute. The five levels of vertical miss distance were -600, -300, 0, +300, and +600 feet. Climbing and descending flight paths appeared the same on the display and were considered symmetrical, therefore climbs and descents were evenly distributed across scenarios. Approaching from the left or right was considered symmetrical, so the 50o approaches were distributed evenly across the right and left portions of the screen. The five levels of the vertical Miss distance variable were evenly distributed throughout the scenarios. The vertical Miss distances could not be considered symmetrical about ownship. This was due to some scenarios being crossovers and others not. A crossover occurred when the intruder flew through ownship's exact altitude before passing ownship and has been found to affect pilots' perceptions of the display in past studies (Hart & Loomis, 1980). This was controlled for by using an equal number of crossover and non-crossover for each condition.

The verbal instructions were followed by four different training scenarios and subsequentltly the 80 experimental scenarios.

During the trials, upon determining how the intruding aircraft would pass ownship, the subjects clicked the mouse button to halt the scenario and display the vertical miss scale. Once the pilot selected a miss distance, the display was then blanked and the next scenario was randomly chosen. Subjects were given a break of up to 10 minutes after the 27th and 55th scenarios.

Time and error data were collected for all trials. Time was measured from the start of the scenario to the point when the subject clicked the mouse button, signifying a readiness to make an estimation of vertical miss. Error was defined as the absolute value of the difference between the actual vertical Miss distance for the scenario and the distance selected by the subject.

Time and error were analyzed using a pairwise Pearson correlation to determine if subjects traded time for accuracy. This tradeoff would manifest itself by the successful outcome of subjects waiting longer in order to make a more accurate determination of the vertical miss. The resulting correlation yielded a coefficient of r=-0.639, n=30, p<0.01.

A four-way within-subjects ANOVA was performed on the dependent variable Time using the factors: ring (two levels), rate (four levels), miss (five levels), and angle (two levels). No significant main effect was found for ring F(1, 29)=0.51, p=0.483. The subjects did not select a miss distance significantly faster or slower when the ring was not displayed (M=38.3 sec.) versus when it was (M=38.8 sec.). The vertical rate of the intruder was found to be significant for time; F(3, 87)=8.39, p<0.001.

A Student Newman-Keuls range test was performed on the four levels of vertical rate. The result was a significantly faster (p<.05) response time for 1000'/min than for 2000'/min and 2500'/min. Response time for 1500'/min was also significantly faster (p<.05) than for 2000'/min and 2500'/min. There was no significance for 1000'/min versus 1500'/min or 2000'/min versus 2500'/min.

A four-way within-subjects Analysis of Variance (ANOVA) was performed on the dependent variable Error using the factors: Ring (two levels), Rate (four levels), Miss (five levels), and Angle (two levels). Error refers to the absolute difference between selected Miss and actual Miss. Again, no significant main effect was found for Ring F(1, 29)=1.99, p=0.169. The subjects did not have significantly more Error when the Ring was not displayed (M=344.1 ft.) versus when the ring was displayed (M=332.6 ft.).

The vertical Rate of the intruder was found to be significant for Error; F(3, 87)=8.85, p<0.001. The results show that when the intruder approached ownship at a vertical Rate of 2500 ft./min., the subjects experienced significantly higher ERROR when compared to all other vertical Rates. Additionally, there was significantly more ERROR associated with 2000 ft./min. than with 1500 ft./min. These results are shown in Table 5.

This study focused on the pilots' ability to quickly judge future vertical separation between own aircraft and a single intruder. It was emphasized in the training instructions that the time required to make a decision and the accuracy of that decision were equally important. Therefore, pilots were to make their choice as soon as they determined a separation distance. They were not to wait and build confidence in their determination. The correlation between Time and ERROR showed that there was a tradeoff of time for accuracy. This is to be expected because as time increases, the difference between the present intruder vertical distance and the Miss distance becomes smaller and thus, easier to judge. The focus on equal importance for time and accuracy may have altered the methods used by pilots to make their decisions. A different focus, such as stressing the need for accuracy by letting the intruder fly in closer, may have resulted in a different outcome.

Pilots were asked during the debrief what methods they used to arrive at a decision. Pilots stated several methods that were based on determining the vertical change of the intruder over a fixed distance. The most readily used distance was the 3.5 nm. point (half-way). Several of the subjects said they used the Ring, when displayed, to make a more accurate determination of the half-way point. Another popular method was to let the intruder fly for three nautical miles and determine the altitude change, then add/ subtract this number from the relative altitude when the aircraft reached three nautical miles from ownship (the range Ring if displayed). One subject stated he used this method because, for him, accuracy was more important than horizontal separation and this was a good compromise.

All the above methods depended upon the intruder not deviating from its course. Changes in the intruder flight path will plague the effectiveness of any display that requires the operator to make predictions. Subjects knew the intruder would not deviate from its path, that it would pass directly over ownship, and that it would climb/descend at a constant Rate. This knowledge undoubtedly assisted the pilots in making more accurate decisions because when an intruder deviates from its original course, there is no longer a linear relationship between time, horizontal distance, and vertical separation.

The second hypothesis that intruder vertical Rate would increase the amount of time to make a decision and also increase the amount of error of that decision, is accepted. There is strong evidence to show that an increase in the vertical Rate resulted in the subjects waiting longer to make a decision and then, being further from the actual distance. One possible explanation for the significance in Time and ERROR with respect to vertical Rate is that the subjects were not used to being involved with aircraft capable of climbing at 2000+ feet/minute due to their general aviation background (general aviation aircraft typically climb at less than 1000 ft./min.). The fact that Rate was found to be significant is more likely due to the process the subjects used to calculate a Miss distance rather than to their past flying experiences. The task for each scenario involved calculating a Miss distance by watching the relative altitude in the intruder's datatag and projecting what this value would read when intruder passed ownship. The subjects may have required more time and had more error at the higher vertical rates because the relative altitude in the datatag made larger changes. This seems logical when the 1500 ft./min. Rate is examined closely. The relative altitude in the datatag changed 100 feet every time the intruder/datatag updated ((1500 ft./min.) / (60 sec./min.) * (4 sec. update rate) =100 foot change). The ease with which the subjects could predict what the successive relative altitudes possibly explains why the 1500 ft./min. Rate was significant for ERROR and Time. If the change had been 99 feet or 101 feet, the change would not have been as obvious and the outcome may have been different.

The last main effect was Angle. The intruders that approached from 50° had significantly more ERROR than those that approached from 0°. This is consistent with previous research which found significant increases in error as intruder approach Angle increased. This is hard to explain because the intruder still flies straight at ownship.

There were two other first-order interactions which were significant. Since the interactions are harder to interpret than the main effects, tests of simple effects were performed to make sense of these results. Miss by Angle was the only significant interaction for Time. Like the main effect, the 600 and -600 foot Miss distances were the fastest, but Angle had an interesting interaction in that +600 feet was significantly faster at 0° and -600 feet was significantly faster at 50°. There is no easy explanation for this.

The final significant interaction was Rate by Angle. This is a compilation of the main effects of Rate and Angle, namely high vertical Rates and the 50° approach Angle result in the most ERROR. The Angle seems have the most effect at 2500 ft./min. The 1500 ft./min. Rate may also have been affected by the fact that the change in relative altitude was easy to project. This would explain why 1500 ft./min. ERROR was less than the ERROR for 1000 ft./min.

Abbott, T. S., & Moen, G. C. (1981). Effect of display size on the utilization of traffic situation display for self-spacing task (NASA TP-1885). Hampton, Virginia: NASA. (NTIS No. N81-29109).

Anderson, R. E., Curry, R. E., Weiss, H. G., Simpson, R. W., Connelly, M. E., & Imrich, T. (1971). Considerations for the design of an onboard air traffic situation display. In G. A. Bekey & C. Janow (Eds.), Seventh Annual Conference on Manual Control (pp. 61-72). Los Angeles, CA: NASA.

Chappell, S. L., & Palmer, E. A. (1983). Primary separation between three aircraft using traffic displays. In A. T. Pope & L. D. Haugh (Eds.), Proceedings of the Human Factors Society - 27th Annual Meeting (pp. 767-771). Norfolk, Virginia: Human Factors Society.

Ellis, S. R., McGreevy, M. W., & Hitchcock, R. J. (1987). Perspective traffic display format and airline pilot traffic avoidance. Human Factors, 29, 371-382.

Hart, S. G., & Loomis, L. L. (1979). Evaluation and prediction of the vertical situation from a horizontal situation display. Proceedings of the 19th IEEE Conference on Decision and Control (pp. 997-1001). New York: New York Institute of Electrical and Electronic Engineering.

Hart, S. G., & Loomis, L. L. (1980). Evaluation of the potential format and content of a cockpit display of traffic information. Human Factors, 22, 591-604.

Jago, S., Baty, D., O'Conner, S., & Palmer, E. A. (1981). Effect of display update interval, update type, and background on perception of aircraft separation on a cockpit display of traffic information (NASA TM-81171). Moffett Field, CA: NASA. (NTIS No. N81-29127).

O'Conner, S., Jago, S., Baty, D., & Palmer, E. A. (1980). Perception of aircraft separation with pilot-preferred symbology on a cockpit display of traffic information (NASA TM-81172). Moffett Field, CA: NASA. (NTIS No. N80-31397).

Palmer, P., Jago, S., Baty, D., & O'Conner, S. (1980). Perception of horizontal aircraft separation on a cockpit display of traffic information. Human Factors, 22, 605-620.

Rooney, B. H. (1992). Effect of intruder vertical rate on pilot perception of separation on a cockpit traffic display. Unpublished master's thesis, Embry Riddle Aeronautical University, Daytona Beach, FL.