Interacting with Computers 17 (2005) 690–710
www.elsevier.com/locate/intcom
Age differences in trust and reliance of a medication
management system
Geoffrey Ho1, Dana Wheatley, Charles T. Scialfa*
Department of Psychology, University of Calgary, 2500 University Drive NW, Calgary, AB, Canada T2N 1N4
Available online 6 October 2005
Abstract
The present study examined age differences in trust and reliance of an automated decision aid. In
Experiment 1, older and younger participants performed a simple mathematical task concurrent with a simulated medication management task. The decision aid was designed to facilitate medication management, but with varying reliability. Trust, self-confidence and usage of the aid were measured. The results indicated that older adults had greater trust in the aid and were less confident in their performance, but they did not calibrate trust differently than younger adults. In Experiment 2, a variant of the same task was used to investigate whether older adults are subject to over-reliance on the automation. Differences in omission and commission errors were examined. The results
indicated that older adults were more reliant on the decision aid and committed more automation- related errors. A signal detection analyses indicated that older adults were less sensitive to
automation failures. Results are discussed with respect to the perceptual and cognitive factors that
influence age differences in the use of fallible automation. q 2005 Elsevier B.V. All rights reserved.
Keywords: Automation reliability; Aging
1. Introduction
The automation of complex tasks has been a successful solution for reducing human errors in many domains, including aviation, medicine, and process control. Advances in hardware, software, and machine learning have made it possible for computers to facilitate
* Corresponding author. Tel.:C1 403 220 4951; fax: C1 403 282 8249.
E-mail addresses: geoffrey.ho@honeywell.com (G. Ho), scialfa@ucalgary.ca (C.T. Scialfa).
1
New affiliation information for Geoffrey Ho is Honeywell Laboratories, 3660 Technology Dr, MN65-2200,
Minneapolis, MN 55418, USA.
0953-5438/$ - see front matter q 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.intcom.2005.09.007
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
691
or take over many manual tasks, making automation commonplace in everyday living.
In the foreseeable future, computers will learn from their past experience, be aware of their
environment, and make decisions for us.
This trend towards greater automation parallels our aging population. It is inevitable that older adults will be interacting with computers in some fashion. In fact, technologies are being developed specifically to help older adults live independently by automating tasks that they find difficult to perform (Bouma, 1998). Automation can facilitate both physical tasks and cognitive tasks such as memory and decision-making (e.g. Jonsson et al., 2005). Yet, there is little research investigating how older people respond to
automation when it fails.
In the present experiments, older and younger adults were given a simulated medication task while concurrently performing a distracting mathematics task. An automated decision aid facilitated the medication task, but with varying reliability. Using this task, we
attempted to answer several questions about the use of automation by the elderly. Do older adults differ from younger adults in their trust of automated decisions aids? Are there age differences in the calibration of trust? Are older users more likely to rely on error-prone automation? In broad terms, these studies explore the cognitive dynamics between the elderly and automated devices and provide important data on an issue that will be critical
for seniors in the near future.
1.1. Trust and automation
Generally, automation is reliable and errors are infrequent, but no system is perfect. When automation errors occur, operators must be diligent enough to detect the error and initiate procedures to correct them. Too much trust leads to complacency, whereas distrust leads to abandonment of beneficial technology. Thus, the trust and reliance an operator
gives to an automated system is central to safety and productivity.
The primary theoretical account of trust (Muir, 1987; 1994; Muir and Moray, 1996) in automation borrowed heavily from social psychological views of interpersonal trust (Barber, 1983; Rempel et al., 1985) to explain trust in automation. Subsequent tests
supported many of the basic tenets of Muir’s theory, namely that the system’s predictability,
dependability and the user’s faith in it were critical to the calibration of trust.
Since Muir’s seminal work, over a decade of research has shown that trust is only one (albeit critical) mediating factor that influences automation reliance. Several factors are now known to influence trust itself, including the frequency and magnitude of system errors (Lee and Moray, 1992), and the user’s culture, personality, and affect (Lee and See, 2004). Other factors interact with trust to determine automation usage, of which self- confidence and mental workload have received a great deal of attention (Lee and Moray,
1994; Riley, 1996).
There are many reasons to believe that, compared to younger adults, older adults will respond differently to automation and to changes in the reliability of that automation. Although partially a cohort effect, older adults are more apprehensive about using
computers and have more trouble learning computer programs (Czaja and Lee, 2001). As a result, one might expect that they would be less trusting of automation. However, age- related deficits in cognitive abilities can cause individuals to experience greater workload
692
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
and lessened self-confidence when performing a task manually. These changes can induce greater trust and reliance on automation.
Only a few studies have investigated trust and reliance of automated decision aids with
the elderly and the results have been mixed. Lee et al. (1997) compared older and younger
drivers using an Advanced Traveller Information System (ATIS) that presented warning messages to alert drivers of upcoming traffic events. They found that older adults trusted the information provided by the ATIS more than younger adults. Other studies have shown
that the elderly alter their behaviour like the young in the face of changing reliability (Fox
and Boehm-Davis, 1998), but in contrast, Sanchez et al. (2004) reported that the elderly trusted automation equally, but they may be more sensitive to changes in the reliability of
automation.
These studies are laudable, but they primarily focused upon driving. Yet the dominant technologies geared for seniors are medical technologies that manage and facilitate older adult care. Furthermore, none of the previous studies have addressed the psychological reasons why older adults trust and rely on automation differently. In the present study, we compared younger and older adults in their use of an automated medication management task. In order to simulate the divided-attention conditions that are common and problematic in the lives of the aged, medication management was carried out while simultaneously performing a simple math task. Across two experiments, we found that older adults can calibrate trust in fallible automation much like their younger counterparts. They also rely on the automation to a greater degree than the young and make more errors under low-reliability conditions. These deficits appear to result from decreased sensitivity to errors in the system. Our results add to the literature by describing some limitations to the use of automation with seniors and additionally, provide a more detailed explanation of
why older adults react to these technologies differently than younger adults.
2. Experiment 1
In Experiment 1, age differences in the calibration of trust in automation were
examined using a dual task consisting of a mathematics task and a simulated medication
management task (Fig. 1). To help participants perform the medication task, two sources
of information were provided; medication ‘labels’ that revealed the correct prescribed time and dosage for medications and a timer to keep track of when to take them.
Alternatively, participants could rely on the automated medication manager (AMM) to determine when to take a particular medication and its dosage. Thus, if the AMM was 100% reliable, there was no need to check the labels or the time. However, the AMM’s reliability varied. If the user decided to check the labels or the timer, it was inferred to be a sign of distrust. If the AMM was not used, participants were instructed to turn off the
device. This was done in order to obtain usage preferences.
It was predicted that decreased reliability would reduce trust, increase monitoring of labels and time information, and increase mental workload. Furthermore, according to Lee
and Moray (1994), participants are expected to use the AMM whenever their self-
confidence exceeded their trust. With respect to age differences, we predicted that older adults would show greater trust (i.e. higher subjective ratings of trust, fewer timer
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
693
Fig. 1. (a) An example of the computer display for Experiment 1 and (b) Experiment 2.
694
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
and label checks), lower self-confidence, and a greater propensity to continue using the
device. Last, because previous studies have found that initial trust has a greater impact on older adults (Kantowitz et al., 1997), it was predicted that they would be less willing to
change trust once an initial level had been established (i.e. an inertia effect).
2.1. Method
2.1.1. Participants
Thirteen (4 male and 9 female) younger (MZ20.73 yrs, SDZ1.68 yrs) and 12 (3 male
and 9 female) older participants (MZ66.93 yrs, SDZ6.29 yrs) were recruited from the University of Calgary and the Calgary community. Everyone was given $15 (CDN) for each experimental session. Participants were screened for education, general health, cognitive impairment, and visual health. Both age groups reported good health and both
age groups had similar years of education (pO0.05). Younger adults (MZ29.75,
SDZ0.62) scored lower than older adults (MZ28.75, SDZ0.87) on cognitive impairment
(pZ0.004), as measured by the Mini-Mental State Examination (MMSE; Folstein et al., 1975), but everyone scored within the normal range for their age group (Crum et al., 1993).
Relative to older adults (MZ1.09, SDZ0.46), younger adults (MZ1.49, SDZ0.44) had
significantly better static acuity (pZ0.04) and there was a trend for younger adults to have better contrast sensitivity (cZ0.06). This is not uncommon (Kline and Scialfa, 1997) and with correction, all participants had good vision. Computer anxiety was low for both
2
groups and not significantly different (pZ0.10), however, younger adults used computers more frequently and used a greater variety of programs.
2.1.2. Stimuli and apparatus
The dual-task was presented on a desktop PC that used a 17monitor, running on
Windows 2000. Static acuity was measured using Post-Script Landolt Cs with eight targets for each level of minimum angle of resolution (MAR), which range from 0.258 to 0.088. Contrast sensitivity was measured using a Vistech near contrast sensitivity chart at 50 cm. Computer anxiety was measured using the Computer Anxiety Rating Scale (CARS) (Heinssen and Knight, 1987). The computer knowledge survey asked about usage,
00
ownership, and familiarity with software applications (see Laberge et al., 2002). Cognitive function was measured using the MMSE (Folstein et al., 1975) and subjective workload estimates were obtained with the NASA-TLX (Hart and Staveland, 1988). Trust and self-
confidence were measured using a 10—point Likert scale and questions similar to those
found in previous work (Lee and Moray, 1994; Muir and Moray, 1996).
2.1.3. Description of simulation
An example of the interface is provided in Fig. 1. Math questions appeared in the upper left corner and four multiple-choice radio-buttons were provided to answer each question. Four icons of pill bottles represented the medications and were labelled Pill A–D. Each icon could be clicked and each click simulated taking one pill. Beneath each pill icon were medication labels that contained information about the appropriate dosage and the time to take the medication. The label information was only revealed when it was clicked for 5 s. This was done to simulate an individual having to re-read the medication label on a bottle to
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
695
refresh their memory on how to take it. Clicking the label was a method to evaluate distrust in the AMM since there was no need to click the label if the AMM was 100% reliable.
Whenever it was time to take a particular pill, the AMM flashed a yellow message indicating which pill needed to be taken and the dosage. The message remained visible until the proper dosage was taken or until the 20 s had elapsed. A power button for the AMM was provided underneath the AMM display, which allowed participants to activate
or deactivate the AMM as they wished.
The time was made available to participants to help them judge when to take a pill. A timer began each block at 0:00 and counted in seconds up to 12 min and 20 s. The time was also hidden from participants and was only revealed for 5 s when they clicked the timer button. Performance on both tasks was measured using an overall score. The score itself
was based upon the points system for each task, which is discussed below.
The math task was chosen to divert attention from the medication task, while simulating a variety of activities that older adults commonly perform (Czaja, 1997).
Participants were given 10 s to answer each math question. Visual feedback was provided
regarding the accuracy of each response.
In the medication task, participants were required to keep track of three key pieces of information for this task: (a) which medication to take, (b) when to take each medication, and (c) how many pills to take. At the appropriate times, they had to click the appropriate medication icon to simulate that they had taken a pill. For instance, if for Pill A, the label instructed participants to ‘Take 2 pills at 3 and 6 min’, they were required to interrupt the math task temporarily, and click the Pill A icon twice at 3 min and at 6 min. A 20 s window was provided, such that they could take a medication correctly 5 s before the specified time and up to 15 s after the specified time. Four error types could occur: (a) failure to take a required medication within the 20 s window, (b) taking a medication outside this 20 s
window, (c) taking the wrong medication and (d) taking the incorrect dosage.
According to Moray and Inagaki (2000), participants need to know explicit payoff structures in order to develop a monitoring strategy. Therefore, prior to the experiment, participants were given the payoff structure of the tasks and instructed to adhere to this
structure to maximize their score. We decided to use a point system that rewarded both tasks
equally when done correctly, but one that would stress the importance of taking one’s
medication correctly. In the math task, 5 points were given for correct responses and 5 points
were deducted for incorrect responses. In the medication task, 5 points were given for
correctly taking medications and 20 points were deducted for incorrectly taking medications.
2.1.4. Procedure
The experiment was divided into three (usually non-consecutive) days. Training was conducted on Day 1 and on the subsequent days, the experiment proper was carried out. On Day 1, all questionnaires and health screening was completed prior to computer training.
Subsequently, a paper copy of the interface was provided and the task was described verbally.
Then participants performed four training blocks. In the first block, the math task was trained alone. In the second block, the medication task alone was trained. On the third block, both tasks were performed together, but without the help of the AMM. On the last block, all components were presented, including the AMM. This hybrid hierarchical part-task/whole task training was done to help participants, particularly older participants, learn the task
696
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
(Kramer et al., 1995). During the third and fourth block, participants were told that both tasks
were important, but emphasis should be placed on taking one’s medication correctly. During
training, when the AMM was activated, it was always 100% reliable.
On Days 2 and 3, participants performed the dual-task with the AMM initially activated. Four 12 min blocks were given on each of the days and each day, a new medication regimen
was provided. Prior to beginning, older participants were given an additional practice session.
All participants were told that the AMM was not 100% accurate. When AMM failures occurred, it alerted the participant to take a medication 30 s after the correct time. Errors occurred randomly, but the AMM reliability was set to be both constant and high (87.5% accuracy) or variable and low (50–67.5% accuracy). The high-constant and low-variabl
reliability conditions were counterbalanced across the 2 experimental days. Participants were
also told that the AMM could be activated or deactivated with the power button and were
instructed to deactivate the AMM if they felt it was not helpful.
After each block, participants were given their scores, which they were instructed to maximize. As well, after each block, participants were asked to fill out the NASA-TLX and the trust/self-confidence questionnaire based on their experience with the AMM on the
preceding block.
2.2. Results
The variables were analyzed by submitting each variable separately to a 2 (Age) X 2 (Reliability) X 2 (Presentation Order) split-plot analysis of variance (ANOVA). For brevity, only marginal and significant effects are reported for the variables central to the hypotheses. Overall task performance as measured by the score and the number of errors suggested that younger adults outperformed older adults on both tasks and the reliability of the AMM affected performance on the medication task, but not the math task. Moreover, improvement was generally greater for those in the Low–High order relative to the High–
Low order. These data will not be reported in any more detail.
Specific hypotheses were tested using mean trust, self-confidence, workload, and the frequency of label and timer monitoring as dependent measures. The relationship between trust, self-confidence and automation usage was analyzed by taking the difference score between trust and self-confidence and plotting these scores with automation usage (Lee
and Moray, 1994), calculated as the total time per block that the AMM was activated.
Neither group trusted the AMM very much, but subjective ratings of trust were
sensitive to varying reliability. Shown in Table 1, trust was rated higher under high reliability relative to the low reliability, F (1, 20)Z6.79, pZ0.009. Older adults rated trust higher than younger adults, F (1, 20)Z7.39, pZ0.007. The interaction between age and
reliability was not significant; thus the hypothesis that older adults would exhibit a greater
inertia effect was not supported.
Age differences were also apparent in ratings of self-confidence and workload.
Older adults were less confident in their abilities to perform the task without the AMM,
F (1, 19)Z8.11, pZ0.01. There was an Age!Reliability interaction with workload, F (1, 20)Z8.81, pZ0.008. For younger adults, workload was reduced when AMM
reliability was high, however, it remained constant for older adults. Reliability also interacted with presentation order. While workload generally decreased on Day 3,
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
697
Table 1
Means and standard deviations for all ratings of trust, self-confidence, workload in Experiment 1. Variable
Age
Order
High reliability Mean Trust
Young
SD 2.25 0.96 2.71 2.45 1.59 0.59 1.34 2.58 2.54 2.00 1.31 2.73 32.07 14.90 29.95 18.22 42.40 21.31 23.71 29.79
Low reliability Mean 2.11 2.30 4.48 5.43 7.97 8.68 6.44 6.04 5.04 7.90 6.25 7.31 48.61 61.20 53.94 33.85 50.43 91.45 26.44 49.94
SD 1.28 1.45 3.51 3.09 2.25 1.11 2.28 2.62 2.34 1.75 1.59 2.54 37.81 17.94 54.52 18.33 44.66 22.50 32.78 32.97
Old
Self-confidence
Young
Old
Workload
Young
Old
Timer checks
Young
Old
Label checks
Young
Old
High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High High–Low Low–High 3.25 3.30 5.08 5.86 8.81 9.25 6.56 6.08 5.14 4.68 7.06 7.00 49.46 63.90 38.31 36.00 57.11 79.45 41.33 46.38
the reduction was greater for those participants who received the low reliability condition
first. The three-way interaction was marginally significant, F (1, 20)Z3.27, pZ0.086. This effect can be explained by the greater reduction in workload experienced by younger
adults who operated under low reliability first.
If trust in the AMM is low as indicated by the subjective ratings, we would expect a high degree of monitoring (Masalonis, 2000; Muir and Moray, 1996). This was indeed the case.
Younger adults checked the time on average every 13 s, whereas older adults checked the time
every 16 s. Checking label information was also very frequent. Younger adults checked a
label every 11 s and older adults checked a label every 17 s. Younger adults performed more
label checks than older adults, F (1, 20)Z4.45, pZ0.024. While age differences in trust may
be the cause of this difference, it may also be that younger adults were faster at checking labels
and thus could check more labels in a given amount of time.
Although the data suggest that participants did not like the AMM’s performance, they left it activated throughout most of the experiment. Several reasons for this are discussed below. Nevertheless, as shown in Fig. 2, automation usage declined as the disparity
between trust and confidence became large (Lee et al., 1997; Lee and Moray, 1994).
2.3. Discussion
The data from Experiment 1 indicate that subjective ratings of trust were positively correlated with reliability. This replicates previous studies (e.g. Lee and Moray, 1992; Muir and Moray, 1996; Wiegmann et al., 2001), that trust can be calibrated to
698
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
Fig. 2. The relationship between AMM activation and trust minus self-confidence for (a) younger adults and (b) older adults in Experiment 1.
the reliability of the automation. However, trust was rated generally to be low relative to the actual accuracy of the AMM and this low self-reported trust in the AMM is consistent with the high frequency of monitoring time and label information. Other studies have
found similarly low trust relative to the automation’s reliability (Lee and Moray, 1992; Moray et al., 2000) and it has been suggested (Lee et al., 1997; Lee and See, 2004;
Muir, 1987) that there might be a bias towards distrust in risky behaviours that is difficult to overcome.
Despite low trust, the amount of time users left the AMM activated was high. In fact, most participants did not turn it off. The prediction that the AMM would not be used once self-confidence was greater than trust was only partially supported. Deactivation of the AMM occurred only when the discrepancy between trust and self-confidence was greater
than 5 units. Thus, unlike the results reported in previous work (Lee and Moray, 1994;
Lewandowsky et al., 2000), participants continued using the AMM or at least felt it was
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
699
not sufficiently disruptive to turn off, even though their self-confidence exceeded their trust considerably.
The reasons participants did not deactivate the AMM may be simply a failure in our manipulation. Participants may not have realized that deactivating the AMM was an option or the demand characteristics of the experiment may have led them to think that deactivating the AMM would be frowned upon. They may also have been waiting for it to
become more useful in the future and thus chose to leave it activated.
Alternatively, there may have been unforeseen advantages of keeping the AMM activated. In previous work (e.g. Itoh et al., 1999; Lee and Moray, 1992), keeping faulty automation activated did not provide any advantage. In the present study, even if the
AMM failed to notify participants to take one specific medication, subjects may have used this cue to check all their medications, since they knew that medication events often co- occur. This would make it advantageous to keep the AMM activated even though it was performing incorrectly. In fact, some participants commented that although the AMM was
unreliable, ‘it was better than nothing’.
Turning attention to age differences, predictions that older adults would be less
confident, more trusting of and more reliant on the AMM were partially supported. Older adults were less self-confident and ratings of trust were greater than in their younger counterparts. This greater tendency to trust automation concurs with previous findings (Fox and Boehm-Davis, 1998; Lee et al., 1997). Older adults also rated their workload to be higher than younger adults. However, contrary to the hypothesis, older adults calibrated
trust similarly to younger adults and did not show any greater inertia effects.
Why are older adults more trusting of automation? This may simply reflect the cohort and cultural differences of the elderly and the more computer-savvy young. Alternatively, because they are less confident in their abilities, they may see automation as taking a supervisory role and expect it to provide more knowledgeable directives (Skitka et al., 1999). As well, because older adults experience greater workload relative to younger
adults, they may be more prone to use automation as a heuristic (Skitka et al., 1999). These
two latter arguments will be examined further in the general discussion.
Given that older adults assigned greater trust to the AMM, it was expected that they would have a greater tendency to use the AMM and to rely on its decisions. This was not supported by the data. Usage patterns appear to be similar for both young and old. Although older adults made more medication errors in general, the Age x Reliability
interaction was not significant. Both younger and older individuals equally benefited from the greater accuracy of the AMM in the high reliability condition. Overall this suggests that although trust and self-confidence differ for older and younger users, this did not affect
their reliance on the AMM.
3. Experiment 2
The lack of age differences in usage of the AMM in Experiment 1 may reflect the failure to effectively capture reliance. After all, simply because someone leaves a system activated does not necessarily suggest they were using the system. Thus, in Experiment 2, age differences in reliance were more closely investigated by examining the errors
700
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
participants made. Specifically, we were interested in complacency errors (Parasuraman et al., 1993) and automation bias errors (Skitka et al., 1999).
Automation-induced complacency (Parasuraman et al., 1993) is a dysfunctional drop in monitoring associated with unjustifiably high trust (Billings et al., 1976; Metzger and
Parasuraman, 2001; Wiener, 1981). Once automation is used, monitoring the automation becomes the operator’s primary role and if monitoring is insufficient, automation errors are missed. For example, Parasuraman et al. (1993) reported that observers detected 82% of automation failures in a variable reliability condition, but only 33% in a constant, high reliability condition. They concluded that complacency with reliable automation leads
observers to reduce the allocation of attention to the monitoring task.
Skitka et al. (1999) argued that complacency is not simply a monitoring problem, but a
decision bias to follow automation directives. Their study demonstrated that subjects not only
missed events because their automation failed to detect an event (omission error), but that they
also followed incorrect automation directives (commission errors). They argued people use automation like a heuristic and like heuristics, they generally work, but will also occasionally
fail. Regardless of the explanation, it is clear that people can rely too much on automation.
It is hypothesized by some that age-related deficits in attention result in greater complacency (Hardy et al., 1995), particularly in cognitively demanding situations
(Vincenzi and Mouloua, 1999). However, this explanation is based on studies which have
two shortcomings. First, they fail to provide a thorough account of the cognitive processes involved in complacency. Second, they generally do not measure trust, excesses of which
are a necessary precursor for complacency (Billings et al., 1976; Metzger and Parasuraman, 2001; Wiener, 1981).
In Experiment 2, using the same task as in Experiment 1, two main questions were examined: (a) are older adults more susceptible to automation reliance errors and (b) is
automation reliance in older adults a result of poor attentional allocation strategies?
With a few exceptions, the design of the study was identical to Experiment 1. The
AMM operated under two conditions of reliability, a high (90%) reliability condition and a low (66%) reliability condition. When the AMM committed errors, it either failed to notify the participant of a medication event, or it instructed the participant to take an incorrect dosage. If the AMM failed to alert a medication event and the participant did not correct
this, an omission error was recorded. If the AMM provided an incorrect directive and
the participant followed the directive, a commission error was recorded. According to Skitka et al. (1999), these errors are the trademark of automation bias, suggesting that the participant is relying too much on automated decisions. It was predicted that the
proportion of omission and commission errors would be greater in high reliability relative to low reliability (i.e. complacency effect) and that older adults would make more errors of
commission and omission relative to younger adults.
In addition, the 12 most appropriate items of the 20-item Complacency Potential Rating Scale (CPRS; Singh et al., 1993) were adapted for an older audience and administered prior to the experiment. The CPRS has been shown to adequately predict one’s potential for complacent behaviour. It was predicted that older adults would score higher, consistent
with more reliance on automation, than younger adults.
As in Experiment 1, participants were provided with a payoff structure-based on the accuracy of the math task and the medication management task. The only change in this
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
701
payoff structure was that we included a negative two points each time a participant checked their labels in order to emphasize the costs associated with monitoring.
3.1. Method
3.1.1. Participants
Twelve (5 male and 7 female) younger (MZ22 yrs, SDZ3.79) and 12 (4 male and 8 female) older participants (MZ67.67 yrs, SDZ4.38) participated in Experiment 2. The Wechsler Adult Intelligence Scale (WAIS) digit span sub-test (Wechsler, 1997) and the CPRS were added to the screening of protocol that was otherwise identical to Experiment 1. Most participants were recruited from the University of Calgary or the surrounding Calgary community. Younger adults (MZ1.62, SDZ0.30) had better static acuity than older adults (MZ1.11, SDZ0.30), pZ0.001. Young adults (MZ7.92, SDZ1.86) also scored marginally better on the WAIS digits backward task relative to older adults, (MZ6.50, SDZ1.57), pZ0.06. Younger adults used a larger variety of computer programs, although there were no differences in computer anxiety (pO0.05) or the CPRS
(pO0.05), a finding that did not support our hypothesis.
3.1.2. Stimuli and apparatus
The computer simulation was modified a number of ways from Experiment 1. Because the power button was removed, the AMM was always activated and the screen interface changed slightly (See Fig. 1b). Also, the time was always visible. Furthermore, checking the label resulted in a 2-point deduction. This change to the point system was added to penalize unnecessary monitoring which detracts from performing secondary tasks. In the present case, the medication regimen changed after each block. Each medication could have a dosage of 1–3 pills, and the medications could be taken 2, 3, or 6 times during the block. The duration of each block was 6 min, plus any time that was needed for
participants to complete their medication task.
3.1.3. Procedure
With the exception of the additional screening tests (i.e. WAIS and CPRS), the
procedure of Day 1 was identical to Experiment 1. The procedure on subsequent days was also identical to Experiment 1 except that participants were given six blocks of the experimental task. As well, on the last block of Day 3, prior to completing this
questionnaire, they were asked to perform a recall task of their medication regimen during the last block. This was deemed important since it was likely that younger adults could hold medication information in working memory and thus be more able to monitor the
AMM without checking their labels.
One critical change was made in the protocol of this study. In Experiment 1, some of the older adults who performed under low reliability first reported considerable stress and as a result, attrition was high. Thus, for ethical reasons, it was decided to present the high reliability condition first for all observers. Although this fixed order may confound effects, the results from Experiment 1 suggest that order effects were minimal. For instance, when the data for Experiment 1 were re-examined to investigate whether participants made medication errors following an AMM failure, order effects were negligible (hZ0.02).
2
702
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
On other variables, where order did have an effect, such as overall performance score, the
order of our present design actually would bias the results in the opposite direction of our
hypotheses. That is, practice would serve to reduce our effects.
3.2. Results
Trust, self-confidence, and workload were measured in a similar manner to Experiment 1. Errors made as a result of automation failures were captured by the number of commission
and omission errors. Monitoring behaviour was measured by taking measures of d0and b (Green and Swets, 1974). All measures were submitted to an Age (2)!Reliability (2) univariate, split-plot ANOVA.
In general, younger adults performed better than older adults and under low reliability, older adult scores declined, but this was not true for younger adults. In the math task,
younger adults outperformed older adults. As well, errors decreased in the math task when reliability was low, likely suggesting a practice effect. In the medication task, younger adults made fewer errors relative to older adults and only the elderly made more errors in
the low reliability condition, suggesting inappropriate reliance on the AMM.
3.2.1. Subjective ratings of trust and self-confidence
Subjective ratings of trust as a function of age and AMM reliability are also shown in
Table 2. Similar to Experiment 1, older adults rated their trust in the AMM higher than
younger adults, F (1, 22)Z3.57, pZ0.036. There was also a reduction in ratings of trust
when reliability decreased, F (1, 22)Z28.91, p!0.001, suggesting that participants were sensitive to change in reliability.
Younger adults were more confident in their abilities than older adults, F (1, 21)Z5.12,
pZ0.016. Counter to expectations, there was a marginal increase in self-confidence with decreasing reliability, F (1, 22)Z3.57, pZ0.072, likely was the result of practice effects.
Table 2
Means and standard deviations for all variables in Experiment 2. Variable
Age
High reliability Mean 5.69 7.40 7.28 5.84 4.37 5.07 15.29 52.21 14.58 66.36 3.72 0.53 2.45 0.24
Trust
Self-confidence
SD 2.12
2.84 1.99 1.83 1.49 1.55
Low reliability Mean 3.62 5.19 7.83 6.11 3.96 5.47 7.01 29.58 5.21 49.68 4.42 1.00 1.12 0.10 SD 1.46
2.67 1.88 1.14 1.40 1.42
Workload
Commission errors
0. d0
b
Young Old Young Old Young Old Young Old Young Old Young Old Young Old 17.75 35.58 31.00 40.01
1.56 1.03 1.87 1.54 14.71 30.89 14.56 37.90
1.40 0.97 2.06 0.96
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
703
3.2.2. Workload
Older adults rated their workload to be greater than younger adults, F (1, 22)Z3.58, pZ0.036. The main effect of reliability on workload was not significant. For younger
adults, workload decreased in the low reliability condition; the opposite effect was seen in
older adults, F (1, 22)Z9.07, pZ0.006.
3.2.3. Automation reliance errors
Commission errors reflect errors made when the AMM provided incorrect
dosage information. Older adults made more commission errors relative to younger
adults, F (1, 21)Z10.93, pZ0.002 and commission errors was more likely in the high reliability relative to the low reliability condition, F (1, 21)Z7.04, pZ0.008. Omission errors were also more common for older adults, F (1, 21)Z15.13, p!0.001 and were greater in the high reliability condition relative to the low reliability condition, F (1, 21)Z 6.41, pZ0.01, suggesting that participants were more reliant upon the automation when it
was operating under high reliability.
3.2.4. Monitoring behaviour
Monitoring behaviour was examined using a signal detection paradigm that incorporated the payoff matrix of the experiment. Sensitivity was assessed using higher values indicating greater sensitivity. Younger adults were more sensitive to automation failures relative to older adults, F (1, 21)Z45.64, p!0.001. Moreover, d0was higher under low reliability, F (1, 21)Z7.40, pZ0.013. This may be the result of practice
d0,
or reliability effects.
Response bias was examined using b. Under high reliability, optimal b was determined to be a conservative 2.52 (see MacMillan and Creelman, 1991, on determining optimal b). In contrast, under low reliability, optimal b was a more liberal 0.54. For younger adults, under high reliability, average b was significantly larger than optimal b, t (11)Z 2.71, pZ0.04. This supports the hypothesis that younger adults are complacent under high reliability automation. For older adults, the opposite results were found. Under high reliability, the response bias for older adults was not statistically different from the
optimal. Under low reliability, older adults did not monitor enough, t (10)Z2.73, pZ0.02.
3.2.5. ROC plots
Perhaps the best method to depict monitoring performance is in receiver operating characteristic (ROC) curves (Fig. 3a and b). Points close to the upper left corner
characterize excellent performance, while chance performance lies along the positive diagonal. It is evident in these figures that younger adults were very sensitive to
automation failures and there was a slight tendency for them to shift their bias to be more liberal under low reliability. Bias for older adults varied considerably. Some older adults did little monitoring and this affected their ability to detect automation failures. Other older adults monitored the labels often and as a result, they detected many of
the automation failures, but also made many false alarms. Regardless of bias, sensitivity
for older adults was considerably poorer than in their younger counterparts.
704
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
Fig. 3. Performance of (a) younger and (b) older adults plotted in ROC space under high (left) and low (right) reliability in Experiment 2.
3.3. Discussion
The results of Experiment 2 suggest that both age groups were susceptible to
automation reliance effects. More commission and omission errors were made in the high reliability relative to the low reliability condition. This supports the data from previous studies involving complacency (Parasuraman et al., 1993), and automation bias effects (Skitka et al., 1999). As previously found by Vincenzi and Mouloua (1999) under high workload conditions, older adults were more susceptible to automation failures than younger adults. In general they made more commission and omission errors relative to younger adults. That is, older adults often missed events if they depended on automation to notify them of the event and if the automation provided an incorrect directive, older adults
were more likely to follow the directive.
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
705
Do these data suggest that older adults exhibit poorer attention allocation? The results
are difficult to interpret. In this experiment, older adults actually did more monitoring. As well, older adults had a more liberal response bias relative to younger adults. That is, older adults had a greater tendency to monitor their labels and check whether the automation was failing. Despite this, older adults made more errors. What is evident from the d0data and the ROC plots is that older adults demonstrated considerably poorer sensitivity to automation failures relative to younger adults. As a consequence, even if older adults monitored optimally, they would have considerably more automation-related errors
relative to a younger adult.
An explanation may lie in age differences in working memory. Memory for the
medication regimen is an important index of task demands because if observers were able to hold accurate mental models of the information that needs to be monitored, then there is little need to perform overt monitoring (Moray, 1981). Thus, a recall test of the pill
regimen was measured at the end of Day 3. Recall accuracy for younger adults (97.92%) was significantly better than older adults (53.13%), t (22)Z5.23, p!0.01 and a post hoc correlational analyses found that memory for the medication regimen was highly correlated with errors of commission (rZK0.71), omission (rZK0.69), and with
(rZ0.66) and explained much of the variance between age groups.
This suggests that younger participants were able to accurately hold medication
regimen in working memory and as a result, did not need to perform any overt monitoring of label information. This would explain their high sensitivity and their low response bias. For older adults, working memory deficits restricted them from using this strategy. This inability to use working memory resulted in a decreased sensitivity in automation failure detection. In order to compensate for their memory, they had to use the labels to check the AMM information that resulted in a greater response bias. For the most part though, they
still did not monitor enough.
Therefore, inappropriate attention allocation is only one factor that plays a role in automation reliance errors. The results suggest that automation reliance effects in older adults reflect deficits in working memory (at least in this study). The questions surrounding automation reliance and older adults become even more complex when trust, workload, and attitudes towards automation are considered. Older adults rated trust subjectively higher than the young in both Experiments 1 and 2, suggesting that they should be more complacent. As well, because cognitive demands are generally greater for aging adults, it is also more likely that they would have a greater reliance on automation. Yet, the results from the CPRS suggest older adults in our sample had no prior attitudes that would yield
greater complacency.
4. General discussion
The primary concern of Experiment 1 was to answer to the following questions: (a) Do older adults differ from younger adults in their trust of automated decision aids and: (b) Are there age differences in the calibration of trust? The results suggested that older adults do tend to trust decision aids more than younger adults, but they do not calibrate trust differently. This higher trust in automated aids leads to concerns when the reliability d0
706
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
of the aid is imperfect. Experiment 2 examined this issue and answered two additional
questions: (a) Are older adults more susceptible to automation reliance errors relative to younger adults and if so, (b) Is automation reliance in older adults a result of poor
attentional allocation strategies? The results indicated that older adults exhibited a greater automation reliance effect. In fact, older adults exhibited greater trust, and committed more errors of omission and commission, suggesting that they were more complacent. However, the answer to the second question is still unclear. It appears that attention
allocation can only offer a limited explanation of the data.
Poor attentional allocation as a result of too much trust has been presumed to be at the
heart of complacency effects. When trust is high, monitoring should decrease (Muir and Moray, 1996; Masalonis, 2000) and as a result, complacency effects occur (Parasuraman,
et al., 1993). For older adults, it has been hypothesized that this problem is exacerbated by
deficits in attention (Hardy et al., 1995; Vincenzi and Mouloua, 1999), and thus older adults should exhibit greater complacency effects.
At first glance, this is exactly what was found, but such a conclusion is premature. When monitoring performance was examined further, it was found that while older adults performed sufficient monitoring, they still failed to detect many automation failures. Their sensitivity for detection of automation failures was quite poor and it appears that many missed signals were the result of perceptual and/or cognitive factors beyond poor
monitoring.
The data from Experiment 2 point to a potential explanation of the age deficits
observed, working memory. If memory allows the user to compare automated decisions with an accurate mental representation, then the user will be better equipped to accept or reject the decision aid (Moray, 1981). However, if the user is ill-equipped to make this comparison, he or she must then determine whether the decision aid is correct, and this
decision may be influenced by a number of factors. For example, Skitka et al. (1999)
proposed that users have a bias to rely on automation, which they treat as a heuristic. Chen
and Sun (2003) and Johnson (1990) have reported that older adults use simpler heuristics
when engaged in cognitively demanding decision-making and this may influence their reliance on automation as well. Older adults may also develop inappropriate sampling strategies because they are less apt at determining stochastic properties and reinforcement
contingencies of the system with which they are working (Chasseigne et al., 2004; Maule
and Sanford, 1980; Sanford and Maule, 1973). Even when a proper sampling algorithm has been developed, the user must still properly attend to the information and extract it.
Changes in the adult attentional system are well documented (see Rogers, 2000 for a
review). As well, perceptual changes within the visual system may degrade encoding in
general and peripheral processing in particular (see Kline and Scialfa, 1997 for a review). At a more complex level of cognition, the elderly are less willing to take risks (Botwinick,
1978) and, in the present study, this may well make them more reluctant to depend on their
own abilities.
4.1. Limitations
In this study, a medication management scenario was used because it is a task that many older adults will encounter and benefit from by having automated assistance. It was not,
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
707
however, an attempt to address the complex problems with medication management.
Because medication management per se was not our primary concern, the ecological validity in the task suffered. The artificiality of the medication regimen placed greater demands on memory than what is normally encountered. A design that more properly
mimics a real medication regimen would add to the generalizability of this study and would be better able to address problems associated with using medication management devices. Another threat to ecological validity is the inclusion of a time-intensive mathematics task. It was chosen to create a demanding workload situation, whereby automation could help an individual perform two difficult, concurrent tasks more effectively. This
demanding workload is a necessary condition for over-reliance in automation to occur (Parasuraman et al., 1993). Thus, any over-reliance in this experiment may be bounded a
higher mental workload than is normally encountered.
4.2. Design implications and future studies
Despite these limitations, we believe that the results of these experiments have significant implications for the design of gerontechnology. The results of this study suggest that system designers must consider a constraint based on the older user’s
tendency to place greater trust in automation. This constraint will determine the degree of reliability a system must achieve before the system is deemed appropriate for market. The sensitivity of any alarm system must balance accurately detecting danger against
producing too many false alarms. Because older adults will respond to false alarms more than the young, the sensitivity of the alarm should be different for senior users. In cases in which automated decision systems reside in high workload environments (e.g. a collision warning system for a car), there will be even a greater tendency for older adults to rely on automated decisions. Thus, designers must also consider the environmental constraints
and understand the implications accordingly.
In addition, decision aids should use caution when providing binary (e.g. go/no go) decisions, particularly when there is uncertainty. Instead, when appropriate, graduated decisions can be provided or instructions can be provided to the user to check decisions
before carrying the decisions through.
A final design consideration involves providing feedback to the user regarding the system state and various component states. Automation often hides valuable information from the operator and as a result, the user cannot diagnose the problem. However,
providing too much information for senior users may lead to unnecessary complexity and may jeopardize the user’s acceptance of the technology. Thus, there is a trade-off between
providing too much detailed information and ease of use.
Future studies can examine the applicability of countermeasures to complacency, such as variable-priority training (Kramer et al., 1995; Metzger et al., 2000) and adaptive task allocation strategies (Parasuraman et al., 1996) since it is uncertain whether these strategies would benefit seniors. Additional research should also examine trust and reliance of automation with older adults under less stressful conditions than the present
experiment and in more realistic ecologically valid conditions.
Mitigating the effects of over-reliance and providing means for the elderly to develop appropriate mental models for autonomous systems will become increasingly important in
708
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
the future. There already exist many medical and personal systems that perform advanced
decision-making and care functions. To accommodate for these changes in technology, human factors issues in gerontechnology will need to expand beyond the usability of desktop computing and evolve to new human–computer interaction issues related to
automated systems, of which reliability and trust will be of critical importance.
Acknowledgements
Thanks to David Stewart, Michael Grahame, and Deanna Brown who provided help throughout. Additional thanks to Dr. Jeff Caird, Dr. Saul Greenberg, Dr. John Lee and the reviewers of this issue for their helpful and insightful comments. This work was supported, in part, by a grant from the Natural Science and Engineering Research Council of Canada.
References
Barber, B., 1983. The logic and limit of trust. Rutgers University Press, New Brunswick, NJ.
Billings, C., Lauber, J., Funkhouser, H., Lyman, G., Huff, E., 1976. NASA Aviation Safety Reporting System
(Tech. Memo. No.TM-X-3445). NASA Ames Research Center, Moffett Field, CA.
Botwinick, J., 1978. Aging and behavior: a comprehensive integration of research findings. Springer Publishing,
New York, NY.
Bouma, H., 1998. Gerontechnology: emerging technologies and their impact on aging in society. In: Graafmans,
J., Taipale, V., Charness, N. (Eds.), Gerontechnology: A Sustainable Investment in the Future. IOS Press,
Amsterdam, pp. 93–104.
Chasseigne, G., Ligneau, C., Grau, S., Le Gall, A., Roque, M., Mullet, E., 2004. Aging and probabilistic learning
in single- and multiple-cue tasks. Experimental Aging Research 3, 23–45.
Chen, Y., Sun, Y., 2003. Age differences in financial decision-making: using simple heuristics. Educational
Gerontology 29, 627–635.
Crum, R., Anthony, J., Bassett, S., Folstein, M., 1993. Population-based norms for the mini-mental state
examination by age and educational level. Journal of the American Medical Association 269, 2386–2391.
Czaja, S., 1997. Using technologies to aid the performance of home tasks. In: Fisk, A., Rogers, W. (Eds.),
Handbook of human factors and the older adult. Academic Press, San Diego, CA, pp. 311–334.
Czaja, S., Lee, C., 2001. The Internet and older adults: design challenges and opportunities. In: Charness, N.,
Parks, D., Sabel, B. (Eds.), Communication, technology, and aging: Opportunities and challenges for the
future. Springer Publishing Company, New York, NY, pp. 60–78.
Folstein, M., Folstein, S., McHugh, P., 1975. Mini-Mental State: a practical method for grading the state of
patients for the clinician. Journal of Psychiatric Research 12, 189–198.
Fox, J., Boehm-Davis, D., 1998. Effects of age and congestion information accuracy of advanced traveler
information systems on user trust and compliance. Transportation Research Record, Safety and Human
Performance 1621, 43–49.
Green, D., Swets, J., 1974. Signal detection theory and psychophysics. Robert E. Krieger Publishing Co.,
Huntington, NY.
Hardy, D., Mouloua, M., Molloy, R., Dwivedi, C., Parasuraman, R., 1995. Monitoring of automation failures by young and old adults. Proceedings of the 8th International Symposium on Aviation Psychology, USA, 1382–1387.
Hart, S., Staveland, L., 1988. Development of a multi-dimensional workload rating scale: results of empirical and
theoretical research. In: Hancock, P.A., Meshkati, N. (Eds.), Human mental workload. Elsevier, Amsterdam,
pp. 139–183.
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
709
Heinssen, G., Knight, L., 1987. Assessing computer anxiety: development and validation of the Computer Anxiety Rating Scale. Computers in Human Behaviour 3, 49–59.
Itoh, M., Abe, G., Tanaka, K., 1999. Trust in and use of automation: their dependence on occurrence patterns of
malfunctions. Proceedings of IEEE Systems, Man, and Cybernetics Conference, Japan, 111715–111720.
Johnson, M., 1990. Age differences in decision-making: a process methodology for examining strategic
information processing. Journal of Gerontology 45, 75–78.
Jonsson, I., Zajicek, M., Harris, H., Nass, C., 2005. Thank you, I did not see that: in-car speech based information
systems for older adults, CHI 2005 Technology, Safety, Community Conference, Portland, OR 2005 pp. 2–7.
Kantowitz, B., Hanowski, R., Kantowitz, S., 1997. Driver acceptance of unreliable traffic information in familiar
and unfamiliar settings. Human Factors 39, 164–176.
Kline, D., Scialfa, C., 1997. Sensory and perceptual functioning: basic research and human factors implications.
In: Fisk, A., Rogers, W. (Eds.), Handbook of human factors and the older adult. Academic Press, San Diego,
CA, pp. 27–54.
Kramer, A., Larish, J., Strayer, D., 1995. Training for attentional control in dual task settings: a comparison of
young and old adults. Journal of Experimental Psychology: Applied 1, 50–76.
Laberge, J., Ho, G., Scialfa, C., 2002. The distribution of attention while reading from a Web page. Proceedings
of the 2002 ACE-AE Joint Conference, Banff, AB, Canada.
Lee, J., Moray, N., 1992. Trust, control strategies and allocation of function in human-machine systems.
Ergonomics 35, 1242–1270.
Lee, J., Moray, N., 1994. Trust, self-confidence, and operators’ adaptation to automation. International Journal of
Human-Computer Studies 40, 153–184.
Lee, J., See, K., 2004. Trust in automation: designing for appropriate reliance. Human Factors 46, 50–80.
Lee, J., Stone, S., Gore, B., Colton, C., McCauley, J., Kinghorn, R., Campbell, J., Finch, M., Jamieson, G., 1997.
Advanced Traveler Information Systems and Commercial Vehicle Operations Components of the Intelligent Transportation Systems: design Alternatives for In-Vehicle Information Displays. (Report No. FHWA-RD-
96-147). Federal Highway Administration, McLean, VA.
Lewandowsky, S., Mundy, M., Tan, G., 2000. The dynamics of trust: comparing humans to automation. Journal
of Experimental Psychology: Applied 6, 104–123.
MacMillan, N., Creelman, C., 1991. Detection theory: a user’s guide. Cambridge University Press, New
York, NY.
Masalonis, A., 2000.
Effects of situation-specific reliability on trust and usage of automated decision aids,
Unpublished doctoral dissertation, Catholic University of America, Washington, DC.
Maule, A., Sanford, A., 1980. Adult age differences in multi-source selection behaviour with partially predictable
signals. British Journal of Psychology 71, 69–81.
Metzger, U., Parasuraman, R., 2001. Automation-related ‘complacency’: theory, empirical data, and design
implications, Proceedings of the 45th Human Factors Ergonomics Society Meeting, 463–467.
Metzger, U., Duley, J., Abbas R., Parasuraman, R., 2000. Effects of variable-priority training on automation-
related complacency performance and eye movements, Proceedings of the IEA2000/HFES2000 Congress, 2–
346 2–349.
Moray, N., 1981. The role of attention in the detection of errors and the diagnosis of errors in man-machine
systems. In: Rasmussen, J., Rouse, W. (Eds.), Human detection and diagnosis of system failures. Plenum
Press, New York, NY, pp. 185–198.
Moray, N., Inagaki, T., 2000. Attention and complacency. Theoretical Issues in Ergonomic Science 1, 354–365. Moray, N., Inagaki, T., Itoh, M., 2000. Adaptive automation, trust, and self-confidence in fault management of
time-critical tasks. Journal of Experimental Psychology: Applied 6, 44–58.
Muir, B., 1987. Trust between humans and machines, and the design of decision aids. International Journal of
Man-Machine Studies 27, 527–539.
Muir, B., 1994. Trust in automation, Part I: theoretical issues in the study of trust and human intervention in
automated systems. Ergonomics 37, 1905–1922.
Muir, B., Moray, N., 1996. Trust in automation, Part II, Experimental studies of trust and human interaction in a
process control simulation. Ergonomics 39, 429–460.
Parasuraman, R., Molloy, R., Singh, I., 1993. Performance consequences of automation-induced ‘complacency’.
The International Journal of Aviation Psychology 3, 1–23.
710
G. Ho et al. / Interacting with Computers 17 (2005) 690–710
Parasuraman, R., Mouloua, M., Molloy, R., 1996. Effects of adaptive task allocation on monitoring of automated
systems. Human Factors 38, 665–679.
Rempel, J., Holmes, J., Zanna, M., 1985. Trust in close relationships. Journal of Personality and Social
Psychology 49, 95–112.
Riley, V., 1996. Operator reliance on automation: theory and data. In: Parasuraman, R., Mouloua, M. (Eds.),
Automation and human performance: Theory and applications. Lawrence Erlbaum Associates, Mahwah, NJ,
pp. 19–35.
Rogers, W., 2000. Attention and aging. In: Park, D., Schwarz, N. (Eds.), Cognitive aging: A primer. Psychology
Press, Philadelphia, PA, pp. 57–73.
Sanchez, J., Fisk, A., Rogers, W., 2004. Reliability and age-related effects on trust and reliance of a decision
support aid. Proceedings of the 48th Human Factors and Ergonomics Society, 48, 586–589.
Sanford, A., Maule, A., 1973. The concept of general experience: age and strategies in guessing future events.
Journal of Gerontology 28, 81–88.
Singh, I., Molloy, R., Parasuraman, R., 1993. Automation-induced ‘complacency’: development of the
complacency-potential rating scale. The International Journal of Aviation Psychology 3, 111–122.
Skitka, L., Mosier, K., Burdick, M., 1999. Does automation bias decision-making? Journal of Human-Computer
Studies 51, 991–1006.
Vincenzi, D., Mouloua, M., 1999. Monitoring automation failures: effects of age in performance and subjective
workload. In: Scerbo, M., Mouloua, M. (Eds.), Automation technology and human performance: Current
research and trends. Lawrence Erlbaum Associates, Mahwah, NJ, pp. 253–257.
Wechsler, D., 1997. Wechsler Adult Intelligence Scale, third ed The Psychological Corporation, San
Antonio, TX.
Wiegmann, D., Rich, A., Zhang, H., 2001. Automated diagnostic aids: the effects of aid reliability on users’ trust
and reliance. Theoretical Issues in Ergonomics Science 2, 352–367.
Wiener, E., 1981. Complacency: is the term useful for air safety?, Proceedings of the 26th Corporate Aviation
Safety Seminar. Flight Safety Foundation, Inc., Denver, CO pp. 116–125.
因篇幅问题不能全部显示，请点此查看更多更全内容