TeamSTEPPS® Teamwork Attitudes Questionnaire Manual
Prepared for:
U.S. Department of Defense, Tricare Management Activity
Subcontract agreement # SK-443-07-AIR-HCTCP
Prime Contract # W81XWH-06-F-0526
Prepared by:
David P. Baker, Ph.D.
Kelley J. Krokos, Ph.D.
Andrea M. Amodeo, M.S.
American Institutes for Research
1000 Thomas Jefferson Street, NW
Washington, DC 20007
Original: 24 September 2008
Revised: 7 October 2008
Contents
Introduction
Development of the T-TAQ
Literature Review
Item Development
Pilot Test
Final Item Selection
Scoring
Administration
Customization of Use
References
Appendix A
Appendix B
Introduction
This document serves as a comprehensive manual for the TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ). It describes the development of the T-TAQ and provides detailed instructions for administering the instrument.
Background
Since the Institute of Medicine’s 1999 report, To Err is Human (Kohn, Corrigan, & Donaldson, 1999), the Agency for Healthcare Research and Quality (AHRQ) and the Department of Defense (DoD) have been Federal leaders in the patient safety movement. A major focus of these agencies has been supporting research and development activities centered on improving team performance in the delivery of care. Many organizations, such as The Joint Commission, the Institute for Healthcare Improvement, the National Quality Forum, and the Accreditation Council for Graduate Medical Education, have cited the importance of teamwork in patient safety. The National Defense Authorization Act of 2001 requires the DoD to deliver such training for staff at DoD Military Treatment Facilities (MTFs) through the Health Care Team Coordination Program, and the Joint Commission recommends hospitals establish team training programs.
In November 2006, AHRQ and DoD in collaboration released TeamSTEPPS as the national standard for team training in health care. TeamSTEPPS, which stands for Team Strategies and Tools to Enhance Performance and Patient Safety, is the result of a multiyear research and development project. Since 2005, numerous organizations and individuals have contributed to the refinement of the TeamSTEPPS curriculum.i
A cornerstone of the TeamSTEPPS approach is the three-phase process that defines the requirements for implementing the curriculum. Phase I involves conducting a site assessment to determine the teamwork-related needs within the unit. Phase II involves implementing the selected TeamSTEPPS tools and strategies, and Phase III involves sustaining the TeamSTEPPS intervention. Therefore, a critical cog in this process is measurement. Knowing that the TeamSTEPPS tools and strategies enhanced participant attitudes toward teamwork, developed participant knowledge about effective team practice, and improved team skills on the unit is critical to knowing the intervention worked.
The T-TAQ was designed to measure individual attitudes related to the core components of teamwork that are captured within TeamSTEPPS. Specifically, individual attitudes toward team structure, leadership, mutual support, situation monitoring, and communication are measured. The T-TAQ can be used to assess specific needs within the unit or health care institution and whether the TeamSTEPPS intervention produced the desired attitude change.
i. For a complete list of contributors consult the TeamSTEPPS instructor guide, which can be ordered at http://www.ahrq.gov/teamstepps/.
Development of the T-TAQ
The development of the T-TAQ involved a series of systematic steps: literature review, item development, pilot testing, item selection, and psychometric testing. Each step is described below.
Literature Review
Our search of the relevant literature revealed that few measures exist that provide assessments of individual attitudes toward teamwork, particularly ones oriented toward health care. Furthermore, none are aligned with the core components of teamwork (Salas, Sims, & Burke, 2005) that are the basis for the TeamSTEPPS program. For example, in aviation, the Cockpit Management Attitudes Questionnaire (Helmreich, 1984; Gregorich, Helmreich, & Wilhelm, 1990) assesses constructs related to crew resource management (CRM) including leadership, coordination, and communication. In health care, the Safety Climate Survey measures perceptions of organizational commitment to patient safety through constructs such as commitment to safety, leadership, interpersonal interactions, attitudes toward stress, and knowledge of how to report adverse events (Sexton, et al., 2006). The Safety Attitudes Questionnaire measures hospital providers’ attitudes about teamwork climate, safety climate, perceptions of management, job satisfaction, working conditions, and stress recognition (Sexton & Thomas, 2003).
Perhaps the most germane measure to the current discussion is AHRQ’s Hospital Survey on Patient Safety (HSOPS). HSOPS measures 12 dimensions related to patient safety culture. Two of these scales focus specifically on teamwork: teamwork within units and teamwork between units. Hundreds of institutions have administered HSOPS, and national norms are available. TeamSTEPPS recommends institutions consider using HSOPS as part of their site assessment for determining their teamwork needs and as an evaluation tool to determine whether HSOPS scores improve as a function of TeamSTEPPS implementation. However, the HSOPS focuses on safety culture, and the teamwork scales do not partition out the critical subdomains of team performance. Therefore, our review of the literature supported the need for the development of the T-TAQ.
Item Development
We developed items on the basis of a test specification that listed the domains to be assessed and the number of items to write for each domain. A test specification ensures that content validity is built into the item development process. Content validity relates to whether the items provide appropriate coverage of the core constructs one is attempting to measure. In this case, we were interested in assessing attitudes toward the core components taught within TeamSTEPPS: team structure, leadership, situation monitoring, mutual support, and communication.
We developed items through an extensive item-writing process that included multiple item writers who were experienced in survey and test development and were also knowledgeable in the principles of teamwork and, more specifically, the TeamSTEPPS training curriculum upon which the items were to be based. As items were drafted, writers linked each item to a specific TeamSTEPPS curriculum module and identified the page in the manual from which the item was written. Throughout the item-writing process, this linking ensured that items captured key teamwork concepts. Item-writing efforts resulted in a pool of 110 items.
Next, we conducted an item review. Item review criteria included social desirability concerns. For example, we used caution with including the phrase "it is important to" in an item, as this may trigger a positive response (i.e., agreement with the item), regardless of the true attitude of the respondent. We also reviewed items to ensure that they asked the respondent about his/her attitude and not that of other team members or the team as a whole. Further reviews sought to eliminate items that were repetitive in terms of teamwork concepts and to limit the number of items within each of the constructs to approximately 10 items. At the end of this review, 49 items remained in the pool. Table 1 illustrates the constructs measured by the piloted T-TAQ and the corresponding number of items.
Table 1. Number of Pilot Test Items by TeamSTEPPS Construct
Construct | Number of Items |
---|---|
Team Structure | 7 |
Leadership | 11 |
Situation Monitoring | 11 |
Mutual Support | 11 |
Communication | 9 |
Total | 49 |
Pilot Test
The process and outcome of the pilot test are described in more detail below.
Data Collection. We administered the pilot version of the T-TAQ to TeamSTEPPS training participants (n=346) at various MTFs throughout the DoD between December 2007 and April 2008. We also distributed the pilot version to participants (n=149) in a mid-Atlantic critical care conference that occurred in Spring 2008. For purposes of this research, we identify this sample as our "civilian" sample to distinguish it from the DoD sample. Participants for the DoD sample completed the pilot version of the T-TAQ for either prior to or immediately following TeamSTEPPS training at the discretion of the individual training facilitators. The T-TAQ for the civilian sample was administered during the conference. All questionnaires were completed in paper form.
Data Quality. To ensure the integrity of the data, we conducted analyses to remove cases with excessive missing data and anomalous response patterns. Specifically, cases in which respondents did not respond to at least 95 percent of the items or answered at least 95 percent with the same response were removed from the data set. In addition, when respondents provided multiple responses to an item, or the response was unclear, the data point was coded as "missing." This resulted in a loss of 46, or 9.8 percent, of the cases and subsequent reduction in the total sample size from 495 to 449.
Final Sample. As a result of the data "cleaning" described above, 449 cases were included in the analyses, with n=311 from the DoD and n=138 from the civilian participants. The large majority of the respondents (N=408 or 91.7 percent) reported that they deliver direct care to patients. Of these, n=175 or 44.3 percent reported they deliver inpatient care, n=89 or 22.5 percent reported they deliver outpatient care, and n=131 or 33.2 percent reported they deliver both inpatient and outpatient care. A total of 211 respondents reported their position as physicians or dentists.ii Eighty-five respondents reported they work as registered nurses and 27 as advanced practice nurses. Respondents reported working predominantly in intensive care: 260 respondents reported working in some type of intensive care unit (ICU), such as the general ICU, a cardiac ICU, or a pediatric ICU. An additional seven respondents reported working specifically in a neonatal ICU. Eighteen respondents reported working in an emergency department and an equal number in a pediatric unit. Both the DoD and civilian administrations asked respondents whether they had prior team training experience, of which the majority (n=242 or 54.3 percent) reported they did not.
Final Item Selection
We used a classical item statistics approach and criteria regarding the desired length of the final T-TAQ to select items. First, we computed and analyzed means, item-total correlations, and standard deviations. We reviewed items and constructs using these analyses and Cronbach’s Alpha "If Item Deleted" results. In addition, we considered the number of items per construct, as we desired scales of equal numbers of items, as well as a final T-TAQ that would be of suitable length (i.e., not requiring much participant time). This process resulted in the deletion of 19 items from our pilot measure.
The final questionnaire (presented in Appendix A) includes a total of 30 items; 6 items for measuring each teamwork construct. Among the final items, four are reverse coded: three in mutual support and one in communication.iii Final constructs and their alpha values are provided in Table 2. Finally, we examined construct independence by intercorrelating the five subscales that comprise the T-TAQ. Table 3 provides Pearson correlation coefficients among the constructs, with coefficients ranging from .36 (mutual support and team structure) to .63 (situation monitoring and communication). These results suggest that while the constructs overlap to some degree, they do also exhibit unique variance.
Table 2. T-TAQ Reliability Coefficients
Construct | Number of Items | Cronbach’s Alpha |
---|---|---|
Team Structure | 6 | .70 |
Leadership | 6 | .81 |
Situation Monitoring | 6 | .83 |
Mutual Support | 6 | .70 |
Communication | 6 | .74 |
Table 3. T-TAQ Construct Inter-Correlations
Construct | Team Structure | Leadership | Situation Monitoring | Mutual Support | Communication |
---|---|---|---|---|---|
Team Structure | 1.00 | .572* | .617* | .356* | .533* |
Leadership | 1.00 | .633* | .481* | .558* | |
Situation Monitoring | 1.00 | .541* | .627* | ||
Mutual Support | 1.00 | .589* | |||
Communication | 1.00 | ||||
N | 449 | 449 | 449 | 449 | 449 |
*p < .01, two-tailed.
ii. DoD and civilian questionnaires included different demographic items. This was due to the specific needs and interests of each group. Some items (e.g., respondent position, department) were the same but provided different response options. In addition, some respondents provided multiple responses for the position and department in which they worked. Therefore, information about each position or department or unit in which respondents work is not necessarily unique to that position or department or unit and may have been collapsed into a more general response category for reporting purposes. Due to this issue, valid percentages are not reported.
iii. The T-TAQ reverse-coded items are mutual support items 20, 21, 24, and communication item 30.
Scoring
T-TAQ scoring can take one of two forms. A total score may be calculated for each teamwork construct or an average score may be derived. Table 4 presents means, ranges, and standard deviations for each construct captured in the T-TAQ. As can be seen in Table 4, respondents had very favorable attitudes toward teamwork across constructs.
Table 4. Means, Range, and Standard Deviations
Construct | N | Range | Mean | Std. Deviation |
---|---|---|---|---|
Team Structure | 449 | 2.00 | 4.20 | .466* |
Leadership | 449 | 1.83 | 4.49 | .425 |
Situation Monitoring | 449 | 2.50 | 4.27 | .463 |
Mutual Support | 449 | 2.67 | 4.19 | .510 |
Communication | 449 | 2.17 | 4.28 | .478 |
Valid N (listwise) | 449 |
Administration
The T-TAQ may be administered as a stand-alone measure to assess attitudes toward the core components of teamwork, as part of an institution’s site assessment to determine training needs, or as a tool to evaluate TeamSTEPPS training. In all of these cases, the basic administration of the T-TAQ should remain the same (refer to Appendix B). What will vary, as noted in the administration options below, is when the T-TAQ is administered. Detailed administration instructions are presented in Appendix B.
Stand-Alone. As an independent assessment of teamwork attitudes, the T-TAQ may be administered at any point in time. In this capacity, the T-TAQ may be administered organization-wide, unit-wide, or to some combination of units. Careful consideration should be given to identifying the population of interest to be surveyed so the correct staff members receive the T-TAQ.
TeamSTEPPS Site Assessment. The T-TAQ may be used as one component of the TeamSTEPPS site assessment process. In this capacity, the T-TAQ should be administered prior to TeamSTEPPS training. Results can be used to identify where poor attitudes toward teamwork exist within a unit or institution and can therefore assist the organization’s TeamSTEPPS change team in selecting specific TeamSTEPPS interventions.
TeamSTEPPS Evaluation. The T-TAQ may be used to assess TeamSTEPPS effectiveness. The basic question answered here is whether the TeamSTEPPS intervention produces desirable attitude changes regarding teamwork. To answer this question, the T-TAQ should be administered immediately prior to and after TeamSTEPPS training. Results from these two data collections should then be compared. The T-TAQ may also be administered several months after TeamSTEPPS training to see if the attitude changes that were achieved at the end of training are sustained.
Customization of Use
Organizations may have different needs with regard to teamwork development. This section briefly describes appropriate options for customization of the T-TAQ.
Background Questions. Background questions regarding a participant’s current job, professional training, age, experience, etc. are not included on the T-TAQ. We have found through our experience that various health care units are referred to differently across organizations, so it is best if organizations develop customized background questions. We recommend that each institution develop its own questions regarding participant characteristics and append these questions to the back of the questionnaire.
Item Modification. Items on the T-TAQ should not be modified. Changing the items can affect the reliability and validity of the instrument.
Scale Use. Scales from the instrument can be used separately if desired. For example, if an organization was interested only in attitudes toward leadership, then this scale could be administered independently. The administration instructions found in Appendix B would still apply.
References
Gregorich, S. E., Helmreich, R. L., & Wilhelm, J. A. (1990). The structure of cockpit management attitudes. Journal of Applied Psychology 75, 682-690.
Helmreich, R. L. (1984). Cockpit management attitudes. Human Factors, 26, 583-589.
Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (1999). To err is human. Washington, DC: National Academy Press.
Salas, E., Sims, D. E., & Burke, C. S. (2005). Is there a big 5 in teamwork? Small Group Research 36(5), 555-599.
Sexton J. B., Helmreich, R. L., Neilands, T. B., Rowan, K., Vella, K., Boyden, J., et al. (2006). The safety attitudes questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Services Research, 6(44).
Sexton, J. B., & Thomas, E. J. (2003). The safety climate survey: Psychometric and benchmarking properties (Technical Report 03-03). Houston, TX: The University of Texas Center of Excellence for Patient Safety Research and Practice.
Appendix A: TeamSTEPPS Teamwork Attitudes Questionnaire
The purpose of this survey is to measure your impressions of various components of teamwork as it relates to patient care and safety.
Instructions: Please respond to the questions below by placing a check mark (√) in the box that corresponds to your level of agreement from Strongly Agree to Strongly Disagree. Please select only one response for each question.
Team Structure | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
---|---|---|---|---|---|
1. It is important to ask patients and their families for feedback regarding patient care. | |||||
2. Patients are a critical component of the care team. | |||||
3. This facility's administration influences the success of direct care teams. | |||||
4. A team's mission is of greater value than the goals of individual team members. | |||||
5. Effective team members can anticipate the needs of other team members. | |||||
6. High performing teams in health care share common characteristics with high performing teams in other industries. | |||||
Leadership | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
7. It is important for leaders to share information with team members. | |||||
8. Leaders should create informal opportunities for team members to share information. | |||||
9. Effective leaders view honest mistakes as meaningful learning opportunities. | |||||
10. It is a leader's responsibility to model appropriate team behavior. | |||||
11. It is important for leaders to take time to discuss with their team members plans for each patient. | |||||
12. Team leaders should ensure that team members help each other out when necessary. | |||||
Situation Monitoring | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
13. Individuals can be taught how to scan the environment for important situational cues. | |||||
14. Monitoring patients provides an important contribution to effective team performance. | |||||
15. Even individuals who are not part of the direct care team should be encouraged to scan for and report changes in patient status. | |||||
16. It is important to monitor the emotional and physical status of other team members. | |||||
17. It is appropriate for one team member to offer assistance to another who may be too tired or stressed to perform a task. | |||||
18. Team members who monitor their emotional and physical status on the job are more effective. | |||||
Mutual Support | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
19. To be effective, team members should understand the work of their fellow team members. | |||||
20. Asking for assistance from a team member is a sign that an individual does not know how to do his/her job effectively. | |||||
21. Providing assistance to team members is a sign that an individual does not have enough work to do. | |||||
22. Offering to help a fellow team member with his/her individual work tasks is an effective tool for improving team performance. | |||||
23. It is appropriate to continue to assert a patient safety concern until you are certain that it has been heard. | |||||
24. Personal conflicts between team members do not affect patient safety. | |||||
Communication | Strongly Disagree | Disagree | Neutral | Agree | Strongly Agree |
25. Teams that do not communicate effectively significantly increase their risk of committing errors. | |||||
26. Poor communication is the most common cause of reported errors. | |||||
27. Adverse events may be reduced by maintaining an information exchange with patients and their families. | |||||
28. I prefer to work with team members who ask questions about information I provide. | |||||
29. It is important to have a standardized method for sharing information when handing off patients. | |||||
30. It is nearly impossible to train individuals how to be better communicators. |
Please provide any additional comments in the space below.
Thank you for your participation!
Appendix B: TeamSTEPPS Teamwork Attitudes Questionnaire Administration Instructions
Read the following instructions aloud prior to administering the T-TAQ:
Please complete the following questionnaire by placing a check mark [√] in the box that corresponds to your level of agreement from Strongly Disagree to Strongly Agree. Please answer every question and select only one response for each question. The questionnaire is anonymous, so please do not put your name or any other identifying information on the questionnaire.
[Optional]: On the last page you will find questions about your background and experience. Please provide your responses to each question in the space provided. Thank you for your participation.