Tip 7. Test a Health Care Quality Report With Your Audience
Members of your intended audience are the ones who will decide whether your report card is worth reading, and whether they can understand and use it. This means that feedback from readers is the “gold standard” of how well your report card is working.
Take the time to show a draft version of your report to some people who represent your intended readers and find out what they think.
The key to successful testing is to do it early and, if budgets and schedules permit, often. This approach allows for necessary changes while the report is still in development.
Individual Interviews Are More Informative Than Focus Groups
Although some people use focus groups to test a draft report, you can learn much more if you do individual interviews instead. Asking questions is a common method of getting feedback from readers, and you can ask questions whether you are doing focus groups or interviews. But interviewing allows you to also use three additional methods for getting feedback:
- Ask people to “think aloud.” You encourage readers to share whatever comes to mind while they are going through the report. It’s a great way to find out what they are noticing and get first impressions while they are fresh in people’s minds.
- Watch what they do with your report. You can observe where they start, where they hesitate or linger, what they skip over, and how long they spend on each part. Observing readers’ behavior can help you figure out how well the layout and organization of the report is working for people who have different ways of approaching and using the material.
- Give readers tasks to perform. To test the “usability” of your report, you can make a request that requires the reader to use the information in the report to draw a conclusion or make a decision. Asking people to do something specific with the report helps you find out how they are interpreting the information in the report and whether they can actually use the report for its intended purpose.
The Best Way To Check on Comprehension
Although tempting, it’s not helpful to ask people direct questions about comprehension, such as “How hard or easy was it to understand the report?”. You can’t take their answer at face value, because they might think they understand but be mistaken. Plus you learn nothing about what meaning they are taking from the information you present—only their opinion about ease of understanding.
To check on comprehension, ask people to talk about how they are interpreting the report and then judge for yourself how well they are understanding it. For example:
- “In your own words, what would you say this chart is about?”
- “What are the most important things to remember about what you just read?”
Using What You Learn
Your interviews with readers will help you look at the draft report from your readers’ point of view and identify possible barriers to their attention, comprehension, and use of the material. Use what you learn to make improvements to the language and other aspects of your report.
If you find problems when you do your testing and you make extensive revisions, try to do more consumer testing to find out whether you have addressed the problems adequately.
Learn More About Testing
- How will you test aspects of your health care quality report?
- An example of how to do testing and use the results: Hoy EW, Kenney E, Talavera AC. Engaging consumers in designing a guide to Medi-Cal Managed Care Quality. California Health Care Foundation, 2004. This resource includes an interview guide in the appendix.
- A step-by-step guide on testing methods for people without a research background: How to collect and use feedback from readers, in the Toolkit for Making Written Material Clear and Effective (forthcoming), Part 6. Written by Jeanne McGee for the Centers for Medicare & Medicaid Services. Available at http://www.cms.gov/WrittenMaterialsToolkit/TKIT/list.asp#TopOfPage.
