Supporting Statement: Part B
Information Collection Request Supporting Statements: Part B
5-Star Safety Ratings Label Quantitative Concept Testing
OMB Control No. 2127-NEW
Abstract:1
The National Highway Traffic Safety Administration (NHTSA) seeks to obtain critical information that will allow NHTSA and the New Car Assessment Program (NCAP) to fulfill a congressional mandate to improve highway traffic safety. As NCAP is responsible for providing consumers with important safety information that will assist them in their vehicle purchase decisions, this research will be used to gather data to enhance the usefulness of that safety rating information and guide the potential redesign of the Government 5-Star Safety Ratings section of the Monroney label (vehicle window sticker) accordingly.
The research study will include two components, both being one-time collections. The first component will involve a series of online webcam interviews that will collect qualitative feedback that will be used to improve the content included in the second component, a quantitative survey. The quantitative survey will be administered online and by phone (and potentially supplemented by mail if needed). Participants in the quantitative survey will be asked to evaluate design concepts and improvements to the Government 5-Star Safety Ratings section of the Monroney label. The intent is to identify the clearest, most communicative and helpful way to display information related to vehicle safety. NHTSA will use the findings from this research to support planned changes to the label requirements and to inform future consumer communications on vehicle safety ratings and safety technology systems performance assessments to assist the public when making vehicle purchasing decisions.
This collection of information will be voluntary. Respondents will include U.S. adult licensed drivers who are shared or primary decision-makers for their households’ vehicle purchasing decisions and who are in the “new vehicle purchasing mindset” – that is, they have purchased a new vehicle in the last six months or plan to do so in the next 12 months. Qualitative reporting will deidentify respondents and no PII will be shared with NHTSA. Reports highlighting findings from the qualitative and quantitative research will be delivered to and maintained by NHTSA.
B. JUSTIFICATION
B.1 Describe the potential respondent universe and any sampling or other respondent selection to be used.
The respondent universe includes anyone who meets the following criteria:
Respondents must be 18 years or older.
Respondents must currently possess a valid driver’s license.
Respondents must be in the vehicle purchase mindset. That is, they have either purchased a vehicle in the past six months, or plan to do so within the next 12 months.
Respondents must be the primary or a shared decision maker for vehicle purchases in their household.
The research will contain two phases—the first being a cognitive test using a series of in-depth webcam interviews, during which participants will complete the survey simultaneously provide feedback to a moderator during the process. The goal of the cognitive test is to ensure each question is clear, consistently understood and offers appropriate response options. The second phase will include a mixed-mode quantitative survey.
The qualification criteria will be identical for the two phases, but participants will be sourced differently. The cognitive testing will rely on a convenience sample and will source participants through proprietary databases maintained by recruiting partners. The quantitative phase will utilize an opt-in, probability panel (this would be an online panel built to be representative of the U.S. and recruited using probability-based methods including online, phone and mail). We expect to partner with NORC and to use their AmeriSpeak panel or a similar probability panel.
AmeriSpeak is the first U.S. multi-client household panel to combine the speed and cost-effectiveness of panel surveys with enhanced representativeness of the U.S. population, an industry-leading response rate and the NORC Card, an innovative sample quality report card. Since its founding by NORC at the University of Chicago in 2015, AmeriSpeak has produced more than 300 surveys, been cited by dozens of media outlets and become the primary survey partner of The Associated Press. AmeriSpeak’s sampling provides better representation than other panels for hard-to-reach populations, including low-income households, less-educated persons, young adults, rural households, persons who are less interested in the news, and social and political conservatives. AmeriSpeak promotes itself as the most scientifically rigorous multi-client panel available in the U.S. market.2
AmeriSpeak uses an invitation-only probability panel design that is representative of the U.S. population. Because the panel is representative of the U.S. population, survey results can be extrapolated to the population under study. The difference between probability and non-probability samples is the ability to confidently project results to the underlying population. While NORC’s AmeriSpeak sample is a panel, it is recruited and built using probability-based sampling techniques. Additionally, NORC will use other samples (including phone and mail if needed) to supplement their panel and access hard-to-reach populations.
B.2 Describe the procedures for the collection of information.
Once the survey is ready for testing, a recruiting partner will begin outreach to find nine English-speaking respondents who meet the screening criteria and are available and interested in participating in the cognitive test. The cognitive test will be conducted using a series of in-depth webcam interviews during which a moderator will guide the participant through the survey and debrief them after each question to see if the wording is clear and easy to understand and if the available response options are adequate. The moderator will take notes and record the interview to create transcripts. Notes and transcripts will be summarized into a final report that presents findings in aggregate and highlights suggested changes to improve the survey for comprehension.
The survey program will be updated based on recommended changes from the qualitative phase, and the quantitative phase will begin. This will consist primarily of online surveys but will also be supplemented with outbound phone calls, mail surveys and intercept surveys as needed. The expectation is that fewer than 5% of completes will be obtained by these supplemental methods, and most, if not all, of the supplemental completes will be collected with phone surveys.
The survey will field until it reaches its target of 1,000 completes. Approximately 10,000 respondents will be screened to identify 1,000 who will qualify and complete the survey. Once fielding is complete, the field partner will provide the research team with a data file that they will use for data analysis. The research team will create a summary report of findings that will be shared with NHTSA. A cleaned copy of the survey data file will be included as a final deliverable to NHTSA. This data file will not include any personally identifiable information.
B.3 Describe methods to maximize response rates.
In keeping with industry standards, participants will be offered a small incentive to compensate them for their time. The incentive for completing the 60-minute webcam interview will be $100 and will be paid to respondents by the recruiter. Participants who qualify and complete the quantitative survey would earn approximately $5 worth of points that could be redeemed for gift cards, a pre-paid debit card or a charitable donation. The $5 value is based on the amount of incentive respondents typically receive for completing a 10-minute survey. The survey will rely on AmeriSpeak’s online panel for the majority of responses. Panelists have previously opted in for receiving and responding to web-based surveys. Those who choose to participate will be incentivized, as noted in Part A (section A9) of this Supporting Statement. They are attuned and receptive to survey invitations. Additionally, as needed, reminders will be sent to participants who have not completed the survey.
Though we estimate that 10% of participants who are screened for the research will qualify, we are unable to predict an exact response rate.3 Response rates are subject to a wide range of factors, including methodology (e.g., online versus telephone interviews), topic interest, incentive size and many more. Academic research has estimated survey response rates at around 44.1% on average.4 However, Pew Research has seen response rates as high as 89% when using an opt-in probability panel.5 Response rates may be higher for opt-in panels because they consist of panelists who have volunteered to be given opportunities to take surveys. Our study will use an opt-in probability panel, so we expect most of the sample who is invited to participate and virtually all who qualify to complete the study. However, our study will supplement with other sources and methodologies as needed to interview hard-to-reach groups, which may shift the response rate in ways that cannot be predicted.
B.4 Describe any tests of procedures or methods to be undertaken.
Once the survey is programmed, the research team will conduct an internal test of the survey to ensure all questions and responses are programmed correctly. Once internal testing is complete, the first phase of the research will begin. The cognitive test allows for a pilot test of the survey among the target audience and will help discover any potential wording issues where questions (or responses) are unclear or not consistently understood. Additionally, it helps ensure the available response options are adequate.
For the basis of the descriptive data analysis, data tabulation sets will be developed based on a tab plan customized to the final screener and questionnaire. Tabulations will each feature up to 21 banner points. This number of banner points, along with their detailed statistical values, is expected to allow for descriptive comparative analyses of responses from sub-groupings of the sample based on behavioral, attitudinal, and demographic similarities/differences. The individual tabulations will include stubs for all closed-ended data points in the survey, means for Likert and Semantic Differential scale ratings frequency and percentage responses.
Through data processing, we will analyze the total data gathered from the sample of respondents. We will also use cross tabulations to analyze summary statistics and coded open-ended responses across demographic groups to explore any apparent differences; however, the intention of this survey is not to guide decisions on communications targeting by demographics. Rather, this analysis will provide NHTSA with the ability to make informed decisions about potential revisions to the Government 5-Star Safety Ratings section of the Monroney label, particularly in terms of effectively incorporating information related to advanced safety technology.
B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design.
The company selected as a contractor for this study is Team Stratacomm. Team Stratacomm includes the prime contractor Stratacomm, a marketing communications and advertising consultancy for NHTSA’s OCCI, as well as the teaming partner Heart+Mind Strategies, a data-driven research consultancy that specializes in custom research, advanced analytics and communication strategy. This team has extensive experience in both qualitative and quantitative research practices and has consulted with NHTSA on all aspects of the design. The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the Monroney Label Quantitative Concept Testing Research:
Ben Custer
Senior Solutions Consultant
Heart+Mind Strategies
12110 Sunset Hills Rd, 6th Floor
Reston, VA 20190
703.474.8082
Dr. Carlos Elordi
Chief Data Scientist
Heart+Mind Strategies
12110 Sunset Hills Rd, 6th Floor
Reston, VA 20190
540.588.7548
NHTSA CONTACT
Mike Joyce
Marketing Specialist, Office of Communications and Consumer Information
National Highway Traffic Safety Administration (NHTSA)
1200 New Jersey Avenue, SE
Washington, DC 20590
202.366.5600
1 The Abstract must include the following information: (1) whether responding to the collection is mandatory, voluntary, or required to obtain or retain a benefit; (2) a description of the entities who must respond; (3) whether the collection is reporting (indicate if a survey), recordkeeping, and/or disclosure; (4) the frequency of the collection (e.g., bi-annual, annual, monthly, weekly, as needed); (5) a description of the information that would be reported, maintained in records, or disclosed; (6) a description of who would receive the information; (7) if the information collection involves approval by an institutional review board, include a statement to that effect; (8) the purpose of the collection; and (9) if a revision, a description of the revision and the change in burden.
2 https://markets.businessinsider.com/news/stocks/norc-expands-breakthrough-panel-based-research-platform-welcomes-bruce-barr-to-oversee-amerispeak-omnibus-1027795079
3 Based on past experience recruiting this audience, we estimate an incidence of 10-15%. We use the more conservative figure of 10% here. This lower estimate also helps account for the possibility that some respondents will drop off the survey after the screener but before answering all questions.
4 https://www.sciencedirect.com/science/article/pii/S2451958822000409
5 https://www.pewresearch.org/politics/2022/03/24/midterms-2022-methodology/
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Sharon Hegarty |
| File Modified | 0000-00-00 |
| File Created | 2025-12-31 |