 |
 |
 |
|---|---|---|
Do you have a question about responsive or adaptive survey design that we can try to tackle for you? No question is too simple or complicated! |
Join the Responsive Survey Design conversation! Just fill out this form and we will add you to the forum discussion. |
Join the RSD Journal Club! Instructors name one RSD article at the beginning of each month and members use a forum to discuss the article for the duration of that month. |
James Wagner and Brady West will present a RSD seminar, Using Responsive and Adaptive Survey Design to Improve Data Collection Efficiency, at the 40th Annual Meeting and Scientific Sessions of the Society of Behavioral Medicine, which will be held March 6-9, 2019, at the Washington Hilton in Washington, DC.
For more information about the 2019 Meeting and Scientific Sessions of the Society of Behavioral Medicine please visit https://www.sbm.org/meetings/2019.
Course Date: June 17th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructor: Brady West
Topics covered: Randomized Controlled Trials (RCTs) are an important tool for tests of internal validity of causal claims in both health and social sciences. In practice, however, inattention to crucial details of data collection methodology can compromise the internal validity test. One crucial example is recruitment and retention of participants – though randomized to treatment, unequal reluctance to participate or unequal attrition from the RCT jeopardize the internal validity of comparisons within the RCT design. Another crucial example is the interaction of treatment and measurement – if the measures themselves change in response to the RCT treatment, then observed treatment and control differences may reflect these measurement differences rather than treatment differences. In both cases, specific tools from survey methodology can be used to maximize the internal validity test in the RCT design. This course will focus on the survey methodology topics most important for maintaining the internal validity of RCT studies and feature specific examples of applications to RCTs. One set of tools will focus on maximizing participation and minimizing attrition of participants. Core survey methodology tools for encouraging participation in both pre-treatment measurement and the treatment itself as well as tools for minimizing the loss of participants to follow-up measures will be featured. These tools include incentives, tailoring refusal conversion, switching modes, and tracking strategies. Links to RSD will also be made. A second set of tools will focus on measurement construction to reduce chances of interaction with treatment. These tools include mode options, questionnaire design issues, and special instruments (such as life history calendars) to minimize reporting error. Each portion of the course will feature examples applying each specific tool to RCT studies.
Course Date: June 17th, 2019
Course Time: 9:00 a.m - 5:00 p.m.
Instructors: James Wagner, Brady West, Andy Peytchev, Frauke Kreuter
This course will provide participants with an overview of the primary concepts underlying RSD. This will include discussion of the uncertainty in survey design, the role of paradata, or data describing the data collection process, in informing decisions, and potential RSD interventions. These interventions include timing and sequence of modes, techniques for efficiently deploying incentives, and combining two-phase sampling with other design changes. Interventions appropriate for face-to-face, telephone, web, mail and mixed-mode surveys will be discussed. Using the Total Survey Error (TSE) framework, the main concepts behind these designs will be explained with a focus on how these principles are designed to simultaneously control survey errors and survey costs. Examples of RSD in both large and small studies will be provided as motivation. Small group exercises will help participants to think through some of the common questions that need to be answered when employing RSD. For more information, please visit the Summer Institute in Survey Research Techniques website.
Course Date: June 18th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Brady West, William Axinn, Joe Murphy and Barry Schouten
This course will explore several well-developed examples of RSD. Dr. West will serve as a moderator of the course, and also introduce a case study from the National Survey of Family Growth (NSFG). The instructors will then provide independent examples of the implementation of RSD in different international surveys. All case studies will be supplemented with discussions of issues regarding the development and implementation of RSD. Case studies will include the NSFG, the Relationship Dynamics and Social Life (RDSL) survey, the University of Michigan Campus Climate (UMCC) Survey, and the Netherlands Survey of Consumer Satisfaction, among others. This variety of case studies will reflect a diversity of survey conditions. The NSFG (West) is a cross-sectional survey that is run on a continuous basis with in-person interviewing. The RDSL (Axinn) is a panel survey that employed a mixed-mode approach to collecting weekly journal data from a panel of young women. The UMCC survey is a web survey of students at UM that employed multiple modes of contact across the phases of the design. The Netherlands Survey of Consumer Satisfaction (Schouten) is a mixed-mode survey combining web and mail survey data collection with telephone interviewing. The focus of the course will be on practical tools for implementing RSD in a variety of conditions, including small-scale surveys. For more information, please visit the Summer Institute in Survey Research Techniques website.
Course Date: June 19th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructor: Danny Almirall
Topics covered: The effective treatment and management of a wide variety of health disorders often requires individualized, sequential decision making whereby treatment is adapted over time based on the changing disease state or specific circumstances of the patient. Adaptive interventions (also known as dynamic treatment regimens) operationalize this type of individualized treatment decision making using a sequence of decision rules that specify whether, how, for whom, or when to alter the intensity, type, or delivery of pharmacological, behavioral, and/or psychosocial treatments. There has been a huge surge of scientific interest in constructing adaptive interventions via the sequential multiple assignment randomized trial (SMART) design. SMART is a type of multi-stage randomized trial design, developed specifically for the purpose of collecting high-quality data for building optimal adaptive interventions. SMARTs are still new to the great majority of behavioral and social science investigators. In this course, we will introduce adaptive interventions, SMART (including simple design principle, cutting-edge analytic methods (e.g., Q-Learning) for SMART data, and discuss how these ideas can guide responsive and adaptive survey designs.
Course Date: June 19th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Heidi Guyer, Joe Murphy and Shonda Kruger-Ndiaye
This course will cover issues associated with implementation of RSD to manage field work. Instructors will provide concrete instruction on active monitoring of key indicators across a variety of environments – small-scale surveys, large-scale surveys, and web, telephone, face-to-face and mixed-mode surveys. Methods for implementing RSD interventions in a diversity of production environments will be discussed. RSD will be presented within the framework of the principles of project management, with a particular focus on risk management. A checklist of steps for implementing RSD will be discussed in detail. This course will draw upon a semester-long graduate course in survey management, which includes sections on RSD. For more information, please visit the Summer Institute in Survey Research Techniques website.
Course Date: June 19th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Barry Schouten and Natalie Shlomo
Topics covered: The response rate has been shown to be a poor indicator for data quality with respect to nonresponse bias. Several alternatives have been proposed – the fraction of missing information (FMI), R-Indicators, subgroup response rates, etc. This course will explore the use of these indicators as guides for data collection when working within an RSD framework. We also explore optimization techniques that may be useful when designing a survey to maximize these alternative indicators. The consequences of optimizing a survey to other indicators will be explored. We will also consider how the response rate fits into this approach. We will end with a brief discussion of methods for post data collection evaluation of data quality.
Course Date: June 19th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Stephanie Coffey, Scott Crawford and Julie Smith
Topics covered: Web surveys can be an inexpensive method for collecting data. This is especially true for designs that repeat measurement over several time periods. However, these relatively low-cost data collections may result in reduced data quality if the problem of nonresponse is ignored. This course will examine methods for using RSD to effectively deploy scarce resources in order to minimize the risk of nonresponse bias. Recent experience with the University of Michigan Campus Climate Survey and the National Survey of College Graduates is used to illustrate this point. These surveys are defined by phased designs and multiple modes of contact. This approach produced relatively high response rates and used alternative contact methods in later phases to recruit sample members from subgroups that were less likely to respond in earlier phases. In the case of the UM-CCS all of this was accomplished on a very small budget and with a small management team. Lessons from these experiences can be directly applied in many similar settings.
Course Date: June 20th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Ben Reist and Stephanie Coffey
Topics covered: This course will provide an overview of challenges and successes experienced in the development of adaptive survey design at the U.S. Census Bureau and the National Agricultural Statistics Service, including illustrations from the National Survey of College Graduates, the National Health Interview Survey and the Survey of Income and Program Participation. The presentation will include a brief history of the evolution of adaptive design capabilities at these agencies. We also discuss the development of a protocol for adaptive survey design that guides implementation and transparent documentation. The case studies covered will show applications of AD in surveys with different designs (cross-section vs. longitudinal, single vs. multi-mode) and different cost/quality objectives. We discuss successes and failures in these applications and factors that will shape future uses of adaptive design.
Course Date: June 20th, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructors: Brad Edwards and Victoria Vignare
Course Date: June 21st, 2019
Course Time: 9:00 a.m. - 5:00 p.m.
Instructor: Brady West
Topics covered: This course will discuss a variety of potential RSD interventions. Many of these have been implemented experimentally, and the course will include evaluations of those experiments. The importance of experimental evaluations in early phases of RSD will be discussed. Methods for implementing interventions will also be discussed, including implementation of experiments aimed at evaluating new interventions. Strategies for implementing these interventions with both interviewer-mediated and self-administered (e.g., web and mail) surveys will be discussed. Methods for the evaluation of the results of the interventions (experimental and otherwise) will be considered. These evaluations will include measures of both costs and errors.
Post in the forum must be related to responsive survey design. Questions or discussions that are off topic will be removed.
Users who post forum spam will have their posts removed. If a post that is otherwise appropriate includes links considered to be spam, those links will be removed.
