“The way we do things is unsustainable” – Robert Groves, Census

This week I attended a great conference sponsored by DC-AAPOR. I’m typing up my notes from the sessions to share, but there are a lot of notes. This covers the morning sessions on day 1.

We are coming to a new point of understanding with some of the more recent developments in survey research. For the first time in recent memory, the specter of limited budgets loomed large. Researchers weren’t just asking “How can I do my work better?” but “How can I target my improvements so that my work can be better, faster, and less expensive?”

Session 1: Understanding and Dealing with Nonresponse

• Researchers have been exploring the potential of nonresponse propensity modeling for a while. In the past, nonresponse propensities were used as a way to cut down on bias and draw samples that should yield to a more representative response group.
• In this session, nonresponse propensity modeling was seen as a way of helping to determine a cutoff point in survey data collection.
• Any data on mode propensity for individual respondents (in longitudinal surveys) or groups of respondents can be used to target people in their likely best mode from the beginning, instead of treating all respondents to the same mailing strategy. This can drastically reduce field time and costs.
• Prepaid incentives have become accepted best practice in the world of incentives
• Our usual methods of contact are continually less successful. It’s good to think outside the box. (Or inside the box: one group used certified UPS mail to deliver prepaid incentives)
• Dramatic increases in incentives dramatically increased response rates and lowered field times significantly
• Larger lag times in longitudinal surveys led to a larger dropoff in response rate
• Remember Leverage Salience Theory- people with a vested interest in a survey are more likely to respond (something to keep in mind when writing invitations, reminders, and other respondent materials, etc.)
• Nonresponse propensity is important to keep in mind in the imputation phase as well as the mailing or fielding phase of a survey
• Re-engaging respondents in longitudinal surveys is possible. Recontacting can be difficult, esp. finding updated contact information. It would be helpful to share strategies re: maiden names, Spanish names, etc.

Session 2: Established Modes & New Technologies

• ACASI>CAPI in terms of sensitive info
• Desktop & mobile respondents follow similar profiles, vary significantly from distribution of traditional respondent profiles
• Mobile respondents log frequent re-entries onto the surveys, so surveys must allow for saved progress and reentry
• Mobile surveys that weren’t mobile optimized had the same completion rates as mobile surveys that were optimized. (There was some speculation that this will change over time, as web optimization becomes more standard)
• iPhones do some mobile optimization of their own (didn’t yield higher complete rates, though, just a prettier screenshot)
• The authors of the Gallup paper (McGeeney & Marlar) developed a best practices matrix- I requested a copy
• Smartphone users are more likely to take a break while completing a survey (according to paradata based on OS)
• This session boasted a particularly fun presentation by Paul Schroeder (abt SRBI) about distracted driving (a mobile survey! Hah!) in which he “saw the null hypothesis across a golden field, and they ran toward each other and embraced.” He used substantive responses, demographics, etc. to calculate the ideal number of call attempts for different survey subgroups. (This takes me back to a nonrespondent from a recent survey we fielded with a particularly large number of contact attempts, who replied to an e-mail invitation to ask if we had any self-respect left at that point)