Author Archive

Are you who you say you are? Using a Multisource Cross-validation Methodology for Panel Membership Information

Abstract:

Over the past few years, market researchers and clients working with data from non-probability online panels have voiced a number of data quality concerns over issues such as respondent identity, increased satisficing, and possible professionalization of survey taking. Recognizing the importance of having real, unique, and engaged panelists, panel companies are responding to these issues by introducing a variety of remedial measures such as name-address verification, email address verification, and validation of key demographic information against third-party databases. Read more

Survey Mode Effects on Data Quality: Comparison of Web and Mail Modes in a U.S. National Panel Survey

Abstract:

Web surveys are being increasingly incorporated into national survey data collection programs in the United States because of their cost/time-efficiencies. Yet, response rates and data quality issues in web surveys remain important challenges. As a basic study designed to better understand data quality in a mixed mode national survey, this article investigates the degree to which web versus mail survey modes affect unit and item responses. Findings indicate that the web survey mode produces a lower unit response rate compared to the mail mode. However, the web mode elicits higher data quality in terms of item responses to both closed- and open-ended questions. Read more

A View from the Top – A Comprehensive Analysis of Post-Recruitment Factors in a Consumer Panel Operation

Abstract:

In a recent study involving a mixed-mode experiment to recruit members to a consumer panel, Rao, Kaminska, and McCutcheon (2010) investigated the effect of various response-inducement techniques such as advance letters, monetary incentives, and telephone follow-up on panel recruitment. The experiment was successful in demonstrating that a combination of recruitment mode and one or more response inducements can maximize recruitment rate and minimize recruitment cost. Read more

Research Synthesis: AAPOR Report on Online Panels

Executive Summary:

In September 2008, the AAPOR Executive Council established an Opt-In Online Panel Task Force and charged it with “reviewing the current empirical findings related to opt-in online panels utilized for data collection and developing recommendations for AAPOR members.” The council further specified that the charge did not include development of best practices, but rather would “provide key information and recommendations about whether and when opt- in panels might be best utilized and how best to judge their quality.” The task force was formed in October 2008. This is its report. Read more

Understanding How Consumers Cope with Food Safety Concerns: Using Qualitative Panel Data to Obtain a Complete Picture

Abstract:

American consumers are increasingly concerned about the microbial safety of the fruits and vegetables they consume.  A random survey of 3,619 consumers from the Gallup Panel performed in October 2008 investigated the extent to which American consumers were concerned about the microbial safety of produce and how they coped with those concerns in the food marketing channel. Read more

Do One-Time Inducements Help Long-Term Survey Participation? – A Case for Survival Analysis

Abstract:

In a recent study involving a mixed-mode experiment to recruit members to a consumer panel, Rao et al. (2010) tested the effect of various response-inducement techniques such as advance letters, monetary incentives, and telephone follow-up on panel recruitment. The experiment was successful in recruiting individuals to a non-incentive-driven consumer panel (i.e., members do not receive any form of monetary incentives for their participation in the panel). While the experiment used response inducements as a onetime stimulus to motivate individuals to join the panel, no such response inducements were provided for their subsequent panel participation. Read more

A Comparison of Web and Mail Survey Respondents within a Mixed Mode National Survey

Abstract:

Web surveys are being increasingly incorporated into national survey data collection programs in the United States because of their cost/time-efficiencies. Yet, response rates and data quality issues in web surveys remain important challenges. As a basic study designed to better understand data quality, this paper investigates the degree to which survey modes themselves affect response rates and data quality in a national mixed mode survey. Read more

The Theory and Evaluation of Anchoring Effect in Visually Administered Extremely Long Response Lists

Abstract:

Numerous studies (Payne 1971; Krosnick and Alwin 1987; Ayidiya and McClendon 1990; Schwarz, Hippler, and Noelle-Neumann 1992; Malhotra 2009; Malhotra 2008) have examined response order effects in visually administered questionnaires. A number of theories have also been proposed to explain this effect, with the most popular one being the theory of satisficing (Krosnick and Alwin 1987; Krosnick 1991). According to this theory, when response alternatives are presented visually, respondents begin at the top of the list and consider each alternative individually, all while establishing a cognitive framework for evaluating later response alternatives. Read more

Recruiting Probability Samples for a Multi-Mode Research Panel with Internet and Mail Components

Abstract:

Survey response rates have been declining over the past several decades, particularly for random-digit-dialing (RDD) telephone surveys (see de Leeuw and de Heer 2002; Steeh 1981). This trend affects research panels such as the Gallup Panel, which uses RDD methodology to recruit its members. If significant improvements in panel recruitment response rates are to be achieved, new approaches must be considered. This paper presents the findings of a mail and telephone mode experiment conducted by the Gallup Panel to analyze the individual and combined effects of incentives, advance letters, and follow-up telephone calls on the panel recruitment response rate. Read more

‘N the Network’? Using Internet Resources for Predicting Cell Phone Number Status

Abstract:

Despite higher hit rates for cell phone samples, inefficiencies in processing calls to these numbers relative to landline numbers continue to be documented in the U.S. literature. In this study, we propose one method for using cell phone provider information and Internet resources for validating number status. Specifically, we describe how we used ‘‘in network’’ options available from three major providers’ web sites to determine the validity of cell phone numbers. Read more