Coming soon · Book Profile
Design, Evaluation, and Analysis of Questionnaires for Survey Research
A systematic, science-based method for designing survey questions, predicting their measurement quality before data collection, and correcting for measurement error in analysis.
A profile of this book is on the way.
What it’s about
This book transforms questionnaire design from an 'art' into a scientific activity. Saris and Gallhofer present a complete program: a three-step procedure for operationalizing complex concepts into concrete survey requests, a thorough mapping of the design choices researchers face (response scales, item structure, batteries, data collection mode), and a rigorous framework for estimating the reliability, validity, and method effects of survey questions using multitrait-multimethod (MTMM) experiments. The crowning achievement is the SQP (Survey Quality Predictor) program, built on a meta-analysis of thousands of MTMM experiments across dozens of countries and languages, which predicts the quality of any survey question from its coded characteristics—before it is ever fielded. The authors then show how to use these quality estimates to correct for measurement error in substantive analyses and in cross-cultural comparisons, demonstrating that ignoring measurement quality leads to seriously biased conclusions about relationships and means.
The through-line
- Who it’s for
- A survey researcher or social scientist who wants to design questionnaires that accurately measure the concepts they care about and yield trustworthy results.
- The problem
- Survey questions contain measurement error that biases estimates of relationships and means, and there is no easy way to know a question's quality before fielding it. The researcher feels uncertain whether their data truly measure what they intend and anxious that their conclusions may be artifacts of poor questions.
- The plan
- Operationalize complex concepts into concrete requests using the three-step procedure.
- Make informed choices about response scales, item structure, batteries, and data collection mode.
- Predict the quality of each question with SQP before fielding and improve weak questions.
- Estimate reliability, validity, and method effects where possible via MTMM designs.
- Correct correlations and estimates for measurement error in substantive analysis.
- The payoff
- The researcher fields higher-quality questions, knows their measurement quality in advance, obtains less biased estimates of relationships and means, and can make valid cross-cultural comparisons.
See our guide
Related profiles we’ve built
- Reliability and Validity Assessment →
- A Theory of Human Motivation →
- Anxiety at Work_ 8 Strategies to Help Teams Build Resilience, Handle Uncertainty, and Get Stuff Done →
- Applied Multivariate Stats Social Sciences Stevens →
- Assessing Change in Psychoanalytic Psychotherapy of Children and Adolescents (Psychology, Psychoanalysis & Psychotherapy) →
- Basics Qualitative Research Grounded Theory Corbin Strauss →
- Bayesian Multilevel Models for Repeated Measures dаta A Conceptual and Practical Introduction in R →
- Beyond Hr Boudreau Ramstad →
Additional reading
- Survey Sampling · Kish, L. (1965)
The book frequently refers to this text, recommending it as a primary resource for its comprehensive discussions of sampling practice.
- Sampling Techniques · Cochran, W. G. (1977)
Recommended by the author as a key text for its thorough treatment of the statistical theory underlying survey sampling.
- Sample Survey Methods and Theory · Hansen, M. H., Hurwitz, W. N., and Madow, W. G. (1953)
Cited as a foundational text for its discussion of the practical application of sampling methods.
- Survey Methods in Social Investigation · Moser, C. A. and Kalton, G. (1971)
Co-authored by the book's author, it provides a broader introduction to survey methods, placing sampling in context.
- Mail and Telephone Surveys · Dillman, D. A. (1978)
Cited for providing detailed methods on how to increase response rates in mail and telephone surveys, a key concern of the book.
- Sampling methods for random digit dialing · Waksberg, J. (1978)
This is the source for the two-stage RDD telephone sampling method described in detail in the book's case study chapter.
- Statistical Theories of Mental Test Scores · Lord, F. M. and M. R. Novick (1968)
Cited as a source for much more extensive discussions of classical test score theory, the foundation of the book's approach to reliability.
- Introduction to Factor Analysis & Factor Analysis: Statistical Methods and Practical Issues · Kim, J. and C. W. Mueller (1978a, 1978b)
Recommended in the editor's introduction as a necessary supplement for understanding the appendix on using factor analysis for reliability and validity assessment.
- Multiple Indicators: An Introduction · Sullivan, J. L. and S. Feldman (1979)
Recommended by the series editor as a companion volume that gives considerable attention to issues of validity and reliability.
- Survey Research Methods · Fowler, F.J. Jr.
Referenced for fundamental concepts like validity, reliability, and the risks of asking hypothetical questions.