2- Survey Questionnaire
Why Should I Care?
Your Questionnaire can sink the scientific validity of your project. There is a right way, and a very wrong way to question your surveyed population.
Sampling
Typically done on large populations. Sample size magic number is 1,000 respondents which is very large.
Samples are usually random, but difficulty and expense of telephone interviews is increasing use of email/internet, for which random samples are not possible.
Types of Surveys - Questionnaire
- Cross-sectional: Very common. Allows to compare opinions across social markers.
- Longitudinal: Less common. Forces smaller samples.
- Trend
- Panel
Scientific Power
Exploratory: can be, but not likely, since it does require time and money to conduct. Researcher would not get a grant without prior research done to make sure the topic is worth further investigation.
Descriptive: most likely. Especially with cross-sectional.
Explanatory: can be. But here the design is key. To be convincing the questionnaire must be clever and perfect.
Steps
See part 4 p.8
- Establish aim / hypothesis
- Population and sample
- Type of survey - instrument
- Write the question list
- Pretest the questionnaire, revise
- Administer / Execute questionnaire
- Code and Collate data
- Interpret Results
- Report Results
Operational Definitions
Before you start writing questions, you must make a list of ALL the variables in your project.
Then, you must think of how you will measure these.
Is one question enough?
You may have to ask two or more questions to get all the data you need to cover one variable.
How to write a Questionnaire
Some mistakes are technical (double-barrelled, ambiguous)
Some mistakes are plain evil (leading questions, biased answer boxes, biased matrix)
Methods of Administering Surveys
- Individually administered survey: most expensive, higher return
- Group administered survey: cost savings vs. individual, high return
- Telephone surveys: cost savings, you can explain questions, low return
CAN BE RANDOM IF PHONE BOOK IS “N”
- Mail-in survey: capacity to reach large sample, low return
- Internet survey: cheapest of all, very low return,
-
CANNOT BE RANDOM
Survey Bias
BIAS: TO HOLD A PARTIAL PERSPECTIVE AT THE EXPENSE OF ALTERNATIVES.
Can surveyors be biased?
- A biased question
- A biased sample (non-random)
- Random = generalizable = no bias
- Size gives accuracy to a sample, if random. Size does not reduce bias.
Advantages
See p. 127
- Generalize to large populations using relatively small samples
- Wide array of issues at same time
- Usually easy to operationalize variables
- Speed
Disadvantages
- Might omit important topics if researcher does not know what is important to people
- Wording of questions can skew results
- Answers are short, may be superficial.
- Answers are in degrees, but thinking may be more complex (Ex: Strongly Agree, Agree, etc.)
- Once distributed, questionnaire cannot be changed.
- Respondents may answer to questions for which they are not knowledgeable.
Reporting
Tables & Graphs
Spatial Maps
Descriptive Text
Revue québécoise de science politique. Société québécoise de science politique.
Canadian Journal of Political Science, Canadian Political Science Association
European Journal of Political Research, European Consortium for Political Research
American Journal of Political Science, Midwest Political Science Association
Preferred Disciplines
Political Scientists, Sociologists, and Geographers.
Ask for postal code, and geographers can map the results.
Economists are not trained in designing surveys. But most of their data comes from surveys such as income, consumption, and unemployment, as “available data”.
Other Non-scientific Disciplines
Applications in…
Politics, such as in-house political party polling, and out-sourced polling services.
Commerce, such as pricing, product marketing, perceptions of brand and reputation.
Public Policy, to build available data such as unemployment rate, GDP, and business confidence.
Not useful for
Historians, Psychologists.