Skip to main content

2- Survey Questionnaire

Why Should I Care?

Your Questionnaire can sink the scientific validity of your project. There is a right way, and a very wrong way to question your surveyed population.

Sampling

Typically done on large populations. Sample size magic number is 1,000 respondents which is very large.

Samples are usually random, but difficulty and expense of telephone interviews is increasing use of email/internet, for which random samples are not possible.

Types of Surveys - Questionnaire 
  1. Cross-sectional: Very common. Allows to compare opinions across social markers.
  2. Longitudinal: Less common. Forces smaller samples.
    • Trend
    • Panel 

Example of a Cross-sectional frequency table

Gender / Age (f)
Young (0-25)
Mid (26-55)
Old (56+)
Total (F)
Male
8
37
34
79
Female
9
34
56
99
Total (F)
17
71
90
178

 

Scientific Power

Exploratory: can be, but not likely, since it does require time and money to conduct. Researcher would not get a grant without prior research done to make sure the topic is worth further investigation.

Descriptive: most likely. Especially with cross-sectional.

Explanatory: can be. But here the design is key. To be convincing the questionnaire must be clever and perfect.

Steps

See part 4 p.8

  1. Establish aim / hypothesis
  2. PopulationIdentify population and sample
  3. Type of survey - instrument
  4. Write the question list
  5. Pretest the questionnaire, revise
  6. Administer / Execute questionnaire
  7. Code and Collate data
  8. Interpret Results
  9. Report Results
Operational Definitions

Before you start writing questions, you must make a list of ALL the variables in your project.

Then, you must think of how you will measure these.

Is one question enough?

You may have to ask two or more questions to get all the data you need to cover one variable.

How to write a Questionnaire

Some mistakes are technical (double-barrelled, ambiguous)

Some mistakes are plain evil (leading questions, biased answer boxes, biased matrix)

Methods of Administering Surveys
  1. Individually administered survey: most expensive, higher return
  2. Group administered survey: cost savings vs. individual, high return
  3. Telephone surveys: cost savings, you can explain questions, low return 

    CAN BE RANDOM IF PHONE BOOK IS “N”

  4. Mail-in survey: capacity to reach large sample, low return
  5. Internet survey: cheapest of all, very low return,

  6. CANNOT BE RANDOM

Survey Bias

BIAS: TO HOLD A PARTIAL PERSPECTIVE AT THE EXPENSE OF ALTERNATIVES.

In social science, bias is systematic favouritism in data collection, analysis or reporting of quantitative research.

Can surveyors be biased?

  1. A biased question
  2. A biased sample (non-random)
  • Random = generalizable = no bias
  • Size gives accuracy to a sample, if random. Size does not reduce bias.
Advantages

See p. 127

  1. Generalize to large populations using relatively small samples
  2. Wide array of issues at same time
  3. Usually easy to operationalize variables
  4. Speed
Disadvantages
  1. Might omit important topics if researcher does not know what is important to people
  2. Wording of questions can skew results
  3. Answers are short, may be superficial.
  4. Answers are in degrees, but thinking may be more complex (Ex: Strongly Agree, Agree, etc.)
  5. Once distributed, questionnaire cannot be changed.
  6. Respondents may answer to questions for which they are not knowledgeable.
Reporting

Tables & Graphs

Spatial Maps

Descriptive Text

Revue québécoise de science politique. Société québécoise de science politique.

Canadian Journal of Political Science, Canadian Political Science Association

 European Journal of Political Research, European Consortium for Political Research

 American Journal of Political Science, Midwest Political Science Association

Preferred Disciplines

 Political Scientists, Sociologists, and Geographers.

Ask for postal code, and geographers can map the results.

Economists are not trained in designing surveys. But most of their data comes from surveys such as income, consumption, and unemployment, as “available data”.

Other Non-scientific Disciplines

Applications in…

Politics, such as in-house political party polling, and out-sourced polling services.

Commerce, such as pricing, product marketing, perceptions of brand and reputation.

Public Policy, to build available data such as unemployment rate, GDP, and business confidence.

Not useful for

Historians, Psychologists.

Back to top