1- Survey

Why Should I Care?

Surveys are very common. However, many are not done well. There are many traps most people don’t know about that reduce their scientific validity.

Definitions

Survey:
the act of measuring objects or phenomena.

Questionnaire:
an instrument made of a written set of questions, mostly made up of closed-ended questions, and organized in a sequence appropriate to the purpose of a survey, or psychological test.

Interview:
an instrument made of a loose set of questions, mostly designed to allow for an open-ended conversation.

Usefulness of Surveys

Surveys allow for a “real-time” expression of opinions and attitudes on a particular topic. It might only take less than 6 days to conduct a telephone survey. This is very useful for marketers and politicians who might want to change their strategy, or tactics, quickly.

For political scientists, sociologists, and economists, survey results allow for a snap-shot of public opinion at different points in time. This allows to see shifts in opinion, or the impact of specific events on the views. Surveys can also allow for cross-sectional analysis since they often measure socio-demographic variables, such as gender, or mother-tongue language, as well as political opinions, or market preferences.

Video - Basic Steps to do Surveys

Survey methodology is controversial because it is an unregulated industry, and many surveys are not conducted following strict scientific methods. Survey firms are private for-profit companies, and provide their services to both private sector industry, and also to political parties. Many surveys are commissioned and paid-for by lobbies and interested parties. The results - if they were to contradict the interests of the client - are not usually published. In fact, most surveys are never published publically, since they are commissioned by private corporations for their internal needs in terms of strategy, marketing and advertising.

The following documentary was prepared by the CBC in 1997, at a time when the media were publishing too many poor quality surveys, as social facts.

Watch - CBC Documentary

Ask A Silly Question

CBC Documentary - 1997
https://curio.ca/en/catalog/a4b8ee23-1dbe-41e6-ad06-0ab78dc40812

Discussion Questions

Identify a surprising element of the documentary. What did you find surprising?
In the case of the negative advertising towards Jean Chrétien, what question did Marzollini ask in his survey, that Gregg had forgotten? How did this affect both parties strategy?
What can the survey industry do to improve the quality of political and public-interest surveys?

Objects of Measurement

Surveys are useful for measuring many variables, but they are not appropriate for the whole array of variables that social scientists may be interested in. Typically, surveys are excellent for measuring opinions, ideologies, and preferences. Surveys are also used widely by governments to gather information, through the use of forms. For example, when taxpayers fill their income declaration

They can also be useful for measuring personal and socio-demographic characteristics such as someone's age, religion, and gender.

This said, surveys are not adequate for measuring aptitudes, abilities, behaviors, actions, and performances. Respondents are prone to lie, to exaggerate, or simply to misinterpret questions. For example, if you ask someone if they can touch their toes, they might say yes. However, if you ask someone to demonstrate touching their toes, they may not be able to do it. In this case, you are no longer using the survey method, but rather using direct observation in what is called Field Work.

Variable Table for Surveys

Here is summary table of the variables you might want to research in social science, and how they would apply to a survey method.

Type of Object	Yes	No	Maybe	Example
Personal Characteristic	X			Eye colour, Height, Accent
Socio-Demographic Characteristic	X			Religion, Language, Income, Gender, Ethnicity, Marital Status
Opinion	X			I don’t like Rice Krispies Cereal
Motivations			X	Fear, Hope, Ambition, Greed
Ideology	X			Right-Wing, Left-Wing
Biases / Prejudice	X		X	I trust tall people more than short people
Preferences	X		X	Commercial example: pricing, marketing
Personal History	X		X	Immigration, Settler, Indigenous, Divorce
Family Dynamics	X		X	Number of siblings, Rank in Family, Class
Cultural History	X		X	Military History, Political History, Class
Perception / Self-Perception	X		X	Other people are smarter than me
Aptitude / Ability		X		Can hit 3-point shot. Can manipulate others
Behavior / Actions		X		Domestic Violence. Narcissist. Kind.
Personality	X			Agreeable, Irritable, Outgoing
Moods / Emotions	X			Sad, Happy
Performance			X	Productivity, Quality
Level of Knowledge	X			School test. Diagnostic quiz.

Sampling

The sampling techniques used in surveys depend on the type of instruments

Hopefully RANDOM and LARGE, especially when using a questionnaire.

This is possible using a list of the population such as a local phonebook, which is difficult to achieve with email and internet surveys.

With interviews, samples are non-random and rather small.

Types of Surveys

Cross-sectional: compare many independent variables to a dependent variable
Longitudinal: compare a few variables over time
- Trend: similar samples taken at different time points
- Panel: same sample followed through time

Instruments

Questionnaire: set question list, closed-ended questions, larger sample
- In-person
- Telephone
- Internet
- Group
Interview: starting question list, open-ended questions, smaller sample
- Field interview
- Formal Face-to-face interview

Scientific Power

Mostly descriptive studies, which focus on 5Ws.

May also be used for explanatory studies which validate hypotheses and their causal relationships (why), using cross-sectional tables.

Warning – “Surveys” are often used by corporations as a commercial ploy to engage a sales pitch, or to build an email mailing list for future advertising.