Our sample

CARP has approximately 300,000 members across Canada (about 1% of the population), and, of these members, about 87,000 receive CARP ActionOnline, the twice monthly newsletter. This is our sample universe. Of these members, typically 25,000 open and read the newsletter. This is our polling panel, and it represents about 8% of our members, or universe. Response rates to our polls are typically 3000 to 4000, or about 15%.

How the CARP Poll™ compares to other panels

These data compare very favourably with the many commercial internet polling panels available in Canada. Many claim to represent the entire population and are made up of about 30,000 to 50,000 members. This is about 0.2% of the population or universe. Surveys conducted among members of these panels typically attract no more than 200 to 500 responses, for a response rate of about 1%.

How representative is the CARP Poll™ ?

We have a database of demographic data on the entire membership, collected during a large-scale random selection telephone survey. Comparison of this database to the demographic data we collect with each poll shows that our panel is representative of the entire membership apart from the following divergences:

• Obviously, members who respond to the polls all have computers and an internet connection, which is the case for about 35% of Canadians over 65, but 60% for members (the average CARP member is 69 years old)

• In a characteristic related to online access, members as a whole are likely to have slightly lower average incomes and educational levels than those who answer the polls

• Poll respondents tend to skew male, while the overall CARP membership is relatively evenly balanced by gender

As a whole, the CARP membership, whether or not they are online, is slightly wealthier and better educated than the Canadians population of the same age.

Nonetheless, we consider the CARP research panel to be comprised of our most active, engaged and informed members, and as such, is an appropriate resource for in-depth input to the goals of a membership advocacy organization.

The CARP electoral preference measure

Since the middle of last year, we have included the political preference question (“If a federal election were held tomorrow, which party’s candidate would you vote for?”) in every poll, and we have a database of occasional probings going back to the last federal election. With the exception of a few discrepancies from the national electoral polls (fewer Bloc supporters, more Conservative supporters), our results track along with, and sometimes presage the national results.

How it works

How does a group of 3000 respondents represent the opinions of 300,000 members? Because it’s a sample of the universe, not a census (or tally of the total number). A census is simple, you count every vote, but it’s terribly time-consuming. A sample isn’t as simple, it involves calculation, but it’s much easier.

Imagine you have a huge jar of jellybeans of all colours. You want to know what proportion of the whole are black jellybeans. You could dump the jar out and count them all, or you could count a sample. If the jar is thoroughly mixed (or randomized), you can reach in your hand, pull out a handful and count that.

You can be sure, within a certain range of probability, or margin of error, that your handful will match the composition of the entire jar. The bigger your handful, the smaller or more accurate your margin of error. Also, the more extreme (high or low) the percentage you’re looking at (the observed percentage), the smaller your margin of error. So, in samples with very big handfuls, like our polls, the margin of error is very low (typically plus or minus 1.5%, 95% of the time), especially when we examine small subgroups.

What the members think of our polls

This summer, CARP polled its panel on their attitudes and reactions to our polls. It turns out they like them, fill them out regularly and find them fun and interesting to complete.

Two thirds of our respondents say they fill out the poll “frequently” or “always” (63%) and we would have to consider this core group of 2000 to 2500 members the heart of our panel. Participation is higher in BC than elsewhere and lowest in Quebec. Younger males are more likely to participate than older females, and those in lower income brackets are more likely to take the poll than others. Conservative party supporters are especially enthusiastic respondents.

Members spend, on average, just over 7 minutes completing the poll (which typically has 20 to 25 questions) which conforms to the polling industry rule of thumb of 3 to 5 responses per minute. Quebeckers spend the longest with the poll, as do older males in lower income brackets who support the Conservatives.

Just less than one third of respondents fill out the poll the night they receive it (30%), while a further quarter (26%) complete it the next day. Filling out the poll immediately is characteristic of older females in lower income brackets in BC who support the Conservatives or the NDP.

Virtually all our respondents (95%) say they find the poll interesting, and about one third use the strongest terms (“very interesting” – 37%) to describe it. Older females in lower income brackets who support the NDP or the Liberals are the most likely to describe the poll as “very interesting”.

Healthcare is the most popular polling topic, followed by pensions:

Which subject is the most interesting to you, and would lead you to be especially likely to complete the poll?

Healthcare 23%
Pensions 21%
Politics 14%
Retirement 11%
Finances 9%
Other social issues 5%
Aging in poverty 4%
Ageism 4%
OTHER 1%
DON’T KNOW 9%

Men are more interested in finances and pensions than women, and women are more interested in healthcare.

Fully three quarters think the CARP Poll is an appropriate length (73%), whereas about one fifth find it “a bit too long” (20%). Males in Quebec who support the Conservative party are the most likely to find the poll too long. More than two thirds (69%) find 20 to 25 questions “neither too long nor too short”.

Close to eight-in-ten respondents (79%) say CARP polls them at the right rate of frequency (typically twice a month) and this is most common in the Atlantic provinces.

The vast majority of respondents find the poll “neither too complex nor too simple to understand” (89%).

While close to half say they complete the poll “only to contribute opinions” (46%), an equal proportion say they fill it out “mostly to contribute opinions but also for fun” (48%).

As for the end-use of their contributions, just less than two thirds say that CARP uses the gathered data well (61%), and this especially the case for females, in the Atlantic provinces and in lower income brackets.