Questions must be asked over low CPIs, says Aurora

Aurora Market Research has considered who is really paying the price for discounted online panels and costs-per-interview (CPI).

It seems that with the credit crunch biting all around us, the world of online panels is also feeling the pressure.

Headline offers of discounted CPIs at seemingly cost or below-cost prices have emerged recently, as panel providers are fighting it out to gain new business.

Now, this all sounds good for the research or sample buyer, but who is really paying the price for these offers?
The obvious answer is the panel companies, absorbing the decline in margins to try and gain new business.

This may well be true, but there are other questions a responsible research buyer should be asking before leaping to take advantage of these offers.

The first is how these low CPIs are funded.

Some panels have reduced their incentive rewards to their panellists to help lower costs, in some cases offering just 10p as a reward for a 20-minute survey.

Now, while this helps everyone's bottom line, how does this affect the panellists and therefore the panel itself?
Is this low incentive sustainable in the long term?
This is certainly something to bear in mind when commissioning a tracking study.

Lower incentives can lead to lower response rates, and more ominously, unrepresentative responses.

For example, if by lowering the incentive response rates also decrease, then you are effectively reducing the size of that panel.

A 100,000-strong panel sounds good, but if only a small loyal core are responding at this incentive level, then that 100,000 quickly becomes a far smaller, and far less representative pool of respondents.

Additionally, unmotivated panellists may also respond accordingly, answering less carefully and therefore producing lower quality data.

You should always check the criteria the sample has been picked on to offer you your CPI.

For example, is the panel using pre-screened profiling information, quick-polls or screeners to narrow down their selection criteria and therefore reduce their costs?
Is your sample provider biasing your sample without you even knowing?
A good panel should always tell you the criteria it is using to pick the sample, but if it does not, and you do not ask, how do you know your sample is representative?
For example, you may wish to interview shoppers of a particular supermarket.

The panel may have supermarket-shopped in most-often profiled for some of its panellists, and pick the sample from this information.

But if the sample was selected among 'most-often' shoppers, rather than any shoppers, this would result in you interviewing a more loyal base of this supermarket's customers than you intended, as you will not get shoppers that do not use it as their primary supermarket.

This could correspondingly affect your results.

Now, this sort of bias is not endemic by any means, but as researchers, we have come across examples such as this, which only came to light when we have enquired into the sample-selection criteria, and it should be pointed out that this type of question should always be asked to ensure that the highest standards of sampling are being applied.

Back to top