i.on.research 2.1

Online Panel Motivations: It's Not (Just) About the Money

2008-02-20T10:47:00.001-08:00

Why on earth would anyone want to participate in online marketing research? The assumption too often made is that the basic and only significant driver or motivation is some kind of monetary reward.

But that is not necessarily the case. There is a complex array of motivations that online panellists cite as being important and motivating to them. A couple of years ago, my firm Prophis Research, building on the academic work of Elisabeth Deutskens, conducted extensive motivational research with its US and Canadian access panel members to dig deeper into its collective psyche. Click on the slideshow below to see the "whole show".

View Upload your own

The figure below shows how relatively important various kinds of motivations were found to be among panel members for participating in research via the online panel (the bigger the balloon, the more important it was rated as a motivator to participate in research).

While money (or something like it, or the possibility of it) remained a key motivator for all panel segments, there was only one significant panel segment (i.e. called the "Cash Only" segment) for which this was essentially the only motivator.

Much value was also derived simply by curiosity or interest, knowing one was helping organizations and firms improve their offerings, or in seeing how their own responses stacked up against others or in being able to interact with others.

Panels that are able to deliver on a number of these key motivational points have a number of advantageous outcomes:

it helps keep panel members active and more likely to participate
helps balance the panel against psychographic and other skews that may arise where motivations are unitary
promotes positive image of marketing research industry

Obviously the existing and emerging Web 2.0 tools specialize in delivering on precisely the kinds of non-cash motivations that people profess are important too. But much of these motivations can be enhanced with Web 1.0 level expertise -- telling people about the good their feedback is doing, for example, is one such no-brainer too often undeveloped that is worthy of consideration.

Optimizing Access Panellist Engagement

2008-02-20T06:48:00.000-08:00

Questions lead to wisdom and answers lead to folly, so the old saying goes. Vogue (and vague) concepts can have the lure of the elixir that cures all ills yet all too often dissappoint. But dissappointment is avoidable, if questions, not answers, drive the analysis.

Case in point. "Engagement" is one of those terms that is flying around with reckless abandone these days. Seminars, webinars, white papers, they all talk about how this is the edge you need to drive your competitive position. Of course, depending on your industry, business, and what drives value for your customers, engagement may be an important component of the value chain.

But in the context of online panel management, engagement presents as a two-edged sword.
For online panel based marketing research whose goal it is to project findings to populations of interest, which relies on talking to "normal" people, this must mean that interactions and activities on the panel are fine if they can lead to greater (i.e. more frequent) participation in research, but not if that these same interactions lead to unknown or unknowable distortions.

For example, Market Tools is a custom panel provider that understands the dangers. They show how the most engaged (i.e. the panel respondents who respond to surveys the most often) are systematically more likely to rate buying intentions and products negatively than respondents who are far less engaged. The problem for specialty panels, that are thematically focused on a particular business or industry, are subject to even greater distortive tendencies.

It is a challenge to managing panels effectively. As a practical matter, it is key that this notion of "level of engagment" is actually something that is actively controlled through the sample selection process, to ensure a mix of respondents.

The other implication is that is may be a good idea to start conceptualizing respondents in terms of life-cycle use. In the early days, a fresh (i.e. in the sense of infrequently engaging in marketing research) respondent sees with the eyes of the public. But after some point (what the point is is a matter to be determined) a respondent that has become unusually expert at giving opinion should be identifed as an "expert panelist". This would effectively exclude them from quantitative projects but qualify (or at least not disqualify) them for valuable input in qualitative research where their expertise can be seen as more asset than liability.

The best access panel management practices inevitably must incorporate not only strategies to promote participation, but strategies to continue active recruitment of new panellists, and those to understand the potential distorting effects that can be bred through panel membership itself.

Marketing Research's Shifting Paradigm

2008-02-19T08:33:00.000-08:00

As in almost every industry sector, Internet technology is driving change throughout the marketing research industry. Whereas the industry used to rely most heavily on phone-based data collection for general population research, now the Internet is gaining significant traction. In so doing, our industry struggles with the question of how to project conclusions about populations based on samples that were not drawn from said populations but rather assembled to resemble them. The struggle is one between two distinct projection models or paradigms: deductive versus inductive.

Since the golden age of the 1960s when pretty much everybody in western countries lived in a household with a landline telephone and most would answer calls from marketing research firms and participate in their phone surveys, modern sampling theory went ahead according to a straightforward deduction.

The logic was simple. Landline telephones were ubiquitous, and because researchers could come up with a sampling scheme using random digit dialing that ensured everyone had an equal chance to get contacted, the deductive tenets of sampling theory (e.g. known universe of respondents, all respondents have equal chance to be included in sample, etc.) were met more-or-less perfectly. Marketing research, public opinion, and social research firms alike all received excellent participation from an enthusiastic public, which further ensured that confounding non-probability error sources (e.g. non-response biases) were basically non-issues. But that was a long time ago.

Soon the success of this data collection methodology contributed to its overuse and slow demise. Other factors such as the rise of telemarketing has also taken a hefty toll on the willingness of the public to talk to people they don’t really know on the phone. Refusal rates have steadily climbed over the past decades.

In more recent times, the rise of the “cell phone” only households has also become an increasing problem in western countries. Pew Research frames this emerging problem for US marketing research practice well. The CDC (US Centre for Disease Control) currently projects that 15% of the US population does not live where there is an active landline phone.

As phone penetration continues to morph, Internet penetration continues to grow. While the medium itself has a range of pros and cons vis-à-vis other methods such as phone, the biggest challenge in the first decade of the Internet’s commercial existence has been figuring out how to deal with sampling the general population and drawing conclusions about it.

Historically, in western countries, when penetration was lower, the demographic, psychographic, technographic, what-you-will-graphic profile of the Internet user was much more “early adopter” hip than “vast majority” normal.

But penetration has been improving all over the world and continues to do so. In some western countries, Internet penetration meets or even exceeds landline penetration rates. This is particularly true if one looks exclusively at the “under 65” group, which is sometimes the defacto general population of interest for marketers.

The reality of spam, however, precludes sensible and ethical professional marketing researchers from going about coming up with ways to reach net users “at random” or “uninvited”. So they have had to be inventive.

Professional marketing research practice is moving towards a new model for drawing samples and conclusions to populations of interest. Online access panels represent this new paradigm.

Firms first recruit a respondent base that is big enough and this is the important point, representative enough (i.e. in the sense that the characteristics of the sample resemble those known about the population to which projections are to be made) so that a sample drawn can be used as the basis on which projections can be made. The more and more varied the sources of recruitment, the better. The extent to which skews (demographic or otherwise) can be known and understood, they can also be built into the model. Think of it as an inductive approach to sampling theory.

There is already proof that this method can work. Election cycles are the best times to test the accuracy of public opinion based research. Companies like YouGov, based in the UK, have come up with a set of collection and analytical practices that puts Internet panel based methods on an equal footing with other so-called traditional methods based research when it comes to producing findings that can essentially be verified by election results. My own firm, Prophis Research, has also conducted public opinion research with our own online panel of US consumers and compared it against conducted by both online and phone-based research against the same populations and come up with findings within the statistical margins of error. Angus Reid Strategies is another firm whose Internet panel projections have been accurately predictive. And these companies are far from the only ones to make the leap. Yet others remain firmly skeptical.

Critics of this inductive approach argue that it is simply not valid. Certainly within their own set of assumptions they may be technically correct. This is NOT the old way of doing things. But proponents of it argue that they are simply trying (and succeeding) in understanding the limitations of their methodology and to overcome it with a superior analytical treatment. Their counter-charge would be that phone methods do not take into account non-response bias associated with it, for example, and will therefore more and more suffer the indignity of returning “wonky sample” results.

The business of marketing research is about being able to make the claim that sample findings mean something with respect to the population that they purport to represent. Otherwise, what’s the point of doing the research?

So not a little bit is at stake as to whether research methods, whatever kind of methods, get it “right” 19 times of 20. The current election cycle in the US will provide an interesting testing ground to see what firms and methods are able to claim their own accuracy victories once the votes are counted.