ESOMAR 26

1.  What experience does your company have with providing online samples for market research?

Datatelligence (referred to sometimes as “DO”) has been in the online industry since the beginning of 2004, and is a subdivision of it’s parent Market Research company CRG Global, Inc., which began in the early 1980s.

SAMPLE SOURCE

2.  Please describe and explain the types of source(s) for the online sample that you provide (are these databases, actively managed panels, direct marketing lists, web intercept sampling, river sampling or other)

We primarily supply sample from our primary actively managed panel, and specifically targeted sample from our sub-panels.

 For over sea projects, we have affiliate partners which manage their own specific panels, depending on country. 

 

3.  What do you consider to be the primary advantage of your sample over other sample sources in the marketplace?

 The primary difference between Datatelligence and other panels are our recruitments methods.  Roughly 50% of our panel is recruited face to face.

Due to our in-person recruitment, specific research methods like Home Use Tests (HUTs) have become a specialty for our panel.  We average an 80% to 95% return rate on single call back/return studies, and 70% return on studies with three or more points of contact, or diaries. 

 

4.  If the sample source is a panel or database, is the panel or database used solely for market research? If not, please explain.

The DO panel is primarily used for Market Research purposes and nothing else.

 

5.  How do you source groups that may be hard-to-reach on the internet?.

DO will periodically send out “incidence checks” to gauge it’s effectiveness in targeting hard-to-get sample groups.  Over the course of a few years, we have built smaller sub-panels and have targets on respondents for over 250 variables from Gamers, Car Owners, Illnesses, Alcohol Drinkers, Cell Phone Users/Providers, etc… 

 

6.  What are people told when they are recruited?

All respondents who join are sent a verification e-mail with their login information and password, with a small description of sub-panels they can join.  This e-mail acts as our second or third opt-in measure and will only be sent to respondents who manually join our community. 

 

PANEL RECRUITMENT 

 

7.  If the sample comes from a panel, what is your annual panel turnover/attrition/retention rate and how is it calculated?

 DO stores data on memberships canceled, bounced e-mail addresses, and the current rate of new panelists joining.

 These rates are calculated at a monthly basis.

 We obtain an estimated 5,000 face-to-face recruitments, in-between 2,500 to 5,000 recruitments through co-registration and about 500 to 1,000 completions directly from our website through word of mouth or respondents stumbling across our web site.  A smaller percentage will come from direct mail or telephone campaigns which we will ask respondents to join if they are interested in further research opportunities.

 Membership for Datatelligence is free and we encourage our panelists to wait for opportunities.  Retention rate is very high; around 98% of our new respondents will remain a member for at least three months and almost all who join will be a member at least the first month.

 Those past the three month marker and are inactive will be marked for our sample “rehab,” a process in which we try to re-engage panelists through a campaign of short studies or Omnibus programs.

E-mail addresses which have “hard bounced” are the primary source of membership removal.  We will remove about 1,000 to 1,500 bad e-mail addresses each month and during peak business season this can jump to about 3,000 e-mails.

 

8.  Please describe the opt-in process.

Face-to-Face:

Respondents are recruited from mall floors, or are asked to join the panel after taking a survey.  They will have three points where they “opt-in.”

 Telephone Interview:

Respondents will be asked if they want to join our online panel after a telephone interview.  They are signed up and sent an e-mail for confirmation.
Co-Registration:

DO’s Co-registration partner will host a web portal on their site in which respondents will read about our panel.  If they decide to join, they will confirm by clicking on a button to be redirected to our specific web-portal for the co-registration partner.  At this point, the respondents is presented with more information about our panel and a web-form, and if completed will complete their double opt-in process.

After opted in, all panelists will receive a confirmation e-mail. 

 

9.  Do you have a confirmation of identity procedure? Do you have procedures to detect fraudulent respondents at the time of registration with the panel? If so, please describe.

We have checks and measures in place to detect duplicate respondent e-mails and will soon have a CAPTCHA system to prevent auto-mated sign-ups.

 Also, all e-mails are verified by a Domain Name System check.  If passed, the respondent’s e-mail will be immediately sent “verification.”  If this comes back unanswered or as a bounced e-mail, the system will flag and remove the respondent.

 

10.  What profile data is kept on panel members? For how many members is this data collected and how often is this data updated?

Datatelligence keeps an optional profile of over 250 variables.  All respondents who joined are encouraged to fill out, and at this point about 70% of our panel has the majority of the extra data fields fill out.

Datatelligence also manages “sub-panels” for respondents to join.  E-mails will be sent out every night to those who have not opt-in or opt-out of join the sub-panels or filling the extra profile data.

 

11.  What is the size and/or the capacity of the panel, based on active panel members given date? Can you provide an overview of active panelists by type of source?

 Datatelligence is currently hovering around 400,000 memberships.  On average, during optimal e-mailing times, given a 24 hour period, our response rate will be in-between 8% and 15%.

 We consider anyone who has clicked in to do a survey within the past six months an “active-member,” using this criteria our active panel hovers around 90%.

 

PANEL AND SAMPLE MANAGEMENT

 

12.  Please describe your sampling process including your exclusion procedures if applicable. Can samples be deployed as batches/replicates, by time zones, geography etc? If so, how is this controlled?

 Datatelligence handles sample through a propriety built software system.  This system was built in field and features were added based on our experience since 2004.

 DO can pull sample from many variables of demographics and information on respondents and deployed by any method.

 Using separate quarantine databases, respondents can be excluded by any factor given by the client (zip codes, states, time zones, regions, etc…).  The same in vise versa, sample can be pulled by any factor.

 Sample outgo can be as specific as the client wants whether it is per time zone, within certain zip codes, or by the default US National Census.

 

13.  Explain how people are invited to take part in a survey. What does a typical invitation look like?

 All Datatelligence invitations are sent via text only unless specifically stated otherwise, all mirror the same content.  Variables which change in the e-mail will be the project number, time invested, and incentive amount.  This is to make sure all sample is as unbiased as possible.

 

14.  Please describe the nature of your incentive system(s). How does this vary by length of interview, respondent characteristics, or other factors you may consider?

 Datatelligence has an array of incentive opportunities for respondents, however primarily using the “Datapoint” system in which respondents can receive on completion of surveys.  At 1,000 Datapoints, respondents may cash out for gift codes of their choice.

 Other incentives include hard cash incentives for HUTs, keeping concept products and gift cards.

 

15.  How often are individual members contacted for online surveys within a given time period? Do you keep data on panelist participation history and are limits placed on the frequency that members are contacted and asked to participate in a survey?

Datatelligence keeps participation history on all respondents.

 The average member completes no more than 1 or 2 interviews in any given month; a maximum of 4 completed interviews is permitted. A maximum of 8 invitations per month are sent to members. Any member exceeding either of these thresholds is quarantined for the appropriate time period. 

 

POLICIES AND COMPLIANCE

 

16.  Is there a privacy policy in place? If so, what does it state? Is the panel compliant with all regional, national and local laws with respect to privacy, data protection and children e.g. EU Safe Harbour, and COPPA in the US? What other research industry standards do you comply with e.g. ICC/ESOMAR International Code on Market and Social Research, CASRO guidelines etc.?

 Datatelligence follows CASRO and MRA guidelines, as the current spam laws, COPPA, and Safe Harbour rules.

 Also, Datatelligence is TRUSTe certified. http://www.truste.org/ivalidate.php?url=www.datatelligence.net&sealid=101

 Please refer to http://www.datatelligence.net/privacy.html for our privacy policy.

 

17.  What data protection/security measures do you have in place?

 Datatelligence uses SSL Certificates and encryption on all online web pages which require user information.

 Respondent interviews links are limited to only one per e-mail address and can not be forwarded after used.  Our survey system uses security keys and matching IDs to validate all incoming respondents.  Any individual trying to access a survey and are not listed in our invitations will be blocked live, automatically.

 Data access on respondents is stored on pass worded servers behind our corporate firewalls.

 All respondent data is kept on separate databases and can only be matched together by an ID which is encrypted.  The IDs can only be unlocked by a special code which changes on a weekly basis.

 

18.  Do you apply a quality management system? Please describe it.

 Yes.  All Datatelligence programs go through several steps before going into field.  To ensure we achieve our client’s goals, we will have kick off meetings with our project managers and the client.  Once everyone is on the same page, we will implement custom checks for the client in our regular QA routines.

Our basic QA routines consist of sending the programmed surveys to our dedicated QA staff.  Any errors or logic gaps caught will be logged and sent to the client for their review.  Once the program is officially reviewed the client, a soft launch will commence for a few completions – enough for our tables/data processing department to look for any holes in the data so we can correct before the full launch.

 19.  Do you conduct online surveys with children and young people? If so, please describe the process for obtaining permission.

 All invitations for surveys to children under the age of 13 will only be sent to the parent for their approval and supervision.

Our teens between 13 and 17 may be a part of our online panel as individuals and may receive e-mails as a normal member.

 PARTNERSHIPS AND MULTIPLE PANEL MEMBERSHIP

  

20.  Do you supplement your samples with samples from other providers? How do you select these partners? Is it your policy to notify a client in advance when using a third party provider? Do you de-duplicate the sample when using multiple sample providers?

 We use outside sources for our international studies outside of the US.  For US national studies, very rarely do we go out for outside sample.  However, when certain studies of low incidence come to rise, we will notify the client we may have to outsource.  If client approves, we will only work with trusted vendors which we have had relationships with for years.

Since we have data quality analysis on all our programs we host, we can gauge the effectiveness/quality of the sample we receive from vendors.  Those of high performance are considered “trusted vendors.”  We do not mind paying higher costs if it means completing our study on time.

Datatelligence is a firm believer that the quality of sample and recruitment sources can affect the outcome of any dataset and is why we always opt for the best and trusted vendors on low incidence programs.

 

21.  Do you have a policy regarding multi-panel membership? What efforts do you undertake to ensure that survey results are unbiased given that some individuals belong to multiple panels?

Due to Datatelligence face-to-face recruitment methods, we have a very low percentage of panel members who are a member of other panels.

DO does keep track of multi-panel respondents, however we do not exclude them as through our research the data outcome is not different from those solely from one panel.. 

 

DATA QUALITY AND VALIDATION

22.  What are likely survey start rates, drop-out and participation rates in connection with a provided sample? How are these computed?

 Within the first 24 hours of a survey, Datatelligence will typically receive a 4% to 8% response from panelists if the sample goes out around 3PM EST.

Response rate is calculated by those who enter a survey divided by those who were invited.  For example, if 100 respondents were sent the invitation, and 15 of them entered the survey, this would be a 15% response rate.

 The response rate can change dramatically depending on the survey, time of sample out go, the text in the e-mail, the demographic group, region, etc… (For example, the 18 – 24 male demographic has a lower response rate than the 34 to 45 female demographic).

 Within a typical seven day study lifetime, our response rates for a non-HUT (Home Use Test) can hover between 8% to 15% and get as high as 25% depending on the sample and groups we send out.

 Drop out rates are highly dependent on the program questionnaire.  For instance, a 20 minute program will have more respondents drop-out than a 5 minute program.  We encourage and work with our clients to make the surveys as engaging as possible for panelists. 

 

23.  Do you maintain individual level data such as recent participation history, date of entry, source, etc., on your panelists? Are you able to supply your client with a per job analysis of such individual level data?

 DO will have information down to a respondent level as to which surveys they have taken, what type of survey, the dates they entered and completed, how many times they were sent invitation for a specific study, where they were recruited from, their incentive, etc…

 Any of this information can be supplied to the client upon request along.  Some information is sensitive and may require the client to sign an NDA.

 

24.  Do you use data quality analysis and validation techniques to identify inattentive and fraudulent respondents? If yes, what techniques are used and at what point in the process are they applied?

First and foremost, DO would not expect a client to use any respondent data that was mutually agreed to be questionable.

With that said, Datatelligence believes the data quality begins at the recruitment level. Our unique recruitment methods have provided us a panel community that is recognized by our small but growing user base, as exceptional.

DO begin at the very onset in its panel development to incorporate the CRG Global Confidata Validation System to increase the overall quality of the data. This was started 2 years before the outcry from industry clients to address the quality issues, it was not something introduced grudgingly. DO focuses on eliminating potential satisficers at the screener level and concentrates on eliminating speeders by measuring time during at least two different intervals of the main body of the survey. These steps are quite simple and prevent respondents with poor behaviors from ever entering the data set. By monitoring this we can successfully eliminate that largest potential for poor data. Our belief is that each type of poor respondent behavior is not mutually exclusive of one another, eliminate the top two and you eliminate the overwhelming majority.

Datelligence attempts to design surveys that are user friendly and more engaging. We prefer to use drag and drop over radio buttons, we avoid scrolling. We minimize the amount of screen real estate used by maximizing the number of questions per page. With client permission we will provide break points in longer surveys. DO is constantly looking to improve the respondent experience and listens to feedback from our panel.  

 

25.  Do you measure respondent satisfaction?

 Yes, we send out newsletters and short surveys asking respondent’s opinions about incentives, program design, and the overall general feel of the panel.

 Due to the “open end” nature of some of these surveys, we will read and aggregate certain issues which need to be addressed to make Datatelligence a better community for its members.

 

26.  What information do you provide to debrief your client after the project has finished?

 Due to our very detailed live reports, there is not much “debriefing” for the client.  We don’t want surprises at the end.

 All dispositions are recorded and updated on online report links live every hour; this includes dispositions and completes on a per question basis or any banner the client requests.

 Some live reports are special, and in the kick off meeting we can have our programmers code special variables to keep track of any counts by any specific logic the client requests.  These counts will also be live.

 The basic live report on a national study will have Age, Gender, Region, and Ethnicity by those who “click-in,” complete, screen out, or is just “active.”  This way the client can tell how much of the sample is mirroring US Census, or by any count they are trying to achieve.  Due to the openness of our reporting, we have perfected our targeting techniques, as we consider our selves second to none in accuracy.