ReachCustomersOnline.com

Connect with low-cost tools and know-how
May 8, 2006 :: Volume 2, Issue 4
Next Issue: May 2007

Use Online Surveys To Support Business Decisions: An Interview With Vivek Bhaskaran

Posted by TimSlavin at February 2, 2004

Run your business without customer feedback and you might as well burn your money. How do you dig out customer ideas that make a huge difference to all your customers? Lately online surveys appear to be an inexpensive way for businesses to easily gain customer feedback. To learn more about online surveys, I asked Vivek Bhaskaran, founding member of online survey service QuestionPro.com, to talk about the pitfalls and opportunities that online surveys offer businesses. Vivek is a brainy guy who knows the science behind surveys. I thought it would be useful to quiz him about the practical side of surveys.

Question: (Tim) Are online surveys a waste for small local businesses? Are they just too complex and expensive? Or is the face to face contact in a store or sales call better or more accurate than an online survey? What are the strengths and limitations of online surveys for these businesses?

Answer: (Vivek) Most decisions that small business owners make, from marketing to sales, are primarily driven by “gut feel” -- being a cash flow operated business myself, I know this. However, as your business grows you need to have a more analytical approach to making decisions that support and validate your “gut feel” - one such analytical approach can be surveys.

Now the question of how do you do it - essentially there are four choices - in-person interviews, mail, phone and online. There are pros and cons to each of these modes.

In-Person Interviews
Pros: In-depth and a high degree of confidence on the data
Cons: Time consuming, expensive and can be dismissed as anecdotal

Mail Surveys
Pros: Can reach anyone and everyone - no barrier
Cons: Expensive, data collection errors, lag time

Phone Surveys
Pros: High degree of confidence on the data collected, reach almost anyone
Cons: Expensive, cannot self-administer, need to hire an agency

Web/Online Surveys
Pros: Cheap, can self-administer, very low probability of data errors
Cons: Not all your customers might have an email address/be on the internet, customers may be wary of divulging information online.

In-Person interviews always are better, but the big draw-back is the trap you might fall into if you don’t do them regularly. It is expensive to regularly conduct interviews and not conducting enough interviews might give you false positives.

Validating your research is almost as important as designing and conducting it. We’ve seen many instances where after the research is conducted - if the results do not match up with the “gut-feel” of upper management - it has been dismissed as anecdotal and a “one-time” phenomenon. To avoid such traps, we strongly recommend that data-collection be done on an “ongoing and regular” basis. This will help you compare and contrast how perceptions change depending upon how you market your product or service. The other issue here is sample size. To be confident with your research you have to interview enough people to weed out the “fringe” elements.

The most important reason I think small businesses should look into online surveys is the ability to self-administer them. You don’t need to hire a marketing agency to do it. There are tools and templates available your make life easy!

Q: What are the steps involved in setting up an online survey? What can you do early on to guarantee success with an online survey? What should people look out for?

A: Software companies have realized that there is a huge market for self-administered online surveys. Accordingly there is massive competition and this is great for everyone. It makes software companies come up with really innovative and cheap solutions and allows consumers to harness this as a tool easily.

Setting up an online survey is easy. The first issue is choosing a provider. I would recommend that you choose an option that does not require a software download. This makes things much easier and simpler. You don’t have to deal with the hassle of setting up the software and servers. There are quite a few companies that offer online hosted survey applications that only require a connection to the internet. They handle the hosting, data-collection and the analysis software. See below for a list of providers.

The next step would be to look at the integrated analysis tools that the provider gives you. I think it is really important that the analysis tools be part of the software solution. This makes life easier and you don’t have to buy special analysis software.

The third thing to look out for is the ease of developing and authoring the survey. The tool should have a point-and-click interface to create a survey and publish it. No programming or HTML skills should be required.

Most vendors offer free trial periods. Use this to create a little survey and send it out to friends and family. Spend a little time familiarizing yourself with the survey tool. This will be a huge benefit to you when you actually surveying your customers.

The biggest problem is that most people think online applications as another website - its not. It is software delivered over the web browsers. Hence take your time to familiarize yourself with an online application, just as you would spend time going through all the menus of Excel!

The other thing I think users should look out for is robust functionality. You don’t want to get locked into a vendor and then find out that you cannot do “branching” or certain question types are not supported. This will hurt you in the long run.

Q: How do you pick people to help you with an online survey? What skills are needed? How do you evaluate these skills?

A: Designing an online survey is no different than designing a paper survey or preparing what questions you need to ask your customers in an interview session. If you are unsure of how to do this, there are many market research companies out there that can help you. They’ll usually sit with you to figure out what your needs are and what your objectives are and then help you design the survey and publish it.

Most of these small market research companies use other online survey software vendors to publish and host the surveys.

Here are a couple of things you that can help you decide:

Has the market research provider done prior business in your industry?
Do they have a pre-defined process and plan that they follow?
What exactly are the deliverables? Are they clear and spelled out?
Choosing a market research agency for a project is just like choosing a professional services firm - for example choosing your lawyer. Make sure you have good references and they are up to the job.

Q: What are the different online survey options available? For example, DIY (e.g., phpESP), QuestionPro, Greenfield Online. How do these options differ? How do you tell which option is most appropriate?

A: Essentially in this marketplace there are three different categories: -

Server Based Installation
In this option, you can buy server software and install it in your servers that you run or host somewhere. Note, however, that if you access the internet via a dial-up, this option is not really viable if you run your own server.

Pros: You have total control of the data. You can write custom software that can access the data and do customized CRM integration.
Cons: Very expensive, need technical knowledge about servers and must have a dedicated 24x7 connection to the internet with enough bandwidth. Infrequent product upgrades.
Examples: Perseus Survey Solutions XP, Object Planet, Survey Analytics
Typical Users: Large Corporate Institutions that already have dedicated IT support and staff to manage a data-center.

Online Hosted Self-Service
In this option, you essentially buy the service for hosting and deploying your survey. Almost all vendors that offer this also offer tools for easily authoring (creating) your survey as well as analysis tools. They usually also offer an option to download the raw-data from the servers in the form of Excel Spreadsheets.

Pros: No technical knowledge required. Create surveys using a point and click interface and view analysis online and real-time. Upgrades to software can be performed without affecting you.
Cons: Data is stored with the service provider. If the service provider goes out of business you might loose your data.
Examples: QuestionPro.com, Vanguard Vista, Quask
Typical Users: Small and Medium size businesses. Market Research companies. Marketing Depts. Of some large businesses also prefer this option to the self-hosted model because of the high ROI.

Managed Full-Service
With this option you engage a market research company to take care of the data collection and analysis. Almost all market research firms are familiar with online surveys. Some have their own internal tools that they use and others use software vendors for their online surveys.

Pros: You don’t have to do a thing!
Cons: Expensive
Examples: Herbert Research, Catalina Marketing
Typical Users: Medium and Large Businesses.

Q: Are template questions a good thing or bad thing? How can you tell? Don't they make people lazy when they set up a survey?

A: At QuestionPro we regularly poll all our users, and about 60% of our users have used Survey Templates as a starting point to customize and create their own survey. Templates give valuable food for thought when you are designing a survey. I don’t know about the “lazy” part, but I can tell for sure that it’s saved many of our users an enormous amount of time.

Our Survey Template Library is one of the reasons many users choose us. We are constantly probing the market for better designs and incorporate them into our template library. We offer these templates as part of the license. In fact we don’t even charge you for the templates. They are free. We do not think people should pay an arm and a leg for information and best practices.

I think that you should use templates to get ideas about how you can structure your survey and see all the different possibilities. Then you look at what you really need to do and choose the closest one and start of modifying the template to suite your needs.

Q: What are the different types of questions possible in an online survey? Which are most common? How do you pick the ones to use? What are their relative strengths and weaknesses?

A: Essentially there 4 broad categories:

1. Dichotomous/MultiChomous - Ask the respondent to choose beween a few options. Respondent may be able to choose one or more options.
2. Ratings - As respondent to rate different items. Typically this is in the form of a Matrix.
3. Constant Sum/Rank Order - Ask users to distribute 100 points over a couple of items, or ask them to rank in the order of preference, their choice.
4. Open Ended - Ask for comments, email address, physical address etc.

The general trade-off is really balancing you analysis requirements with easy of use for the respondent. For example, for a demographic question you can ask respondents to Choose their age:

a. You can ask them to input their date of birth or age
b. You can ask them to choose 1 out of say 7 age ranges.

Choosing option (a) gives you more flexibility in terms of your analysis. You can do granular analysis, but increases the frustration on the respondents part.
Option (b) on the other hand is easy for the respondent, but limits your ability for analysis.

What you decide is a balance that is acceptable for your analytical needs and at the same time does not put too much cognitive stress on the respondent.

Q: How might your new site ResearchAccess.net help people understand and use online surveys? Where did the idea come from? What are your goals for the site?

A: We’ve launched ResearchAccess.NET with the hope that it becomes a place for people to ask questions and share opinion on Online Surveys and research. We received a lot of queries and found that there was not a good “resource” site that just had information for people to look at. We believe that online surveys are a form of bi-directional communication with your users. If you are not able to quantify in a meaningful way what your customers really want, you won’t be able to deliver.

We hope that the site becomes a portal and encourages people to do online surveys and in general provides a platform for users to learn more about different technologies that are our there and share and learn for each other’s experiences.

Q: Finally, what's the best way to analyze online survey results? Do you have to be a statistician to benefit from online surveys? How rigorous do you have to be?

A: The most basic form of analysis is the frequency analysis. Essentially you look at how many people chose each option. This, in itself can reveal a lot of things. And no - you don’t have to be a statistician to understand this.

The next step would be to perform “Crosstabs” - Crosstabs measure the co-relation between two questions. For example :-

Q1 - What is your age?
Q2 - How satisfied are you with our product?

What Crosstabs give you is the ability to look into the relationship between Q1 and Q2. Is the age-group 18-30 more satisfied with your products?

The third analysis I usually do is grouping analysis. In this you define target groups - For example -
Group 1 -- “males AND age 18-35”
Group 2 -- “females AND age 45-60”

You can then measure differences in responses to all the other questions to your survey for each of the groups.

Even non-analytical people can benefit from this because all this can be done using a point and click interface.

At QuestionPro we’ve led the industry in technology and are pushing the limits everyday. QuestionPro has the most comprehensive analytical toolset and configuration options in the industry. You can create simple one page questions to complex logic rule-based surveys with looping and branching - all using your browser.

In terms of integration, QuestionPro is the only provider that has built-in integration to post surveys to other online applications like blogs (blogger api) and salesforce.com (sforce api.)

I believe that the software marketplace will be shaped by “niche” vendors providing software that collaborate with each other. Essentially I think that the web will be the medium for users to pick and choose best-of-breed software. Companies like Microsoft, IBM and Sun have made great strides with the advent of “Web-Services” - this has enabled small players like QuestionPro offer great value add services to other companies like SalesForce.com. In November 2003, SalesForce.com launched the Sforce initiative. We at QuestionPro have always believed in integrating with other products and services instead of being an island. We took this opportunity to use the sforce api to integrate with SalesForce.com - to provide users with the ability to send surveys out to their contact database (hosted at Salesforce) and then record the responses as they come in and pump that back into the salesforce database.

Resources Mentioned In This Article


QuestionPro.com
ResearchAccess.NET
Steps in Preparing an Online Questionnaire - How to Effectively Conduct an Online Survey (article by Vivek Bhaskaran)




Comments

Post a comment




Remember Me?