Conducting internet survey research

[robotsahoy]

I'm currently looking into doing some independent internet survey research and am looking for a few pointers. There are several services that allow you to design surveys & collect the data, but how do you go about getting people to the surveys? One common sense idea is looking for user forums related directly to the subject (e.g., making a post on vnboards.ign.com if you're researching video gamer populations), but does this lead to problems with having a population that is already biased because only a minority of such a population uses those forums?

How about taking out Google AdSense ads? This seems like a cheap way to get a high number of hits (I've seen some reports of over 100,000 hits over 6 months for a cost of $24 or so).

---

Members don't see this ad.

 

[Therapist4Chnge]

Are you sure it is $24 for 100,000 that seems awefully low. Do you get charged per view or per click through? The person would then have to take the time to complete a questionnaire. You could blow out your budget on poor matches, depending on the key words, cost/hit, etc.

Just some thoughts.

-p

 

[robotsahoy]

Ah, yea my numbers were way off. What I saw was 409 clicks out of 259,000 hits (~0.16% click rate) and its billed per click through apparently. The billing came to around $23 after 6 months, which doesn't seem shabby for 409 click throughs. However, these numbers are only for click throughs the ads. I have no idea what percent of click throughs would follow up and take a survey.

Cost effectiveness of this might be pretty bad after even if only say 10% of the 406 click throughs followed through on a survey. Any other ideas? Maybe I should email Nick Yee about how he got his project rolling.

 

[Therapist4Chnge]

$.056 isn't too bad.....it really depends on the popularity of the word(s). The more targeted you need your subjects to be, the worse the fit will be. There is a reason why college kids are often the population surveyed (easy access).

-t

 

[Ollie123]

Realize this isn't exactly your question but I wanted to throw out there that www.surveymonkey.com is a great site that seemed to work very well for a study I put together. Its dirt cheap and easy to use. Definitely take a look if you haven't yet

Unfortunately we just directed undergrads to it, so I'm useless as far as your actual question is concerned. Do keep in mind that solely online survey means you will probably get a population skewed towards the younger, higher SES side. That's just common sense though I've seen papers published on that as well, I can go try and track them down if you have troubles finding them yourself.

Depending on what your study is though, it isn't necessarily a disastrous problem.

 

[robotsahoy]

Realize this isn't exactly your question but I wanted to throw out there that www.surveymonkey.com is a great site that seemed to work very well for a study I put together. Its dirt cheap and easy to use. Definitely take a look if you haven't yet

That is a great site and definitely my top choice so far.

The SES issue shouldn't be too much of a problem, as I'm looking into researching personality traits among users of a particular subset of internet sites.

 

[Therapist4Chnge]

Good site!

I've never hosted a survey there, but I've heard good things, and the interface seems pretty easy to use (from the user side).

-t

 

[JockNerd]

I'd suggest you check out the papers by Andrews, Nonnecke, and Preece for some guidelines on conducting internet surveys, if you haven't already. Their work contains some great stuff on standardization for internet sample collection as well as some review of how samples were obtained in work they reviewed.

Sample representativeness can be a huge problem in net-based sample collection. For example, in the area I research (LGBT identity), I've seen work that draws a net sample from online dating (and sex solicitation) web sites. So, obvious this isn't representative of same-sex attracted people at large, but rather that minority using net sites to meet people for anything from friendship to anonymous sex.... Applying it to the example you used, how many gamers visit those kinds of gaming forum sites? My guess would be a significant minority, and very likely it would be people for whom gaming constitutes an aspect of their social identity. Not too sure about that!

I tried to get permission from my dept. for my honours thesis data collection to be online (using the standard uni students, but having them do the survey from home rather than in a lab). My school's ethics board wanted me to buy a dedicated server and keep it in a locked room in the dept. building. Ugh. I just did it pen-and-paper.

 

[psychwhy]

I used SurveyMonkey and while I agree the price is certainly managable, their interface is very basic. There are several other survey hosting companies. Some, for a higher price, obviously, will send your survey to a group of subscribers. (The subscribers are offered inducements by the host company.)

You should be VERY careful venturing out to the Web yourself. Due the proliferation of spam, most commercial sites simply do not permit you to post a link to another site. Non-commericial usenet (discussion) groups often have moderators who do the same thing. Even if they don't you might find yourself set upon by board members who feel your solicitation notice violates their "community."

If I were to do another online survey, I'd probably opt for the survey host with the subscriber base (though, of course, your sample is then limited to those who have subscribed! )

 

[Ollie123]

Good point about surveymonkey....it IS very basic.

I don't know what you need for your survey...if its anything complicated or if it requires some very careful formatting of questions you might be best off looking elsewhere.

If its a straightforward sequence of multiple choice questions or something, it should be fine.

 

[LosingMyMind]

I used Surveymonkey for my honors thesis... I had two surveys up... the population for one of which probably overlaps the one you want. I passed out the link to students in all the intro psych classes. Out of 450-500 students, I was able to get 250 responses... Fantastic, I think.

The set IS very basic, but it is certainly helpful to have all your data already entered for you... I know that's probably standard for all computerized surveys... but man, was it helpful!

Basically, as was said, if you don't need anything too complicated, surveymonkey is a very useful tool and there are some very helpful features you can take advantage of...