How do I find a co-rater?

[amcate]

---

Members don't see this ad.

Hello,

I'm a doctoral student in education and am developing an instrument in an area of educational psychology for my dissertation. Specifically, the instrument assesses moral reasoning. I'm trying to find someone to be my co-rater so that I can find the inter-rater reliability of my instrument.

Here're my questions:

1. How do people find their co-raters?
2. How much should I pay the person?

Please advise. Thank you very much. 🙂

 

[Phipps]

Hello,

I'm a doctoral student in education and am developing an instrument in an area of educational psychology for my dissertation. Specifically, the instrument assesses moral reasoning. I'm trying to find someone to be my co-rater so that I can find the inter-rater reliability of my instrument.

Here're my questions:

1. How do people find their co-raters?
2. How much should I pay the person?

Please advise. Thank you very much. 🙂

Hi,

I just finished my thesis on memory recall and emotion in bilingual speaker and was faced with the same issue. In order to calculate Cohen's Kappa, I needed an inter-rater. My initial idea was to approach fellow students but our department was unable to find any. I then found a person of a friend of a friend...and had to introduce him to the entire research topic and how to code what I had analyzed before. I didn't pay him but invited him for dinner 🙂

Ideally, more than two inter-rater are needed though.

Hope that helps.

Good luck.

 

[amcate]

Phipps,

Thank you so much for sharing your experience with me. This is very helpful. I'll try to look for a friend or a friend of a friend like you did.

May I ask how long did it take your inter-rater to analyzed your data? Mine is likely to take several days even weeks. That's why I'm trying to figure out an appropriate compensation for that person.

Thanks again for your reply.

 

[Phipps]

Phipps,

Thank you so much for sharing your experience with me. This is very helpful. I'll try to look for a friend or a friend of a friend like you did.

May I ask how long did it take your inter-rater to analyzed your data? Mine is likely to take several days even weeks. That's why I'm trying to figure out an appropriate compensation for that person.

Thanks again for your reply.

Sure. Glad I can be of help. I did a pilot study with seven interviewees only and had everything well prepared for my inter-rater, i.e., counting sheet, typed up meaning units, section where to put in what, and I also wrote up a detailed 'manual' for that person, etc. I believe it took him about three days. Not sure how many hours though.

Since he is a friend of mine, I now owe him 🙂 I would have preferred grad students (and more than more other rater) but you have to roll with what you get...I guess.

 

[amcate]

Thank you very much 🙂

 

[roubs]

Do you have undergrad psychology students at your school? You can pay them in recommendations! (About what a great research assistant they were!)🙂

 

[amcate]

Thank you roubs. You made a good point about the recommendations.

My university has both undergrad and grad students in the psychology department. When I talk to the psychology department, I'll keep in mind to mention that I'll be more than happy to provide great recommendations to my raters.

 

[Phipps]

Thank you roubs. You made a good point about the recommendations.

My university has both undergrad and grad students in the psychology department. When I talk to the psychology department, I'll keep in mind to mention that I'll be more than happy to provide great recommendations to my raters.

How about the opportunity to earn extra credit for a class?

 

[roubs]

Thank you roubs. You made a good point about the recommendations.

My university has both undergrad and grad students in the psychology department. When I talk to the psychology department, I'll keep in mind to mention that I'll be more than happy to provide great recommendations to my raters.

I think it's usually a given that if someone does good work for you that you'll provide a rec, it's probably not necessary to highlight that. Many depts do offer directed research credit to undergrads and you could see if you could get hooked into that process

 

[amcate]

The course\research credit idea is really interesting. I never thought about it. It would be a win-win situation if they let me hook into that.

Thank you so much for your extremely helpful suggestions.

 

[AcronymAllergy]

The course\research credit idea is really interesting. I never thought about it. It would be a win-win situation if they let me hook into that.

Thank you so much for your extremely helpful suggestions.

This is how we find the majority of our RAs in my department (i.e., through students who sign up for a special class in the psych department and subsequently receive course credit).

Even if such a program doesn't exist, there are likely to be undergrads in your university who are looking to gain whatever research experience they can find in preparation for applying to grad school. Get the ok from your department first, and then look into posting flyers in the education and psychology buildings recruiting co-raters. If you felt the urge to pay them for their time, I would think minimum wage + letter(s) of rec + the boost to their grad school apps would be more than fair.

 

[psypsypsy]

The course\research credit idea is really interesting. I never thought about it. It would be a win-win situation if they let me hook into that.

That's how almost all the undergrad RAs in our department work as well. There is a course number they sign up for and they're expected to do 3 hours of work per credit hour per week. (Our RAs sign up for 2 or 3 credits of research per semester, and we require a 2 semester commitment, which is never an issue).

I find one aspect that helps with recruiting them is that at our university, the course is signed off by a professor, not you. So, the class they're taking will officially be under the supervision of Dr. XX (probably your mentor). I had a group of undergrads working for me for my dissertation project, and I think that one of the advantages was that they got close contact and mentoring with me, but I also scheduled a few meetings with my mentor, so they got some exposure to him as well. I helped write the rec letters for grad school, so they were personalized to their experience, but the letter officially came from Dr. XX, which is a much bigger and better name to get a recommendation from. So, check in with your mentor, but I thought that system worked great, and undergrads are happy with it.

 

[amcate]

AcronymAllergy and psypsypsy, thank you so much for your suggestions. I'm really appreciated your comments.