Stimulus Materials and Researcher Instructions – Spring, 2021 – Fake News Study
Instructions: This Spring, we are going to run a series of studies looking at how comments regarding “fake news” on Facebook impact participant ratings of Facebook users. Our independent variable in the study (the variable that we control) is how a Facebook user responds when others call a story the user shared “fake news”. That is, the fake news poster will either politely agree that the story was fake, politely disagree that the story was fake, or rudely disagree that the story was fake. We will measure what participants think about our fake news poster as well as the fake news itself.
1). For your first experimental study, you will play the role of researcher, and you will collect data from three different participants (though you will combine your data with other class members, so your final data set will have 100 to 140 people!). There are two phases to this study. In the first phase, you will orally ask participants if they are willing to participate in a research study. In the second phase, participants will complete a five-part survey. In Part One, participants read the Facebook page for Corey McMillan, getting some general information about Corey and looking at a recent Facebook story that Corey shared as well as several comments regarding the validity of that story. In Part Two, participants will rate their impressions of Corey (Note that the name “Corey” is gender-neutral. This way all participants can complete ratings about Corey that don’t rely on gender expectations). In Part Three, participants will rate Corey’s comments as well as the comments of two other Facebook users. In Part Four, participants will complete demographic questions. Finally, in Part Five, participants will tell us what they recall about Corey’s final response. To run this study, use the following steps:
A). Your first task is to approach three different participants (not all at the same time!). Preferably, they will be people that you do not know, and cannot be taking a psychology research methods class during the Summer or Fall semester of 2020 or the current Spring 2021 semester. Please DO NOT complete this study yourself, and if possible use only FIU students as participants (no family / friends – You will use them in a later replication study toward the end of the Spring semester, and they cannot participate twice). There are 48 students in our class, so with each getting data from 3 people, our final sample will be around 140 participants total.
1). Note that there will be a “Covid alternative” to data collection if you are unable to collect data yourself. Ask your instructor about that option, but there is a good chance that you will already see some pre-completed “Covid Alternative” documents in Canvas.
2). Even if you use the “Covid Alternative”, read the information below, as it will help you write your future papers. You don’t need to mention that you used the Covid Alternative, but you will pretend like you did collection the information yourself.
B). Phase I: Informed Consent
1). Informed Consent:
- Ask the potential participant if he or she is willing to participate in a study for your research methods class. You will get their informed consent verbally. Tell them:
“Hello, I am conducting a study for my research methods class. I was wondering if you would be willing to participate. The study takes about five to ten minutes. There are no risks to participating, and the main benefit is that I can complete my class assignment. Will you participate?”
- An oral Yes or No response is fine. If they say no, thank them and find a different participant. If they say yes, move to the next step (Phase II – Questionnaire).
C). Phase II: “Questionnaire”
1). General Instructions
- After getting participant’s oral informed consent, randomly give them ONE of the three “Research Study – Florida International University – Spring 2021” documents. These three documents contain our independent variable for the study (Again, note that we have one independent variable with three levels). One third of our research participants will be in the “Rude Disagree” condition, one third will be in the “Polite Disagree” condition, and one third will be in the “Polite Agree” condition (More on this below!)
- Ask participants to follow the instructions at the top of the questionnaire. Tell them to read EVERYTHING on the Facebook page, as they will answer questions about it later and will need to do so through memory. They can move through the five “Parts” in this survey at their own pace. Make sure they complete all questionnaire parts (though they can leave some demographic questions blank if they do not want to provide the details).
2). Questionnaire
- In Part I, we ask participants to look at the Facebook “About” page for a person named Corey McMillan. The page contains a picture masthead (full of American flags) and Corey’s profile picture (a silhouette of a man and woman hugging – Note that it is not specified which person is “Corey”, so Corey could be either the man or the woman). The page also includes a generic “Intro” section, “Photos”, “Friends”, and “Life Events” on the left side of the page. On the right side of the page is a comment from Corey followed by a story Corey shared (“FCC forces CNN to change its accreditation from “news” to “entertainment”). The story is followed by two comments by other people (Peyton Halliburton and Riley Anderson) who both say that the CNN story Corey shared is fake news. Please note that EVERYTHING on Corey’s Facebook page up to this point is identical across all three of our conditions (but don’t tell participants that!). So what differs? Corey’s last post at the bottom of the page.
- In Condition RD, or the “Rude Disagree” Condition, Corey responds in a rude manner to Peyton and Riley, disagreeing that the story is fake. Corey mocks the two commenters (who called his shared story “fake”) by going on a rude tirade. Specifically, Corey says:
- “You liberal idiots never think, do you? Of course the government can regulate cable news networks when it comes to news items. Like I said, look it up if you don’t believe me (If you can pull your heads out of your butts long enough to do a quick google search you’ll find out for yourselves!). The US government holds news stations to really high standards when it comes to accuracy and truthfulness. You morons can’t call the story I shared fake just because it doesn’t support your own point of view.”
- In Condition PD, or the “Polite Disagree” Condition, Corey responds in a more polite manner, though still disagrees that the story is fake. Corey says:
- “You gave me some stuff to think about. Unfortunately, you guys are wrong. Of course the government can regulate cable networks when it comes to news items. Like I said, look it up if you don’t believe me (If you do a quick google search I’m sure you’ll find that out for yourselves!). The US government holds news stations to really high standards when it comes to accuracy and truthfulness. You can’t call the story I shared fake news just because it doesn’t support your own point of view.”
- In Condition PA, or the “Polite Agree” Condition, Corey responds in a polite manner and agrees that the story is fake. Corey says:
- “You gave me some stuff to think about. Unfortunately, I was wrong. The government doesn’t regulate cable news networks. I told you to look it up if you didn’t believe me (Then I did a quick google search myself and found out you were correct!). Although the US government should hold news stations to really high standards when it comes to accuracy and truthfulness, cable news isn’t regulated. I don’t like it when people call something fake news just because if it doesn’t support their own point of view, but in this case the story I shared was fake.”
- A quick note for you (the researcher): If you look at the bottom of the survey in the fine print Facebook information (Privacy, Terms, Advertising, etc.), you will see the letters RD, PD, or PA. Those relate to the condition for that specific survey. The RD stands for Rude Disagree, the PD stands for Polite Disagree, and the PA stands for Polite Agree. The initials will help YOU quickly know which survey is which, though the initials should be meaningless for participants. No need to mention this in your future papers, but it is a god way to keep track of your surveys
- In Part II, participants will give their impressions of Corey McMillan (the original Facebook user and fake news sharer). There are eight statements about Corey that participants are asked to agree or disagree with. All eight are rated on an interval scale ranging from 1 (Strongly Disagree) to 6 (Strongly Agree). These statements include, “Corey seems like a reasonable person”, “Corey seems like a polite person”, etc.
- Although you can look at any (or all) of these eight questions when you write Paper II (which focuses on the methods and results for this study), we will probably only focus on one or two. The ones I find most interesting are question #3 (“Corey seems like an open-minded person”) and question #6 (“Corey seems tolerant of opposing opinions”).
- In Part III, participants rate seven additional statements, with most focused on the Facebook comments. These questions probe into the extent to which participants thought the CNN news story was fake (before and after reading the comments), whether the participant would add their own comment if they saw the same conversation on social media, and the emotions they felt after reading Corey’s last post.
- The ones I find most interesting are question #3 (“If I saw this Facebook conversation on my own social media, I would add my own comment”) and question #7 (“I would be willing to engage in a back-and-forth conversation with Corey”).
- In Part IV, participants will complete demographic questions. Most of these items are easy to complete without violating participant’s privacy, but they will know they can leave blank any question(s) they feel uncomfortable answering.
- In Part V, participants will tell us what they recall Corey saying in response to the two people who commented on the fake news story (“Corey politely disagreed that the shared CNN story was fake”, “Corey rudely disagreed that the shared CNN story was fake”, or “Corey politely agreed that the shared CNN story was fake”).
- Unlike the statements in Parts II and III (which used interval scales, allowing us to analyze them with t-Tests or ANOVAs), the nominal scale used in Part V (three answer options in no particular order) only permit us to use a chi square analysis. We’ll discuss those more as we get to Paper Two.
D). Once participants have completed the questionnaire, debrief them regarding the study. That is, tell them about Corey’s different responses to the fake news comments as well as your main hypothesis. Read them the following:
“Thank you for participating. The purpose of this survey is to study participant impressions of “fake news” on Facebook and the comments of social media users discussing fake news. To test this, all participants read the Facebook page of a user named Corey McMillan. Corey shared a news story about the Federal Communication Commission forcing CNN to alter its accreditation license from news to entertainment. Two additional Facebook users commented on this story, noting that it was “fake news”. The main difference between study conditions was how Corey responded to these comments. For one third of our participants, Corey responded in a very rude manner, mocking and insulting the commentators while insisting that the story was not fake. One third of our participants saw a polite response from Corey, who similarly disagreed that the story was fake, but disagreed more respectfully. Finally, one third of our participants also saw a polite response from Corey, but in this final version Corey agreed that the story was fake.
In general, we predicted that participants who read about a social media user who rudely disagreed with feedback that a news story the user shared was “fake” would rate the user less favorably than participants who read about a social media user who either politely disagreed with the feedback or politely agreed with feedback, with user favorability ratings not differing between these latter two polite conditions.
More specifically, we predicted that participants who read about a social media user who rudely disagreed with feedback that a news story the user shared was “fake” would find that user less reasonable, less polite, less open-minded, less likeable, less civil, and less tolerant of opposing views, and more rude and more willing to mock other’s opinions than participants who read about a user who either politely disagreed or politely agreed with the feedback, with no differences emerging between these two polite conditions. Similarly, we predicted that participants who read a rude comment would feel more angry, less hopeful, and be less likely to engage in a conversation with the rude user and less likely to post their own comment on social media than participants who read polite comments, though no differences were expected between conditions in which the polite user agreed or disagreed that the shared story was fake.
We will test these hypotheses in our methods course this semester. Thank you for participating!
**Methods Students: Note that the underlined paragraphs above will be helpful when you write Paper I! In fact, you can use that underlined paragraphs in your first paper if you like (just copy and paste it into your hypotheses). However, the predictions ARE NOT INCLUDED in your minimum page count. That is, you can copy/paste the predictions, but they do not count in the page minimum!
Also note that in the first sentence of the last paragraph, I highlighted eight different dependent variables (reasonable, polite, open-minded, etc.). Since you are not required to analyze every dependent variable in your survey, feel free to edit this sentence to include ONLY the main dependent variable(s) that you plan to analyze (this applies mostly to Paper II when you figure out which DVs you want to focus on in your Results Section analysis). Similarly, the last sentence in the second paragraph discusses four different dependent variables (angry, hopeful, likely to engage in a conversation, and likely to post). Feel free to edit this as well. There is no point in making predictions about dependent variables that you did not actually analyze, so just focus on the two dependent variable most relevant to your own study in your predictions.)
2). Hold onto the completed questionnaires, as you will use them in an upcoming lab. You will enter data into SPSS and analyze it during your lab. Important note: Each student researcher is responsible for collecting data from three participants (one participant for each study condition – PA, PD, and RD). However, we will combine survey data from ALL students in your lab section, so your final sample will include at least 100 to 140 or so participants. In your papers (especially Paper II), you will use this total set of research participants (at least 100), NOT just the three that you collected yourself. Don’t even discuss “Three participants”, as that is not correct. Discuss ALL participants in your papers
3). One last note: Pay close attention to these instructions! You can use them as the basis for Paper II later this semester when you discuss your methods section. That being said, these instructions are too long for a methods section, and includes information you will need to omit for Paper II. When writing that paper, make sure to only report the important aspects (what you actually did in the study). Write about what you actually did in the study!