Writing Materials and Methods
The Methods section describes in detail how the study was conducted…a complete description of the methods used enables the reader to evaluate the appropriateness of your methods and the validity of your results. It also permits experienced investigators to replicate the study. (APA Publication Manual, 6th ed, 29)
The Methods section a research report is essentially a recipe for the study; it succeeds to the extent that the reader feels confident they could replicate the study given the details. Precision and exact details are key to this section, but do not include irrelevant material. This concrete information is usually presented in simple past tense, either active voice (We collected water samples every three days) or passive voice (Samples were collected every three days). The passive is more likely when human or animal subjects are involved. More on this below.
Interestingly, most of the sources on how to write a research report don’t have much to say about the methods section, probably because most scientists find this section the easiest to write. The researcher is most familiar with this activity; after all, s/he experienced it! The Experimental Details are written much how they are performed: materials followed by procedure in chronological order. Precision and accuracy are critical — use an outline to help you keep track of all the information you need to write. The methods section generally includes three types of information:
- subjects / participants / substances
- apparatus / materials / instruments
- procedure/process
Most commonly, some mention of “participants/subjects/substance” (whatever passes for that which was experimented upon) is discussed first, though this is not always the case. For example, if you performed the same procedure on three different groups, you may choose to explain the procedure in detail first, then lay out the composition of the groups. Or, if you have three different procedures on the same type of group, you’d more logically start with the “group” and proceed to procedure. Also, if your work involves multiple tests, then the Methods may be organized topically according to test with parallel organization shared among all.
For example, the outline to A Psychological Exploration of Engagement in Geek Culture shows 7 studies with different methods but over the same population pool (bolded in the excerpt), identified in an opening paragraph labeled “The Present Study”:
 |
The present research aims to: (a) provide preliminary tests of the above hypotheses by exploring the individual differences and social behaviors associated with geek culture engagement and (b) operationalize geek culture by creating measures of geek culture involvement and identity. All studies (except Study 2) used participants from Amazon’s Mechanical Turk (MTurk), which have been shown to give data of similar quality to traditional samples [42,43]. All studies in this paper were approved and monitored by the Institutional Review Board of the University of Georgia (Approval numbers: Study 1 2013106420; Study 2 STUDY00000229; Study 3 STUDY00000203; Study 4 STUDY00000563; Study 5 STUDY00000273; Study 6 STUDY00000413; Study 7 STUDY00000783). Participants gave informed consent by clicking “I consent” on a consent script in all studies except for Study 2; for Study 2, participants gave written consent by signing a consent form. Studies varied between using general samples and self-described geek samples. Details of all samples are reported in Table 1. Where appropriate, the series mean was imputed for all missing data values in this and all remaining studies. For the majority of questions in all studies less than 1% of values were imputed, and the highest percentage was 3% in Study 4. However, the results of Study 4 did not differ whether or not missing values were imputed. |
The Methods section of Socio-Psychological Factors Driving Adult Vaccination: A Qualitative Study has the traditional title, “Method”, then an opening statement situating the protocol as part of a larger study, then narrowing down to how participants were recruited. Note that when a study protocol is published elsewhere, the authors need only cite the original publication. The methods section will specify what is unique to this study.
 |
This research was approved by the Imperial College Research Ethics Committee. Participants were presented with a research information sheet and briefed on confidentiality/anonymity of their data before they were asked to sign a consent form. The overall research protocol and methods used in our research program are reported in detail elsewhere [26]. We summarize them below and include specific information about the UK data collection and analysis.
Sampling and recruitment Interviews were carried out in three regions in the UK: London, South East and West Midlands. Although representative samples are not required in qualitative research, we sought to attain relevant perspectives by recruiting participants from areas where the majority of the UK population reside. We used a purposive sampling strategy to select 20 adult participants who were both vaccinated or not vaccinated against influenza and tetanus, and represented a range of socio-demographic characteristics associated with vaccination uptake, particularly age and health status (see Table 1). We excluded pregnant women and healthcare professionals (HCPs), as their vaccination decisions are significantly influenced by those they care for and/or regulated by healthcare authorities and professional bodies, thus some of their motivations and concerns are likely different [27], [28]. To reduce recall bias [29], only those who had been vaccinated in the past 12 months were eligible as vaccinated participants. A screening question was used to exclude participants who were fundamentally opposed to vaccination, as this stance represents only a small minority of the non-vaccinated population and thus could bias the results [30]. Participants were recruited via telephone using random dialing, sourced from telephone directories by Ipsos MORI, an international market research company. |
Subjects and Participants
Broadly speaking, in current APA nomenclature, both human and animal studies use the term “Subjects” whereas “Participants” is reserved for humans only. Readers need to know who or what was experimented upon, with as much detail as possible (or as much as you collected). This should include description of subjects, recruitment strategies (with human subjects), inclusion/exclusion criteria, environmental accommodations (for animal studies), and a final statement of who the actual subjects turned out to be; that is, the subjects upon whom your results are based. This final statement may be in its own paragraph, and include demographic and statistical data.
While concision is much appreciated in science prose, your working definition of “concise” should include the concepts relevance and accuracy. Say as much as the reader needs to assess whether study subjects are a good fit for the research question and your eventual interpretation of results.
Example: Animal Study — Revealing a Latent Variable: Individual Differences in Affective Response to Repeated Injections
Animal Housing and the HR/LR Phenotype Screening
Animals were treated in accordance with the National Institute of Health guidelines on laboratory animal use and care. All efforts were made to minimize animal suffering and to reduce the number of animals used. A grand total of 42 male Sprague–Dawley rats (Charles River) arrived at postnatal day (PN) 50 and were kept on a 12-hr light/dark cycle (lights on at 7:00 a.m.). Rats were pair-housed throughout the entire experiment and food and water were available ad libitum. Animals were allowed to habituate to the housing conditions for 7 days. On PN 57, all rats were screened for locomotor reactivity to the mild stress of a novel environment for 1 hr using Plexiglas locomotion chambers with stainless steel grid flooring, made in-house. Activity was monitored by means of photocells equally spaced along the sides of the box. At the end of the screening session, total locomotor activity (i.e., rearing and lateral movements) was pooled and the rats were ranked as HRs (i.e., rats that exhibited locomotor scores in the highest third of the sample; n = 14) or LRs (i.e., rats that exhibited locomotor scores in the lowest third of the sample n = 14). The intermediary responders (IRs) were only used as stimulus rats in the social interaction test described below.
Example: Human Study — Sport Fans and Sci-Fi Fanatics: The Social Stigma of Popular Media Fandom
Method
Students from a large Northeastern university were recruited from introductory communication courses to participate in a 2 (Fan type: sci-fi/fantasy fan or sport fan) 2 (Target fan sex: female or male) between-subjects design online experiment, in exchange for extra course credit. Participants were told that the study was on how people form impressions about other people and they were asked to read a descriptive profile of another college student and consider their gut-level impressions of that person. After reading the descriptions, participants proceeded to take an online questionnaire with questions regarding how socially, task, and physically attractive they perceived this person to be.
Participants
A total of 298 students participated in the study, but 23 (7.71%) participants who did not respond correctly to one of two manipulation checks that asked them to identify the type of fan and sex of the fan in the description were excluded from analyses. 3 The final sample consisted of 275 participants. Slightly less than half of the sample was male (46.5%), 42.5% was female, and 10.9% did not report biological sex. 4 The ages of participants ranged from 18 to 55 (M 21.25, SD 3.57). Over three quarters of the sample identified as White (77.1%). African Americans were the next most frequently represented group in the sample (5.8%), followed by Asians (3.6%), and Hispanics/Latino/as (1.8%). In total, 11.7% of participants reported identifying with another racial or ethnic group or did not report their race or ethnicity.
Example: Human Study — Emotion Word Comprehension from 4 to 16 Years Old: A Developmental Survey
Samples
Data was collected from 6 age groups (4–6, 7–8, 9–10, 11–12, 13–14, and 15–16), corresponding with UK school year bands (Years 1–2, 3–4, 5–6, 7–8, and 9–10, respectively). Confirmation of age bands was checked using the Annual Schools Census for the academic year 2004/5, provided by the Schools Statistical Unit at the Department for Education and Skills, UK. Participants were all attending mainstream education in the Cambridgeshire area (UK), and included a mix of both state and private schools, in order to attempt to achieve a representative sample of the community. The total sample was therefore n = 377, and they comprised an equal number of males and females. There were n = 30 age 4–6 years old, n = 34 age 7–8 years old, n = 26 age 9–10 years old, n = 87 age 11–12 years old, n = 134 age 13–14 years old, n = 66 age 15–16 years old.
Analysis: Note that in each example, the participant pool is defined, allowing the reader to assess how well the population fit the research question. Demographic information is provided. Finally, the writers account for the number of participants by indicating both the initial number and final number (“n”), and clarifying how the change in number occurred. This will be true regardless of whether you’re conducting quantitative or qualitative research.
Materials / Instruments / Apparatus
Following subjects/participants, psych researchers report on the materials, apparatus, and instruments used to conduct the research: essentially, all the “tools” needed to complete the work. As elsewhere in the Methods, the key is providing sufficient and relevant detail; readers must have enough information to feel confident they could replicate the study. Many studies do not include a separate “materials” section (see “Procedure” below), but it is common when multiple instruments (e.g. surveys, questionnaires) are used — the reader needs a separate explanation of eachs. Citations are included that point to the research publications that explain or introduce validated instruments.
Example (Quantitative) || A Psychological Exploration of Engagement in Geek Culture
Materials.
To test the nomological network of geek engagement, we included measures we theorized to have relationships with geek engagement along with related traits (e.g., we included all of the Big Five Personality Traits [33], although openness had the most theoretical interest). To this end we included measures of grandiose and vulnerable narcissism and entitlement (predicted by the great fantasy migration hypothesis), and the SDT basic psychological needs (relatedness, predicted by the belongingness hypothesis) [46]. We also included measures of SDT motivational orientation (i.e., how oriented an individual is toward aspects of the environment that encourage autonomy, are controlling, or are under the control of the individual) [47], although we made no specific predictions relating to these measures, and depression and life satisfaction, as these would be negatively related to fulfilled ego or belongingness needs.
To measure geek engagement, we created the Geek Culture Engagement Scale (GCES), by generating a list of 37 geek activities (e.g., cosplay, gaming), interests (e.g., science fiction, fantasy) and lifestyles (e.g., lolita, furry) based on the listing of fan tracks on the Dragon*Con website. We also included the item “your real (daily) life” to explore whether participants who were more involved in geek activities were less involved in daily life. We then asked participants to indicate to what extent they participated in each item on a scale from 1 (a little) to 5 (a lot). See Table 2 for the specific items assessed by the GCES. See Appendix for the full scale.
The Narcissistic Personality Inventory (NPI) [48] is a 40-item nonclinical measure of dimensional narcissism (Samples A and B Cronbach α = .89). For each item, participants choose which of two statements (e.g., “I like to be the center of attention”/ “I prefer to blend in with the crowd”) best describes them. Scores range from 0–40 with higher scores indicating more narcissism.
The Hypersensitive Narcissism Scale (HSNS) [49] is a 10-item scale (Sample A α = .73; Sample B α = .80) designed to measure vulnerable narcissism. Items such as “my feelings are easily hurt by ridicule or the slighting remarks of others” are rated on a Likert-type scale from 1 (not at all like me) to 5 (very much like me). Items range from 0–50 with higher scores indicating more vulnerable narcissism.
The Rosenberg Self-Esteem Scale (RSES) [50] is a widely used 10-item measure (Sample A α = .91; Sample B α = .93) of explicit self-esteem. Items such as “on the whole, I am satisfied with myself” are rated on a scale from 1 to 5 with 1 signifying “this statement does not describe me in the slightest” and 5 signifying “this statement describes me perfectly.”
The Center for Epidemiological Studies Depression Scale (CES-D) [51] is a 20-item self-rating inventory (Sample A α = .93; Sample B α = .94) that is widely used in the measure of nonclinical depression symptoms (e.g., “I felt depressed”). Respondents rated a list of symptoms on a scale from 0 (rarely/none of the time, less than 1 day) to 3 (most or all of the time, 5–7 days) as to how often they have experienced each symptom in the past week.
The Psychological Entitlement Scale (PES) [52] is a 9-item measure of generalized entitlement (Samples A and B α = .89), which is one of the central components of narcissism [53], defined as the belief that one deserves better treatment than others. The PES allows for a more targeted assessment of entitlement than the NPI or HSNS [52]. Participants indicated their agreement with items such as “great things should come to me” on a scale from 1 (strongly disagree) to 7 (strongly agree).
The Five Factor Model Rating Form (FFMRF) [54] is a 30-item measure of the Big Five personality traits. Participants indicated their identification with each individual facet of the Big Five traits, including agreeableness (e.g., “straightforwardness”; Sample A α = .67; Sample B α = .70), extraversion (e.g., “gregariousness”; Sample A α = .73; Sample B α = .77), conscientiousness (e.g., “competence”; Sample A α = .80; Sample B α = .82), neuroticism (e.g., “anxiousness”; Sample A α = .78; Sample B α = .81), and openness to experience (e.g., “fantasy”; Sample A α = .67; Sample B α = .66) on a 5-point Likert scale.
The Diener Satisfaction with Life (SWL) scale [55] is a 5-item scale (Samples A and B α = .92) of subjective well-being. Participants rated items such as “I am satisfied with my life” on a 7-point scale (1 = strongly disagree; 7 = strongly agree).
The General Causality Orientations Scale (GCOS) [56] is a measure of self-determination in personality [57]. It features 17 vignettes describing hypothetical situations. For each vignette, participants rated the likelihood of their pursuing three possible courses of action on a 7-point scale (1 = very unlikely; 7 = very likely). These courses of action represent three dimensions of self-determination, autonomy (Sample A α = .84; Sample B α = .87), controlledness (Sample A α = .71; Sample B α = .73), and impersonal (Sample A α = .85; Sample B α = .84).
The Basic Psychological Needs Scales (BPNS) [46,58] is a collection of scales measuring the basic motivational needs of autonomy (Sample A α = .76; Sample B α = .75), competence (Sample A α = .76; Sample B α = .77), and relatedness (Samples A and B α = .80) in the workplace, in relationships, and in general.
Analysis: This explanation of materials contains 2 critical parts: an explanation of the purpose for the material/instruments (colored blue) and brief descriptions of each instrument that includes the name, accepted abbreviation, citation, what the instrument measures, and an example of question and format (italics in the original).
Example (Qualitative) || Socio-Psychological Factors Driving Adult Vaccination: A Qualitative Study
Piloting
The interview schedule was designed through expert consultations and a review of the relevant literature. The schedule was then tested with two researchers from Imperial College and two researchers from Ipsos MORI who were not involved in the present study. The duration and flow of the interview were discussed and the schedule was finessed as a result. The refinements to the schedule were related to wording (e.g. using ‘flu’ instead of ‘influenza’ for simplicity) and reordering and/or deletion of redundant probes. These interviews were not included in the final sample. Piloting was subsequently carried out for the first three interviews, whereby the research team observed each interview conducted by an Ipsos MORI trained interviewer behind a one-way mirror and evaluated its quality in real-time. At the end of the session, minor improvements were made to the interviewers’ instructions included in the schedule.
Analysis: Qualitative research almost always includes a justification for the method itself in addition to an explanation of how the data was gathered and/or excerpted from the rich stream of information generated by this methodology (see — here — for an additional example). In this case, the testing and refinement of interview questions is spelled out (the authors list the interview questions themselves in a figure in the procedures section).
Note: Err on the side of providing the reader too much details about materials. The gold standard by which Methods are judged is replicability — can someone replicate the study based on the information provided? If they can’t, the reader immediately downgrades the credibility of the work. This is especially true for qualitative methods, which are often considered non-replicable because of the highly local nature of the population. However, when you provide the reader a close description of the protocol, it increases your credibility and provides a foundation for a similar study.
Procedure/Process
If the “materials” section is the ingredient list, the “process” section is the instructions. In this part, researchers write how the experiment was conducted. Each step is laid out chronologically.
The examples below include an animal study, a quantitative study and a qualitative study. For added spice, main clause verbs written in the active voice are marked in orange; those in the passive are blue. We’ll discuss this below.
Example: Animal Study — Revealing a Latent Variable: Individual Differences in Affective Response to Repeated Injections
Chronic Saline Injections
Following phenotype screening, cages were reorganized in a way that each rat was housed with a cage mate of the same phenotype. Each cage was assigned to injection or control handling groups (n = 6–8). Five days after phenotype-screening, animals in the chronic injections groups received ip saline injections (1 ml/kg) once a day for 2 weeks. Control handled rats underwent the identical handling procedure as the saline injected animals daily, for 2 weeks but did not receive the injections. The day after the final saline injection or handling session all rats underwent pretesting for the FST. One day after the pretest session, rats were tested on the social interaction and forced swim tests as described below.
Social Interaction Test (SIT)
Testing took place in an open topped, rectangular, transparent social interaction box. The stimulus rat was a conspecific of similar age and weight and had no previous contact with the experimental rat. Rats were placed simultaneously in the box and the amount of time the experimental rat spent initiating social interaction (i.e., grooming, sniffing, following, and crawling over or under) with the stimulus rat was scored for 5 min.
Forced Swim Test (FST)
Testing was performed according to a published protocol (Porsolt et al., 1977) with some modifications. A 15-min pretest was conducted the day before testing. On the day of testing, rats were placed in a cylinder filled with water at room temperature. Each rat was tested individually and the cylinder was cleaned and filled with fresh water following each animal. Time rats spent immobile (lack of movement except necessary movements to keep the head above water), as well as swimming and climbing were scored for 5 min.
Statistical Analysis
Locomotor reactivity in response to a novel environment was analyzed by a one-way ANOVA. Data pertaining to SIT and FST were analyzed by two-way ANOVAs: phenotype (LR, HR) and exposure (INJ, HANDL). Significant main effects and interactions were followed by post hoc comparisons using the Bonferroni-Dunn corrections.
Analysis: In this study, different tests were performed — each test gets its own subheading and descriptions. Note that “materials” are included along with process. You’ll also see the statistical tests are named.
Example (Quantitative) || A Psychological Exploration of Engagement in Geek Culture
Procedure. (Study 1)
Samples A (N = 350) and B (N = 317) completed the study online. For Sample A, the Amazon MTurk posting was worded to attract people who are engaged in geek culture and to discourage people without interests in geek culture activities from participating (exact wording for all studies is posted online at https://osf.io/u25x9/). For Sample 2, The MTurk posting was worded as generically as possible so as to recruit participants with a variety of geek engagement levels. Participants indicated their consent by clicking “I consent” on a consent script and completed the measures via an online survey hosting website before being compensated via MTurk. Thirty participants were found to have already participated in Sample A and were removed from Sample B.
Procedure. (Study 2)
Participants (N = 202) were approached in downtown Atlanta, GA during the Dragon*Con geek convention. The researchers targeted persons wearing badges indicating they were attendees of the convention. Participants were informed as to the purpose of the study, gave consent by signing a consent form, and completed two pages of brief surveys. Then, with the participants’ consent, their picture was taken using a digital camera. Participants were given no specific instructions as to how to pose or whether to smile. They received no compensation for their participation.
Analysis: Recall there were seven studies in this report (the labels above are not in the original). Each Procedure section lays out the process’s steps in chronological order. Note that actions are written in the past tense; descriptions may be written in the present, e.g. “who are engaged in geek culture”.
Example (Qualitative) || Socio-Psychological Factors Driving Adult Vaccination: A Qualitative Study
Procedure
Participants were interviewed face-to-face in their homes or at a central interviewing facility for approximately 60 minutes. Each participant received £70 in return for their time. Half of the interviews were conducted by an academic researcher (AW) from Imperial College and half by trained interviewer from Ipsos MORI. Two versions of the interview schedule were used: one for vaccinated and one for non-vaccinated participants (see Table 2).
The interview schedule comprised six sections. Section1 obtained an overview of participants’ life and values, to build rapport with the interviewee and to identify important issues to assist with probing throughout the interview. Section 2 elicited participants’ general information-seeking behaviors and influences. Section 3 examined participants’ views toward health, HCPs and adult vaccines. Section 4 evaluated individual participants’ decision making ‘journeys’ to vaccination or non-vaccination. Section 5 examined participants’ attitudes toward pediatric influenza and tetanus vaccines. We aimed to understand whether or not people’s views about adult vaccines correspond with their views about childhood vaccines. Finally, in section 6 we explored participants’ knowledge of the two diseases and vaccines (i.e. influenza and tetanus) to understand to what extent their decision-making is influenced by factual information.
We explored the set of circumstances and emotional factors that drove participants to accept or refuse vaccination, aided by qualitative interviewing approaches aimed at obtaining information which explicit enquiry (i.e. a direct question) may fail to capture – as follows. First, throughout the interview we used an elicitation technique called ‘laddering’, which provides a simple and systematic way of establishing people’s core values and beliefs, and the linkages between these and key behaviors, in this case, vaccination [31]. In section 3, general views on adult vaccines were evaluated by asking participants to spontaneously arrange and rank, using a category of their choice, cards depicting five adult vaccines (influenza, tetanus, pneumonia, hepatitis and measles, mumps and rubella (MMR)) into one or more groups. In section 4, we employed the “Journey to vaccination” approach [26], a visual exercise in which the interviewer and the participant jointly draw a timeline that captures salient events that lead the participant to get or not to get vaccinated. These results will be presented elsewhere, as they require different analysis. Briefly, a journey to vaccination for influenza and other for tetanus is drawn for each participant. Differences and commonalities emerging from these data are identified and synthesized, and typical journeys proposed.
Analysis: This Procedure section includes details about the interview questions as well as the process by which interviews were conducted (paragraph 1). The interview questions are listed in “Table 2” while the middle paragraph supplies the core mission of each question. The third paragraph explains other interviewing techniques used that supplemented the direct questions — this is particularly valuable for the reader because the varied elicitation strategies compliment the directed interview questions, enriching the data considerably. The paragraph is a mini-tutorial in advanced qualitative data gathering.
Style Note: Stop worrying about passive vs active voice. Passive does not signal evil intent. . For a brief explanation, see below. If you’re feeling really curious, head the the Stylistics page.
Grammatically speaking, sentences in English must include nouns and verbs arranged so that nouns are (grammatical) subjects and objects while verbs convey the action of the sentence. Think of “grammar” as a layer of structure that joins the nearly infinite number meanings you carry around in your brain with rules & patterns to manage mutual comprehension; in other words, grammar forces us to make choices about how we say (or write) something so that someone else understands us. While the meanings in your head are infinite, grammar is mostly certainly not. Grammar makes us choose how to divvy up the things we see, experience, feel, and think into categories, and in English, the big three are S, V, and O — Subject, Verb, Object — often called the “parts of speech”. Each part of speech has 3 roles to play simultaneously.
- Role 1 — English mandates word order, making it a Subject – Verb – Object (SVO) language. A sentence must have the correct SVO relationships to be considered grammatical. Word order has the greatest impact on mutual comprehensibility.
- Role 2 — Semantically speaking, the nouns in the sentence (S & O) are either “doing” or “receiving” the activity expressed in the verb. This is easy to see in simple sentences such as “The dog chased the cat” — the dog is doing the chasing (is the agent or actor), and the cat is receiving the activity (the receptor or undergoer; truth be told, English has no antonym of “agent” which is not also derogatory, so there is no good semantic term for the thing on the receiving end of an activity. I’ll use “receptor” because it has fewer syllables and is fairly neutral).
- Role 3 — Grammar must provide a way to change the grammatical “topic” — which is literally who or what the sentence is about. Grammatically, we call this “voice”. Sometimes we want to make the agent the star of the sentence, and that’s what plain S-V-O in the active voice. But nouns can be switched up so that objects can play the starring role: “The cat was chased by the dog”. This is the so-called “passive voice”. The passive is a grammatical transformation that moves the noun on the receiving end of an action to the subject position so that the speaker/writer can make the sentence about that noun. Contrary to popular belief, the motivation for using the passive is NOT to hide who is doing the action (though passive can be employed for this reason). Passive changes discourse topics and hence discourse is changed by they passive option.
- Page 2 — Introduction
- Page 3 — Methods
- Page 4 — Results
- Page 5 — Discussion
- Page 6 — Action Plan and Outlines