Between and Within Groups Research Designs

Between and Within Groups Research Designs

Chapter Learning Outcomes

After reading and studying this chapter, students should be able to:

• understand and articulate the four basic building blocks of research designs (pretest-posttest or posttest only; one independent variable or more; between, within, or mixed design; and randomization method).

• describe the advantages and disadvantages of a between groups design, recognizing the importance of main effects and interactions.

• appreciate the special challenges and approaches used in between groups designs, such as demand characteristics and single-blind and double-blind experiments.

• recognize the characteristics of within groups research designs (split plot, repeated measures) and understand the principles applied in using within groups designs.

• comprehend the special considerations of within group design usage (ceiling and floor effects, carryover and order effects) and appreciate the limitations on any experimental design.

Digital Vision/Thinkstock

lan66845_03_c03_p063-100.indd 63 4/20/12 2:47 PM

64

CHAPTER 3Introduction

Introduction

In research, the type of design that you choose influences the type of conclusion that can be drawn from the research. It is important to evaluate the needs and circum-stances of your research project before selecting a design. We’ll begin this chapter with a basic overview of the major types of research designs, and then we’ll focus the rest of this chapter on between and within groups designs.

Voices from the Workplace

Your name: Shelley D.

Your age: 46

Your gender: Female

Your primary job title: Residential Care Facility Administrator

Your current employer: Hope Haven Area Development Center Corporation

How long have you been employed in your present position?

3 months

What year did you graduate with your bachelor’s degree in psychology?

1983

Describe your major job duties and responsibilities.

I oversee a 15 bed residential care facility. This includes supervising a staff of 14, being responsible for the budget of the facility, making sure all policies and procedures are followed according to local, state, and national guidelines.

What elements of your undergraduate training in psychology do you use in your work?

Frequently look at diagnosis and medications. Have to be aware of what medications are used for what mental illnesses, what symptoms of various mental illnesses are and how to interpret IQ testing.

What do you like most about your job?

I enjoy working with the residents and assisting them in obtaining various skills in order to reach their potential.

What do you like least about your job?

The constantly changing regulations.

Beyond your bachelor’s degree, what additional education and/or specialized training have you received?

I have ongoing training through our organization. This includes CPR and first aid, Adult Abuse report- ing, various leadership and supervisory training, Employment Specialist training.

What is the compensation package for an entry-level position in your occupation?

We offer health and dental insurance as well as paid life insurance. We have a retirement program in which Hope Haven will provide matching funds up to 3% of your annual salary. We also offer paid sick, vacation and casual days.

What benefits (e.g., health insurance, pension, etc.) are typically available for someone in your profession?

These are generally the common and standard benefits available in this area to entry level positions in this field. (continued)

lan66845_03_c03_p063-100.indd 64 4/20/12 2:47 PM

65

CHAPTER 3Section 3.1 The Basic Components of Research Designs

What are the key skills necessary for you to succeed in your career?

Being good with people, good communication skills both verbal and written, ability to use good judg- ment, be a self starter, good organization skills, good typing skills and flexibility.

Thinking back to your undergraduate career, what courses would you recommend that you believe are key to success in your type of career?

Abnormal psychology and some type of pharmaceutical course. Due to the wide variety of people that I interact with, I think a variety of courses are important including child development courses.

Thinking back to your undergraduate career, can you think of outside of class activities (e.g., research assistantships, internships, Psi Chi, etc.) that were key to success in your type of career?

We had to complete a variety of research projects that students had to be involved in. I think those were especially helpful. Any internship in a facility would be beneficial.

What advice would you give to someone who was thinking about entering the field you are in?

You are not going to become independently wealthy, however, the intrinsic value of the job and know- ing that you are helping to better someone’s life is a great responsibility and reward.

If you were choosing a career and occupation all over again, what (if anything) would you do differently?

Probably go on to get a master’s degree in counseling.

Copyright . 2009 by the American Psychological Association. Reproduced with permission. The official citation that should be used in referencing this material is R. Eric Landrum, Finding Jobs With a Psychol- ogy Bachelor’s Degree: Expert Advice for Launching Your Career, American Psychological Association, 2009. The use of this information does not imply endorsement by the publisher. No further reproduction or distribution is permitted without written permission from the American Psychological Association.

3.1 The Basic Components of Research Designs

When thinking about research designs, there are fundamental components or building blocks that need to be considered. The first of these is to consider whether the dependent variable is measured before and after (pretest-posttest) the introduc- tion of the independent vari- able, or just after (posttest only). But first, let’s briefly review independent and dependent variables.

The independent variable is the variable that is manipulated, controlled, or organized by the

Voices from the Workplace (continued)

The independent variable of an experiment is the variable that the researcher can control and change.

Glow Wellness/SuperStock

lan66845_03_c03_p063-100.indd 65 4/20/12 3:33 PM

66

CHAPTER 3Section 3.1 The Basic Components of Research Designs

researcher. For example, in studying the behavioral effects of caffeine in college students, a researcher may desire to control or manipulate the consumption of caffeine during the experiment. Different students receive different amounts of caffeine, measured in milli- grams (mg). Here caffeine consumption is controlled by the experimenter, defined as the number of milligrams consumed by the student. In fact, the researcher could have various levels of caffeine consumption, such as 50 mg, 100 mg, 200 mg, and 400 mg. This type of independent variable is called a non-subject variable because the actual value of the inde- pendent variable (in this case, the number of milligrams of caffeine received) is not deter- mined by the person receiving the caffeine, but by the researcher. Sometimes the value or level of the independent variable is determined by the individual participant, and this is known as a subject variable. Subject variables can only be arranged or organized by the researcher—they cannot be controlled or manipulated. For example, a person’s level of extroversion is not manipulated by the experimenter; however, it may be measured and that person assigned to a specific group (high, medium, or low extroversion). Vari- ables such as gender, personality traits, natural hair color, and race are subject variables: a characteristic each person possesses that can only be organized into different groups in a study (neither controlled nor manipulated).

Just as there are different types of independent variables, there are different types of dependent variables. Remember that the dependent variable is the one that is measured— hopefully the direct result of the manipulations of the independent variable. For example, dependent variables can be either qualitative or quantitative. A qualitative variable is one in which the responses differ in kind or type. That is, there is a difference in quality (what form) rather than quantity (how many), and the outcomes of these qualitative variables are usually described in words. On a survey, if you asked someone to write a few sen- tences telling you about his or her experience today at the mall, this would be qualitative data. Quantitative variables differ in amount; there is more or less of some known entity. Quantitative variables are usually described by numbers, and psychologists tend to strive to develop measures of behaviors (dependent variables) that yield a number. On a survey, if you asked someone to answer multiple questions about his or her experience at the mall where 0 = terrible experience and 10 = best experience ever, this would be quantitative data. Remember that an appropriate dependent variable is the result of careful, systematic observation that is translated into a clear measure of behavior.

Pretest-Posttest or Posttest Only

For one of your courses you are currently taking, there may be a cumulative final exam. This is one way of thinking of a posttest only scenario. Researchers often like to use X’s and O’s to describe research designs. The X in a research design stands for some sort of intervention or independent variable manipulation—X marks the situation where some- thing is happening.

The O in a research design stands for an observation or a measurement—that is the depen- dent variable. For this example, the X would be the course (Xcourse) and the O would be the cumulative final exam (Ofinal exam). We could put them in a linear sequence, as below, and this is an example of a posttest only design reading from left to right.

Xcourse Ofinal exam

lan66845_03_c03_p063-100.indd 66 4/20/12 2:47 PM

67

CHAPTER 3Section 3.1 The Basic Components of Research Designs

Notice above that there was no pretest at the beginning of the course—if there had been, that would be a pretest-posttest design, and it would look like O X O. In fact, it could be interesting to give the course final exam on the first day of class, and again on the last day of class. A pretest-posttest design looks like this:

Ofinal exam Xcourse Ofinal exam

Not to get too far ahead, but we could add a control group to this design—give the pretest and posttest to a different group of students not enrolled in the course (a control group). That design would look like this.

Ofinal exam Xcourse Ofinal exam

Ofinal exam Ofinal exam

We’ll come back to the advantages and disadvantages of pretest-posttest and posttest only designs at the end of this chapter. For more on these designs, see Meltzoff (1998).

One Independent Variable or a Factorial Design

A second basic component of knowing about research designs is to know the number of independent variables being manipulated, controlled, or arranged. One independent variable is simply referred to as one independent variable, but more than one indepen- dent variable is called a factorial design. Factorial designs have some distinct advantages, namely, the ability to understand interactions between multiple independent variables.

Between or Within Groups Design

Another major component of the basic building blocks of experimental design is whether the research design is a between groups design, a within groups design, or a mixture of both—in that case, a mixed design. Briefly, the between groups design is intended to mea- sure differences between separate groups of participants in a study. For example, if your col- lege or university offers a course to help students prepare for the GRE (a test often required for admittance to many types of graduate programs), and you were interested in whether freshmen, sophomores, juniors, or seniors would benefit most from the GRE course, this would be a between groups design. Four different, separate groups of individuals (fresh- men, sophomores, juniors, and seniors) were utilized to see if the GRE course was success- ful in helping students improve their GRE scores. In this case, the focus is on the difference between groups. Of course, we could have more than one between groups independent variable. We could add gender as a variable, making this a 4 (year in school) × 2 (gender) between groups design.

The most common example of a within groups design would be when we are looking for a change within a participant over time (there are more complicated versions of the within groups design, and we’ll save those for later in this chapter). The pretest-posttest design (without the control group) is a good example. If you were to take a “cumulative” final exam at the beginning of the course, take the course, and then take the cumulative final exam again, this is a within groups design. The goal of that design is to see if you changed over

lan66845_03_c03_p063-100.indd 67 4/20/12 2:47 PM

68

CHAPTER 3Section 3.1 The Basic Components of Research Designs

time (that is, if taking the course led to increased scores). Of course, there are all kinds of reasons why the scores could have changed over time, and we’ll address design issues later that help us make meaningful conclusions from data. The goal of a within groups design is typically to examine how a person may change over time, whereas the goal of a between groups design is typically to examine how groups of people may differ from one another.

Often the research question you are interested in dictates the type of design used. For example, if you wanted to test whether right-handed individuals have more legible hand- writing than left-handed individuals, this research design dictates a between groups design (MacKenzie, 2008) because it will take two different groups of people to make the comparison. But let’s say you wanted to know if the more you practice a new skill, the better your skill level becomes. In this case, you are looking for a change in a person over time, such as using a typing program to gain more proficiency at keyboarding skills. To detect skill development over time, within groups designs are used. But if you wanted to look at skill development over time (within groups) depending on three types of typing training programs (between groups), you could include both between groups and within groups design features into your research. This is called a mixed design.

Randomization, Matching, or Blocking

A final key component of experimental designs concerns how participants are assigned to certain conditions or variations of the experiment. Typically, randomization is the stron- gest or most powerful approach to assigning participants to experimental conditions. Take the case of the pretest-posttest design that includes a control group (shown again here as a reminder):

Ofinal exam Xcourse Ofinal exam (Experimental Group)

Ofinal exam Ofinal exam (Control Group)

The issue of random assign- ment comes into play when we have to determine which group of students composes the experi- mental group and which group of students composes the con- trol group. That process of how students are assigned to groups is very important because if true random assignment can be used, that helps to strengthen the con- clusions drawn from our data.

However, this is also a good example of a case where ran- dom assignment is not typi- cally used. As researchers, we typically don’t have the power to randomly assign students to

Assigning subjects to experiment condition randomly is the best approach. You can use dice or numbers to help you randomly assign subjects to your experiment.

age fotostock/SuperStock

lan66845_03_c03_p063-100.indd 68 4/20/12 2:47 PM

69

CHAPTER 3Section 3.2 Between Groups Designs

course sections—students usually pick their own courses. Thus, in cases where random assignment may not be possible (Gribbons & Herman, 1997), we turn to other methods, such as matching in between groups designs and blocking in within groups designs. There are also situations where random assignment may not be an ethical choice, such as testing the effectiveness of a child welfare program and including a control group where benefits are withheld (Bawden & Sonenstein, n.d.). In an experimental design, knowing how participants are placed into the conditions of the independent variable is key—typi- cally, the strongest option is to use random assignment. Even if complete randomization is not possible, we will always want to know how participants were assigned to groups or conditions.

That concludes our overview of the key components of experimental designs. Now, let’s turn our attention exclusively to between groups designs, how to use them, and what we can learn with effective implementation.

3.2 Between Groups Designs

In a between groups design, the researcher is interested in comparing or detecting differences between the groups. This could be that the researcher expects males and females to behave differently, or Republicans, Democrats, and Independents to vote differently, that psychology and nursing majors have different expectations and career paths, and so on. There is no need to make this more complicated than it has to be; the goal of a between groups design is to detect if there is a significant difference between the groups being tested. To be an independent variable, the values (or levels) of that vari- able must be manipulated, controlled, or arranged (such as in subject variables). To be a between groups independent variable, there must be at least two groups, although there could be more than that. Participants can be “claimed” in only one level of the variable for it to be between groups. Using our examples from above, between groups is appropri- ate if you can only be placed in the male or female group; the Republican, Democrat, or Independent; or the psychology major or nursing major group. If you are a double major in psychology and nursing, then a between groups design would not be the appropriate design for testing.

In considering a between groups factorial design, there are two types of effects that we look for: main effects and interactions. A main effect gives us information about the overall effect of each of the independent variables, whereas an interaction effect allows us to look at the combinations of the levels of the independent variables to examine if these combinations lead to different outcomes compared to other possible combina- tions. Let’s say you were doing some research on the optimum conditions for students to learn. For this example, learning is defined as a score on a test. In your research design, one between groups independent variable is how tests are administered; a test is either administered online or in a traditional classroom setting. A second independent variable in this research is how the course is delivered—either online or through a classroom set- ting. Both of these variables are between groups variables, and you will need four dif- ferent groups of participants to complete this research as envisioned. Below is how this design would look, graphically:

lan66845_03_c03_p063-100.indd 69 4/20/12 2:47 PM

70

CHAPTER 3Section 3.2 Between Groups Designs

This is known as a completely crossed design, because four conditions are being tested— two levels of test administration times two levels of course delivery. If you think about it, three of these four conditions make sense. If you were taking an online course, you could see yourself also taking your tests online. It’s not that unusual for a course taught in a classroom to sometimes use online testing, as a matter of efficiency. It would also be fairly typical for some courses that are taught in the classroom to be testing in the classroom. The last combination would be a bit unexpected; taking a course online, but being tested in a classroom. Perhaps this could happen if the instructor felt that there was a higher than normal chance of cheating that would occur if the test were given online. In any case, this is a 2 × 2 between groups design, and the way this is laid out, this design will require four separate groups of students to assess the effects of the independent variables (test admin- istration, course delivery) on the dependent variable (student test scores).

A main effect examines the overall effect of one independent variable. Using the above example, will there be a main effect of test administration? That is, is there a significant difference between scores for students who are tested online versus those tested in a class- room? Essentially, this main effect looks to see if the scores in the rows in the table are different from one another. The second main effect is to look at the effect of the method of course delivery, to see if teaching the course online versus in the classroom leads to a difference in test scores. In this case, the main effect examines if the scores in the columns are different.

However, what is more fascinating about this design is whether there is an interaction effect that is statistically significant. In other words, is there a combination of rows and columns (that is, a particular cell) that stands out and leads to superior student perfor- mance on tests? You can see how this information would be valuable. If the best combi- nation of student learning occurs when online instruction is followed by online testing, that would be vital information for educators to have. However, if no one combination is better than any other, that would be important too, because we would know that we were not depriving students of a learning experience that was more beneficial than another. We could have a situation where just the test administration main effect was significant, or just the course delivery main effect was significant, or different combinations, including a significant interaction. The following graphs depict these different types of outcomes.

Course Delivery

Test Administration

Online

Online

Classroom

Classroom

This is an example of a 2 × 2 experimental design, where the rows depict the independent variable of test administration (2 levels; online and classroom) and the columns depict another independent variable, course delivery (2 levels; online and classroom).

Figure 3.1: A 2 × 2 design

lan66845_03_c03_p063-100.indd 70 4/20/12 2:47 PM

71

CHAPTER 3Section 3.2 Between Groups Designs

Figure 3.2 shows an example a test administration main effect. Regardless of how the course is delivered, classroom testing is superior, leading to higher test scores. In this case, there is not a significant interaction; a significant interaction would be evidenced by a dif- ferent pattern of bars.

Classroom DeliveryOnline Delivery

S co

re o

n T

es t

Online TestingClassroom Testing

This graph depicts a main effect for the testing administration variable. That is, regardless of the mode of class delivery, student scores in classroom testing are higher than student scores using online testing.

Figure 3.2: One type of main effect

Classroom DeliveryOnline Delivery

S co

re o

n T

es t

Online TestingClassroom Testing

This is a depiction of a main effect of course administration. Regardless of whether the test was administered online or in a classroom, those students receiving classroom course delivery scored higher on the test than those receiving the course via online delivery.

Figure 3.3: Another type of main effect

lan66845_03_c03_p063-100.indd 71 4/20/12 2:47 PM

72

CHAPTER 3Section 3.2 Between Groups Designs

Figure 3.3 is an example of data where there is a main effect of course delivery, but no interaction. Scores on the test are higher no matter what type of testing is given, so long as the course is being delivered in a classroom. Although there are numerous examples of what an interaction might look like, Figure 3.4 shows an example of what the data would look like with an interaction taking place.

There is not a simple answer to the question “who scores best” limited to the effects of only one independent variable. Here is the place where an interaction is the most meaningfully interpreted. There is one combination of test administration and course delivery that leads to the best combination of test scores—as you can see from the graph in Figure 3.4, that best combination is when the course is taught in the classroom but the test is adminis- tered online. This information would be highly valuable to educators and students alike. Given that test scores are measured on an interval/ratio scale, an appropriate statistical analysis would be a two-way ANOVA (two-way meaning two independent variables). The ANOVA is a statistical procedure that allows for the detection of differences when there are three or more levels of an independent variable, or two or more independent variables (with any number of levels). The interaction effect would turn out to be statis- tically significant (or not). There would be further analyses of interest. The significant interaction tells us that there is a difference between the four groups—it does not desig- nate the exact nature (or location) of the differences. For those data, post hoc analyses (or follow-up analyses) are needed. With interactions, this analysis is called simple effects, or simple main effects (Newsom, n.d.). Using a test of simple effects, we could iden- tify how the online testing/classroom delivery condition is different from the other three combinations of the independent variables. If we suspect ahead of time (i.e., a priori) that certain conditions may be different (such as online testing/online delivery versus online

Classroom DeliveryOnline Delivery

S co

re o

n T

es t

Online TestingClassroom Testing

This is an example of a classic interaction effect. Rather than test scores being higher due to type of testing (a main effect) or type of course administration (a main effect), here is an example of how two independent variables can interact.

Figure 3.4: An interaction effect

lan66845_03_c03_p063-100.indd 72 4/20/12 2:47 PM

73

CHAPTER 3Section 3.2 Between Groups Designs

testing/classroom delivery), a planned comparison can be conducted without the need of the two-way ANOVA (Newsom, n.d.). Sometimes the answer to complex questions is “it depends,” and interactions allow us to figure out what is happening to our dependent variable when multiple independent variables may be influencing one another.

Classic Studies in Psychology: Cognitive Dissonance (Festinger & Carlsmith, 1959)

In 1957 Leon Festinger published an influential theory in social psychology called cognitive dissonance theory. As Festinger and Carlsmith (1959) originally characterized the theory, when a person privately holds an opinion but is pressured publicly to argue against the privately held opinion, a form of dis- comfort or dissonance will occur because of the conflict. Festinger thought of these two cognitions as not fitting together psychologically. As this theory was further studied and refined, cognitive disso- nance was also thought of as a situation where a person’s attitudes and behaviors are in conflict, and the amount of dissonance would become a predictor in the degree of motivation to resolve the disso- nance, either by changing the attitudes or changing the behaviors (Aronson, 1992). What is so interest- ing about cognitive dissonance theory is that it makes specific predictions about changes in attitudes and behaviors, and sometimes counterintuitive results occur in determining what changes attitudes (Festinger & Carlsmith, 1959).

Cognitive dissonance theory provided a wealth of opportunities for future research. In the Festinger and Carlsmith (1959) study, the actual participants were students who were asked to lie to other students in a study who were about to perform a series of truly boring tasks. The participants were assigned to one of three conditions. In the control/baseline condition, the participant wasn’t asked to lie about the upcoming task to a participant waiting to complete the study. In the “one dollar” condi- tion, the actual participant was paid $1 to lie to the waiting participant, and tell him or her that the upcoming tasks were interesting, enjoyable, and fun. In the “twenty dollar” condition, the participant told the same lie as in the $1 condition, but was paid $20 to lie. In the $1 and $20 conditions, disso- nance was present—the participants knew that the tasks were dull and boring but lied about it. The participants were asked a number of questions about the study, and their responses are the depen- dent variables that Festinger and Carlsmith were most interested in.

A key dependent variable question for Festinger and Carlsmith (1959) was “how enjoyable tasks were,” and the true participants (control, $1, $20) answered this on a −5 to +5 scale. The control partici- pants average was −0.45; the average for the $1 condition was +1.35; and the average for the $20 condition was −0.05. Control participants were not asked to lie to waiting participants, and when asked, they were slightly negative toward the upcoming dull and boring tasks (M = −0.45). However, the remaining two experimental conditions did lie to the waiting participants, saying that the upcom- ing tasks were interesting, enjoyable, and fun when, in fact, those tasks were indeed dull and boring. According the Festinger and Carlsmith, the participants in the $1 condition felt the most dissonance, hence they changed their own perception of the experiment (M = +1.35) to match the lie they were telling. In the $20 condition (M = −0.05), there was lesser dissonance as compared to the $1 condi- tion (M = +1.35). Why this pattern? To tell a lie for such a small amount of money, the participants had to change their own perceptions (cognitive dissonance theory). But to tell a lie for a much larger amount, participants remember it’s just a lie and take the $20 without changing their own attitudes. “The greater the reward offered (beyond what is necessary to elicit the behavior), the smaller was the effect” (Festinger & Carlsmith, 1959, p. 208). To put it another way (Aronson, 1992), “people believe lies they tell only if they are under-rewarded for telling them” (p. 304).

As Aronson (1992) and others have written about, the theory of cognitive dissonance may be one of the most importance contributions of social psychology (see also Jones, 1976), and it has inspired thousands of studies. Cognitive dissonance theory was a welcome relief for many psychologists who were influenced by behaviorism at the time and the prevalence of reinforcement theory as a simple explanation for human behavior. Cognitive dissonance allowed for the study of human (continued)

lan66845_03_c03_p063-100.indd 73 4/20/12 2:47 PM

74

CHAPTER 3Section 3.3 Participant Selection Challenges in Between Groups Designs

3.3 Participant Selection Challenges in Between Groups Designs

Ideally, we would select a random sample from our population of interest to be used in our research. In a truly random sample, every member of the population has an equal chance of being selected. It is not often in experimental research that we have a compre- hensive and accurate listing of every member of a population. Think about doing research in the town where you live. The roster of residents probably changes often, perhaps even daily, with people moving in, moving out, going on vacation, and so on. So researchers are often left with a nonrandom selection strategy—examples include availability sam- pling (sometimes called convenience sampling), quota sampling, and snowball sampling (sometimes called chair referral sampling) (Festinger & DeMatteo, 2008). Availability sampling means that those who are selected to participate were conveniently available to

behavior in a new light, in addition to the reinforcement approach, as Aronson (1992) restated Festinger’s impact in this context: “If a person held two cognitions that were psychological inconsistent, he or she would experience dissonance and would attempt to reduce dissonance much as one would attempt to reduce hunger, thirst, or any drive” (p. 304). Another important contribution of Festinger’s theory and work was to offer alternative methods for changing behaviors. Prior to this research, it was generally considered that if you wanted to change behaviors, you needed to first change a person’s attitudes—that is, our attitudes drive our behaviors (Aronson, 1992). Cognitive dissonance theory predicts that when attitudes and behaviors are in enough dissonance, behaviors may indeed change to match attitudes, but attitudes can also change to match behaviors. Aronson (1992) pointed to the convincing example of desegregation of schools in the South in the 1950s. Some psychologists sug- gested that attitudes needed to change first before changing the behavior (desegregating the schools), but cognitive dissonance theory allowed for the prediction that if you change the behaviors (integrate schools), that event can set in motion a change in attitudes, which in fact did occur (Aronson, 1992). Cognitive dissonance theory is still powerful today and has been used to analyze citizen responses to the events of September 11, 2001 (Masters, 2005). When attitudes and behaviors conflict (or simulta- neously held cognitions conflict), we are motivated to resolve the dissonance.

Critical Thinking Questions

1. Can you think of situations in your own life where an attitude you publicly held was not in sync with your private behavior? According to Festinger and Carlsmith (1959), one of those two condi- tions must be resolved for the dissonance to fade. In your personal situation, which won out—did you change your attitude or your behavior?

2. Think about how cognitive dissonance might be purposely used to help attitude or behavior change. Would it be useful to point out to individuals how their attitudes and behaviors are not in sync? Thinking about your knowledge of psychology from this and other courses, what principles and theo- ries would be useful to apply to achieve an intended attitude change? An intended behavior change?

3. Think about how the idea of cognitive dissonance applies to major problems that society faces. There is a heightened awareness about global warming and environmental concerns, but look around your local parking lot and check out the types of cars being driven. Is public transportation well utilized where you live? We know about the negative effects of poverty and homelessness, but think about the efforts in your community (e.g., fund-raising, shelters). We hold certain atti- tudes and we possess knowledge, but what facilitates behavior change? Why do so many people see the problems and fail to act? How might cognitive dissonance theory explain (a) a level of relative inaction, and (b) how dissonance might be leveraged for society-level changes?

Classic Studies in Psychology: Cognitive Dissonance (continued)

lan66845_03_c03_p063-100.indd 74 4/20/12 2:47 PM

75

CHAPTER 3Section 3.3 Participant Selection Challenges in Between Groups Designs

the research, such as in a subject pool for a college or university, or an online post with a URL widely distributed, and so on. Quota sampling refers to when a particular makeup of participants in the sample is desired—typically to match the makeup of the population. So if you were doing political research in a county where there the registered voters are 41% Republican, 43% Democrat, and 16% Independent, in quota sampling you would want those same percentages in your sample to match the population. Snowball sampling is an informal procedure where the researcher makes an initial round of contacts to solicit participants for a study but then invites those contacts to invite others—so you might invite someone to participate in a study via Facebook, but then ask that person to invite his or her Facebook friends to partici- pate as well.

In a between groups design, the overarching goal is to obtain roughly equivalent groups prior to the introduction of the indepen- dent variable manipulation. I use the phrase “roughly equivalent groups” here because although we’d prefer exactly equivalent groups, that is unlikely with chance operating and consider- ing the sheer nature of two sepa- rate groups of people being con- sidered for any type of study.

Special Situations in Between Groups Designs

You determined that to answer your research questions of interest (that is, to address your working hypotheses), you settled on a between groups design. With any type of design, some things should be considered when making a design decision. What follows are con- siderations with regard to between groups designs.

First, demand characteristics are considered from the viewpoint of the participant. That is, sometimes experimental participants try to “figure out” the nature of the research and “help” the researcher by giving into his or her “demands” (of course, the research doesn’t demand anything, but oftentimes participants are eager to please, and researchers are eager to be pleased). To avoid demand characteristics (if possible), sometimes a research study can be designed such that the participants do not know what you (the researcher) are looking for. Recent research demonstrated that demand characteristics can influence the behavior of experimental participants (Nichols & Maner, 2008). Participants may know there is a treatment group and a control group, but the participant is unable to figure out which one he or she is in. This is called a single-blind experiment.

In a single-blind experiment,participants are unaware of the experimental condition they are in. Take, for example, a study that involves weight loss as a dependent variable

In a true random sample, every member of the population would have an equal chance of being selected. Is this always practical when selecting participants?

Stockphoto/Thinkstock

lan66845_03_c03_p063-100.indd 75 4/20/12 2:47 PM

76

CHAPTER 3Section 3.3 Participant Selection Challenges in Between Groups Designs

measure. A researcher has designed a new over-the-counter medication to assist in weight loss. If you knew you were in the experimental group, you might be more vigi- lant about your diet, or you might take the stairs more often because you know you are receiving a drug designed to help lose weight. Although those are good ideas (eating better, exercising more), for the sake of evaluating the effectiveness of the new drug, those changes in behavior are confounds. A confound is an event or occurrence that hap- pens at the same time of your study, is not part of your designed study, but can influence the outcome of your study. For instance, if you had students in a computer lab complet- ing online surveys, and the power went out on the lab computers the students were using, this is an event that occurred during your study, not part of your study, but could influence the results—students starting the study all over again when the power came back on might be frustrated and respond differently than they did originally. As for the single-blind experiment, it would be better if both groups were kept “in the dark” about their group membership until after the study was over (of course, this research would involve participants providing informed consent prior to the study beginning). In research involving psychotherapy, the demand characteristics may be the cues that communicate the therapist’s expectations, wishes, and general approach that the thera- pist uses with clients—of course these influences have the possibility of unduly influ- encing the behavior of the client during psychotherapeutic research (Kanter, Kohlen- berg, & Loftus, 2002).

Case Study: Analyzing Research-Based Journal Articles

It will appear at times that journal articles published in psychology are written in a foreign language. Psychology has its own jargon and methodology for presenting research findings just as so many other social sciences and sciences do. Of course you can read about how to read about journal articles, but perhaps the best practice is to just dive in and practice, practice, practice.

Perhaps you have already conducted a literature search and you have some journal articles available to you. Or you have copies of journal articles you used for a previous research paper. Locate some of those articles now. To the best of your ability, try to answer the following items below, linking these questions to the answers you would extract from the journal article you have selected. The more you practice this skill, the better you will become at this skill. If you wish, you can try this on your own at first and then work with others on the task.

After reading the journal article, working by yourself or in groups, think about and generate possible answers to the following questions:

1. What are the design elements and operational definitions? 2. What are the potential confounds? 3. What are the strengths and weaknesses of the study design? 4. Researchers strive to randomly assign the participants to the experimental and control

conditions. What other strategies could they have used for random assignment in the study you selected?

5. What possible threats to internal validity might be created in this study? Internal validity repre- sents our confidence that the scores being measured truly represent the psychological concepts we think the scores represent. That is, the dependent variable really is what we say it is.

6. External validity represents our confidence that the conclusions from one particular study can be generalized beyond that particular study. That is, can we apply the results from the study to other situations, places, and times in history?

lan66845_03_c03_p063-100.indd 76 4/20/12 2:47 PM

77

CHAPTER 3Section 3.3 Participant Selection Challenges in Between Groups Designs

The Use of Placebos and Double-Blind Experiments

In a between groups design, the single-blind experiment is designed to keep the partici- pants “in the dark” about which experimental condition they are in, so that the partici- pants do not spontaneously change their behavior due to expectancies of the research (experimental expectancy) or their own perceptions of what they are being asked for in the experiment (demand characteristics). Depending on the type of experiment being conducted, the participants may want to ask a researcher in charge of the study about the study, ask questions for clarification, and so forth. In some cases, the experiment- er’s answers to these questions might give the participants a clue about the experimen- tal condition they are in. In our weight-loss study example, if the experimental group received a pill to help them lose weight, and the control group received no pill, then it would be pretty easy for the participants to determine which group they were in. In this

case, the use of placebos and double-blind experimentation may be warranted.

You should know that the use of a double-blind experimental procedure is not without criti- cism. For instance, both Glass (2008) and Hoffer (1999) raised issues about the use of placebo controls in research, especially in conjunction with clinical tri- als to develop new medica- tions (e.g., new medications to help in the treatment of schizo- phrenia or alcoholism). Hoffer (1999) passionately argued that double-blind experiments do not achieve the goals they set out to achieve:

Do these patients know instinctively that these double-blind experiments, the gold standard of modern medicine, are perhaps best labeled as “fools gold” standard, that they are unethical, that they do not remove bias in the evaluation of treatments and that they remove the most essential element of any doctor-patient relationship, hope?” (p. 179).

Of course, this refers to medical research and clinical trials, but psychologists need to be aware of such concerns when including a placebo group and double-blind study condi- tions. Much of the above discussion focuses on medical research and the clinical trial using a placebo double-blind study, but how would this design translate to an example in psychology? When I was an undergraduate, I was fortunate enough to be able to do an independent research project, studying the effects of caffeine on human performance (Landrum, Meliska, & Loke, 1988). I used a double-blind procedure in this caffeine study. Students in the experimental and control (placebo) conditions each received an 8-oz

Criticisms for using placebos in double-blind experiments stem from using placebos in conjunction with clinical trials to develop new drugs.

iStockphoto/Thinkstock

lan66845_03_c03_p063-100.indd 77 4/20/12 2:47 PM

78

CHAPTER 3Section 3.4 Within Groups Designs

serving of Diet Coke. In the experimental condition, 200 mg of caffeine was dissolved in 10 ml of distilled water, and added to the Diet Coke. For the placebo control group, just 10 ml of distilled water was added to the Diet Coke. At the conclusion of the study, we asked participants to guess whether they had received caffeine or the placebo, and a chi- square analysis indicated that participants could not reliably predict their own condi- tion—this manipulation check helps to make the case that participants were truly “blind” to the condition they were in.

My undergraduate mentor, Dr. Charles Meliska of Monmouth College, prepared the 8-oz servings + 10 ml of distilled water, placed them in paper cups, and labeled the cups either “A” or “B”—he knew whether A or B contained caffeine, but I was unin- formed until after the data were collected. I was the experimenter in the room with the participants. When they asked me if they were receiving caffeine or not (of course some participants were curious—a good example of demand characteristics in play)— I honestly could not answer the question so I could not give participants any potential clues. I (as the experimenter) was blind to conditions, and participants were blind to whether they were receiving caffeine or placebo—thus double blind. After consuming the drinks, all participants were then subjected to a number of cognitive tasks to mea- sure (the dependent variables) whether or not caffeine consumption led to changes in cognitive performance.

3.4 Within Groups Designs

Whereas the first section of this chapter was devoted to a discussion of between groups designs, including key features and limitations, we’ll now discuss within groups designs, along with their key features and limitations. Certain types of designs lend themselves to certain specialty areas in psychology. For example, if you were to examine the areas of learning and memory, psychophysics, and perception, you would see that many of these areas are studied utilizing within groups designs (Keren, 1993). A typical memory study would be if we had a participant memo- rize multiple lists of words—for example, a list of high-frequency words (words we com- monly hear in everyday language) and a list of low-frequency words (words not often heard in everyday language). Participants in this within groups design would learn both lists, because a key feature to a within groups design is that the participants are exposed to every level of the independent variable (word frequency), not just one level of the inde- pendent variable. Many studies in the areas of social psychology and personality rely on a between groups design. Within groups designs tend to fall into two categories: (a) the same participant is repeatedly measured on all levels of the independent variable (the participant experiences the same or similar stimuli on multiple trials), and (b) there is a comparison of scores for the same participant, but the scores come from substantially dif- ferent conditions of the experiment (Hellier, 1998; Keren, 1993).

There are some key advantages in using within groups designs that are summarized here, with help from Keren (1993), Hellier (1998), and Reeves and Geiger (1994). First, there is a statistical advantage to using a within groups design. By using the same par- ticipants repeatedly in a within groups design, you increase your statistical power by

lan66845_03_c03_p063-100.indd 78 4/20/12 2:47 PM

79

CHAPTER 3Section 3.4 Within Groups Designs

minimizing the variability of participants. Instead of recruiting 60 individuals for a between groups design, you might only need 20 individuals, repeatedly measured, for a within groups design. This reduction in variability due to the reduction in the numbers of individuals needed increases power. The within groups design also translates into higher degrees of freedom, and with higher degrees of freedom it is easier to reject the null hypothesis. Degrees of freedom is a term from statistics that refers to the number of scores that are free to vary. Because of the design of statistical formulas in inferential statistics, adjustments are made (often such as N − 1) when calculating sample statistics, so the scores that are free to vary are reduced in number to acknowledge the amount of estimation taking place.

A second advantage comes from the ability to compare participant performance across conditions. Thinking back to the previous learning and memory example, what if a third condition were added to the study—memorize a list of mixed-frequency words (some high frequency, some low frequency). If the memory task were to first memorize a list of mixed-frequency words, followed by the exclusively high- or low-frequency words, then this first “mixed” condition could serve as a baseline. Rather than have a separate control group (as you might in a between groups design), each participant would have his or her own control comparison—his or her performance on the mixed-frequency list. In this type of within groups experiment, each participant serves as his or her own control. This type of within groups comparison also helps to reduce variability and increase power, because rather than having a control group of separate individuals for comparison, each person serves as a built-in control group for him- or herself.

A third advantage of the within groups design is the efficiency of the design. In the previ- ous examples, I’ve commented that fewer participants may be needed, and one still has the ability to obtain robust numbers for degrees of freedom for statistical calculations. Reducing the numbers of participants needed for an experiment makes the research pro- cess less expensive and more efficient for hypothesis testing. These are practical matters to consider when determining a program of research—whether it be doing research for your company with a bachelor’s degree in psychology, conducting research in graduate school, or if you are psychologist.

The research design decision can be complex, depending on the hypotheses to be tested, access to a participant pool, type of research being conducted, and so on. Furthermore, some design features can be mixed and matched so that the best features of each are uti- lized in a research design. However, the terminology can become confusing. For example, a factorial design indicates that there are at least two independent variables, but that label alone does not tell you if the independent variables are between groups variables, within groups variables, or a mixture of the two—a mixed design.

Mixed Designs

A mixed design also implies more than one independent variable, but this term commu- nicates a bit more information than the factorial design label. In a mixed design, there is at least one between groups independent variable and at least one within groups indepen- dent variable. There can be more than two independent variables, but we know from the mixed label that there is a mixture of between and within groups independent variables.

lan66845_03_c03_p063-100.indd 79 4/20/12 2:47 PM

80

CHAPTER 3Section 3.4 Within Groups Designs

You should know, however, that in some instances, the term mixed design may refer to a mixture of random- and fixed-effect variables, rather than a mixture of between and within groups variables (Maxwell & Delaney, 2004). A random-effect variable is one where a sample is drawn from a population it hopes to represent (e.g., selecting participants from a population into a sample), while a fixed-effect variable is a variable assumed to be measured without error (e.g., such as the assignment of participants to experimental con- ditions controlled by the researchers) (Newsom, 2006).

Split-Plot Designs

A split-plot design is a type of mixed design, and a factorial design as well. Let’s consider the easiest example of a split-plot design—one with two independent variables. One of the variables, variable “A,” has at least two levels, and each level has a group of randomly assigned participants. The other variable, variable “B,” contains the same participants at every level of A. During 2008–2009, I worked with research assistants to design a booklet that helps students become more testwise (that is, knowing the tips and tricks for test- taking that will help you score better, even when you don’t know the answer). We devel- oped a testwiseness booklet that delivered content to the students, and we also designed a control booklet that was about college in general but did not contain testwiseness tips. Essentially, this is an experimental group (treatment group) and a control group—this is our “A” variable, and participants were randomly assigned to treatment or control.

The dependent variable measurement involved the answers to general trivia questions, but within a set of 60 trivia questions, 20 items each were of easy, medium, and hard dif- ficulty (this was taken from published norms about the trivia items). The “B” variable here was item difficulty, and every participant was exposed to all three levels of difficulty (easy, medium, hard), no matter what condition of “A” he or she was in. It may be easy to think about this in picture form, so Figure 3.5 is a graphic that depicts this split plot design.

Control BookletTestwiseness Booklet

A1

Easy

B1

Medium

B2

Hard

B3

Easy

B1

Medium

B2

Hard

B3

A2 Between Groups

Independent Variable— random assignment

Within Groups Independent Variable—fixed assignment

In a mixed group design, there is at least one between groups independent variable and at least one within groups independent variable; thus mixed. In this example, one group of participants received the testwiseness booklet and a separate group received a control booklet. However, all participants received easy, medium, and hard problems to solve—a within groups variable.

Figure 3.5: Example of a mixed group design

lan66845_03_c03_p063-100.indd 80 4/20/12 2:47 PM

81

CHAPTER 3Section 3.4 Within Groups Designs

So, in a split-plot design, each level of A (A1 and A2 above) contains a different group of randomly assigned participants; A1 were students who received the testwiseness booklet, and the A2 students were a different group of students who received the control book- let. Additionally, in a split-plot design, each level of B (B1, B2, B3) at any given level of A contains the same participants. For A1, these participants receive all the levels of B (easy, medium, and hard trivia items). In the language of research designs, the whole plot factor is the variable that is applied widely, and in our example, that would be the A variable, whether or not the participant was in the experimental group or in the control group. The split-plot factor is where a variable is divided in multiple subplots, and in our exam- ple that is the B variable, or the difficulty levels of the trivia items. Connecting with our random-effect, fixed-effect discussion from earlier, in this example the A variable is the random-effect variable, and the B variable is the fixed-effect variable.

Repeated Measures Designs

A repeated measures design is often employed when there is some desire to moni- tor change over time. The types and varieties of repeated measures designs are fairly straightforward. One typical use of the repeated measures design is to test the same person on the same task over time. Over short intervals of time, this might be referred to as a before-and-after situation, or a pretest-posttest condition (Hadzi-Pavlovic, 1986; Price, 2000). Or, the intervals could be longer, in which case the design could be referred to as a longitudinal design. No matter the time interval, this repeated measures design focuses on the same individual responding to the same experimental situation over two or more instances. Another variety of the repeated measures design is when the same individual is exposed to multiple (yet) related levels of the same within groups independent variable (Hadzi-Pavlovic, 1986; Price, 2000). For example, this could be that in a drug study, the same person receives dosages of 5 mg/kg, 15 mg/kg, and 25 mg/kg over multiple trials in the study. In a study that I will describe in more detail later, Landrum and Clark (2006) used a repeated measures design to determine which type of chapter outline (traditional, graphical, or alpha- betical) students preferred when reading a text- book. The participants answered the same sur- vey three times, but the stimulus materials they were responding to differed in each condition.

Repeated measures designs typically yield data that are correlated, because the dependent vari- able observations that repeatedly come from the

A repeated measures design is used when a researcher wants to study a change over time, such as how this experimenter is studying the brain function of a participant over the course of the study.

age fotostock/SuperStock

lan66845_03_c03_p063-100.indd 81 4/20/12 2:47 PM

82

CHAPTER 3Section 3.4 Within Groups Designs

same individual should, in fact, be related (Gueorguieva & Krystal, 2004). The correla- tion between the multiple scores in a repeated measures design is known as sphericity, and if the sphericity assumption is violated (such as an extended length of time between the repeated measures), then the statistical analysis is weakened while inflating the probability of a Type I error (Gliner, Morgan, & Harmon, 2002). In a repeated measures design, participants are more efficiently utilized because they provide data over and over again, reducing the number of participants needed. In addition, there is a statisti- cal advantage in the reduction of variability because of the need for fewer individuals, as well as the ability to compare individuals’ results to earlier data points (“each person serves as his/her own control”). These two advantages are formidable when appropriate for the experimental situation (Gliner et al., 2002). Gliner et al. (2002) also enumerated the disadvantages to a repeated measures design, such as carryover effects (mentioned later in this chapter). Basically, if in one of the conditions of the independent variable a participant has learned a new skill, for example, that skill cannot be readily “unlearned.” When looking for change, there are other concerns as well, such as ceiling and floor effects, which are addressed later.

Participant Assignment: Matching or Blocking

Matching and blocking represent two strategies that are used to either reduce the num- ber of explanations of how an independent variable influences a dependent variable, or to add a finer-grained explanation about variables. Although not always the case, the matching procedure tends to be associated with a between groups design and the block- ing procedure tends to be associated with a within groups design. Part of the confusion in understanding these methodological and statistical approaches is that researchers are not always consistent in their use of terminology, and sometimes specialty disciplines use their own terminology, which may or may not be widely adopted. For example, the term “split plot” comes from agriculture research, where plots of land were split or divided into different conditions to be tested. Although we use the term split plot, you’ll sometimes see the term randomized block used synonymously for split plot (Wuensch, 2005). Matching typically refers to the pairing of participants based on similar measures on a targeted variable. Matching may allow for some statistical analysis advantages regarding the reduction of group variability, but “its greatest allure, however, is the hope that matching will produce equivalent groups or control for variability between groups used for matching” (Camp, 1995, p. 54).

Imagine that you work in the Institutional Assessment and Research (IAR) office at your institution and you’ve been asked to conduct a study that is interested in determining a student’s satisfaction with his or her undergraduate experience. To help forge fond memories of the collegiate experience, the institution is about to require a “capstone experience” course, but for now this is just an elective, and the point of the study is to see how the capstone experience might (or might not) influence a graduate’s satisfaction level. Understanding a student’s satisfaction level could be an important variable for col- leges and universities. So we set out to do a study, and of course we review the literature first, and we discover that in some studies, a student’s GPA may be related to perceptions of satisfaction.

lan66845_03_c03_p063-100.indd 82 4/20/12 2:47 PM

83

CHAPTER 3Section 3.4 Within Groups Designs

A student’s GPA would be considered a subject variable, because as the researcher you cannot assign someone randomly to a GPA level—it is a characteristic that students already possess. With this type of variable (a subject variable that we suspect is related to our dependent variable of interest—student satisfaction), we would consider using a matching or blocking strategy. Let’s consider matching first. You might retrieve the cur- rent GPAs from the registrar’s office of all of this year’s graduating seniors. In the follow- ing example, 20 seniors make up the sample. To see the benefit of matching, let’s consider the worst-case scenario first. If you retrieved those 20 GPAs from the registrar’s office and rank ordered them from high to low, they might look like what is depicted in Figure 3.6.

From these 20 students, you ran- domly assign 10 to the “experi- mental” condition, which would be to take the capstone experi- ence course, and 10 to the control condition, who do not take the course. Your comparison will be between the experimental condi- tion and the control condition on the dependent variable, col- lege satisfaction. But if you know from the review of the literature that GPA is a key variable that influences college satisfaction, you might want to control for GPA. One method of controlling for it would be to have roughly equivalent groups prior to the introduction of the independent variable manipulation (the cap- stone course). But say, in this worst case scenario, that you left group assignment to chance, and by some fluke of randomization, the top 10 students in the list were assigned to the experimen-

tal group, and the bottom 10 students were randomly assigned to the control group (See Figure 3.6; the solid line separates the top 10 from the bottom 10). These two groups would not be roughly equivalent on GPA before introducing the capstone experience independent variable. In fact, the mean for the top 10 students is 3.68 (SD = 0.2), and the mean for the bottom 10 students is 3.00 (SD = 0.2). Sometimes leaving group assign- ment to randomization alone may not be good enough (especially with a small N). One approach to help achieve at least roughly equivalent groups would be to use matching, as depicted in Figure 3.7.

4.0 3.9 3.9 3.8 3.7 3.7 3.6 3.4 3.4 3.4

3.3 3.2 3.1 3.1 3.0 3.0 3.0 2.9 2.8 2.6

In a sample of 20 students, this could be a distribution of GPAs that have been arranged from high to low. Even by using randomization, groups can be non-equivalent prior to the start of a study.

Figure 3.6: A hypothetical distribution of GPAs

lan66845_03_c03_p063-100.indd 83 4/20/12 2:47 PM

84

CHAPTER 3Section 3.4 Within Groups Designs

In this rank-ordered list of GPAs, the nearest pairs have been matched (as indicated by the ovals), and then from within each oval, there has been ran- dom assignment into either the experimental condition or the control condition. This procedure won’t guarantee equivalent GPAs for the two groups prior to the introduction of the independent variable, but the groups should be much closer. In fact, the mean GPA for the experimental group is now 3.36 (SD = 0.4), and the mean GPA for the control group is 3.32 (SD = 0.4). The benefit of match- ing is that this procedure roughly equates the groups on the extra- neous variable (GPA) before the study begins. Thus, after the study is over, if the capstone experience does indeed positively influence student satisfaction, we can say with some confidence that GPA is not the driving force behind the improvements in student satisfac- tion, because GPA was roughly equivalent in each group prior to the capstone experience. By using matching, we attempt to wipe out any possible influence of GPA.

Blocking, however, takes a differ- ent approach. Rather than wipe out the effect of GPA, by using blocking, GPA is turned into a variable of interest in the experiment, which allows us to examine potential interactions between GPA and how the capstone experience might influence student satisfaction. With blocking, we turn the potentially extraneous variable (GPA in our example) into an independent variable, which will allow us to examine if this variable interacts with the intended independent variable, capstone experience. Perhaps the capstone experience course is only beneficial for influencing student satisfaction with high GPA students. If that is the case, then the blocking design will capture that interac- tion, whereas in the matching design, the effect of GPA is erased by matching the pairs and then randomly assigning to experimental and control conditions. So in keeping with our example, we would start by “blocking on GPA,” as depicted in Figure 3.8.

–experimental3.6

–experimental3.4

–experimental3.3

–experimental3.1

–experimental3.0

–experimental2.9

–experimental2.8

–experimental3.7

–experimental3.9

–experimental3.9

–control4.0

–control3.8

–control3.7

–control3.4

–control3.4

–control3.2

–control3.1

–control3.0

–control3.0

–control2.6

This is a graphic depiction of matching, which may be used to avoid a situation where randomization alone may not ensure roughly equivalent groups. After arranging the participants from high to low on a GPA measure, matched pairs are identified (the circled pairs), and then a coin flip is used to determine which participant is assigned to the experimental group and which participant is assigned to the control group.

Figure 3.7: An example of matching

lan66845_03_c03_p063-100.indd 84 4/20/12 2:47 PM

85

CHAPTER 3Section 3.4 Within Groups Designs

In this example, we have purposely grouped or blocked the higher GPA students together and the lower GPA students together, because we are going to “block on GPA” and turn GPA into an independent variable. In blocking, an individual block needs to be composed of a homogeneous (similar) group of individuals—in this case, relatively high GPAs. Thus, this new independent variable has two GPA levels—high and low. In the next step of blocking, each group of homogeneous participants is split so that the effect of the capstone experience independent variable can be estimated—see how this is depicted in Figure 3.9.

3.6

Higher GPA Group

Lower GPA Group

3.4

3.7

3.9 3.9 4.0

3.8

3.7

3.4 3.4

3.3

3.1

3.0

2.9 2.8

3.2

3.1

3.0 3.0

2.6

In blocking, a group of individuals is “blocked” together as a group (as compared to matched in a matching design). Eventually, the block will be split into experimental and control groups; sometimes this is called a split-plot design.

Figure 3.8: An example of blocking

lan66845_03_c03_p063-100.indd 85 4/20/12 2:47 PM

86

CHAPTER 3Section 3.4 Within Groups Designs

Thus, with the randomized block design, participants were grouped into blocks based on their GPAs, then each block was split randomly, and then each split plot was randomly assigned either to the experimental group (capstone experience) or control group. In this way, GPA becomes a variable in the experiment, and now we can test to see if GPA inter- acts with the capstone experience in affecting student satisfaction.

Both the matching and blocking design offer advantages and disadvantages. For the matching design, the effect of a potential extraneous/nuisance variable can be washed away, and a potential confound can be minimized. However, the matching procedure adds a great deal of effort to the experimental procedure. All of the matching must be completed prior to the actual study commencing. With GPAs from the registrar’s office, this might not be so much work, but if you were to match on variables that require more effort (such as self-esteem levels or intelligence), this would add to the time and resources necessary for matching—and researchers are able to match on more than one variable, so the complexity of matching can increase dramatically. The advantage of blocking is that it adds the ability to ascertain the influence of the variable in question (GPA) into the experimental mix, and rather than wiping it out (as in the matching approach), blocking allows us to observe potential interactions. However, blocking involves more work, and it is not always easy to find homogeneous groups of participants that can be adequately split. In sum, both techniques can be useful additions to your methodological arsenal, but it is important to know what each can do for you, as well as be aware of the additional costs in terms of time, resources, and energy expenditure.

3.6

Higher GPA Group

Higher GPA Split Plot

Higher GPA Split Plot

Lower GPA Split Plot

Lower GPA Split Plot

Experimental Group

Control Group

Experimental Group

Control Group

Lower GPA Group

3.4

3.7

3.9 3.9 4.0

3.8

3.7

3.4 3.4

3.3

3.1

3.0

2.9 2.8

3.2

3.1

3.0 3.0

2.6

After forming groups of homogeneous participants on a variable of interest (in this example, GPA), the high GPA group can be split into two plots, and then one receives the independent variable manipulation and the other serves as the control group. This procedure is again repeated for the low GPA group; split into plots and then experimental conditions are assigned.

Figure 3.9: An example of a split-plot design

lan66845_03_c03_p063-100.indd 86 4/20/12 2:47 PM

87

CHAPTER 3Section 3.5 Special Issues Using Within Groups Designs

3.5 Special Issues Using Within Groups Designs

As with any type of design approach, there are advantages and disadvantages, and some of these ideas have been alluded to in previous parts of this chapter. When utilizing a within groups design, there are a number of issues to consider, but many of these issues have strategies for resolution. In addition, occasionally there are design limitations that may hinder the interpretation of an outcome (or point to an alterna- tive approach that may potentially yield superior results). We’ll address some of the major issues facing within groups designs, such as accounting for pretreatment differences and ceiling and floor effects, and dealing with order effects through counterbalancing.

Floor and Ceiling Effects

In within groups designs that are particularly focused on the analysis of change over time, such as in a repeated measures design, the consideration of potential floor and ceiling effects is important. Floor effects and ceiling effects are related to the notion of regression toward the mean (Altermatt, 2008). A floor effect occurs when you are working with scores at the very low end of the distribution of scores. If you start with individuals with very low scores, and you give them the same test again (as you would in a repeated measures design), those at the low end cannot go any lower than low, and some of the scores might increase by default. In essence, very low scores have nowhere else to go but up. The problem with this floor effect is that you may see an increase in scores and think that your independent vari- able is effective, when the increase in scores is due to something else: regression toward the mean. Think about the distribution here. The example that Altermatt (2008) uses to indicate the floor effect is when giving a group of second graders a sixth-grade spelling test. Second graders would typically not do so well on such a test. But after the first administration of the spelling test, you take all the children who scored a 0 (at the very bottom of the distribu- tion), give them the same spelling test again, and some might get a couple of words right, just by guessing or having a second exposure to the words. Thus, scores could increase, and the average score would creep toward the mean. The floor effect occurs when starting with scores so low that there is an inability for those scores to go any lower than low.

The floor effect occurs when you start with low scores, and the goal is to change scores and make them lower. The starting location of the scores (low) has an impact on the ability to measure change, even if the intervention was believed to be effective.

Figure 3.10: The floor effect

lan66845_03_c03_p063-100.indd 87 4/20/12 2:47 PM

88

CHAPTER 3Section 3.5 Special Issues Using Within Groups Designs

As you can imagine, there is a similar process at the other end of the distribution, which is called a ceiling effect. Using Altermatt’s (2008) example, you would give a group of sixth graders a second grade spelling test and then take those tests with the highest scores and re-administer the second grade spelling test. Those students scoring at the high end can either retain the same high score, or score lower, which would be a regression toward the mean. The decrease in scores is merely an artifact of starting so high (i.e., the ceiling effect), and the scores have no other direction to go than down.

Ceiling and floor effects are a concern in within groups designs, especially repeated mea- sures designs, because of the emphasis of detecting a meaningful change over time. But ceiling and floor effects limit meaningful change interpretations because of the regression toward the mean factor. So what would be the better solution?

One alternative would be to include a control group and see if the rates of change over time are similar or different when comparing an experimental group to a control group. Another solution would be to try to avoid selecting individuals from the extremes of the distribution—if scores have the ability to move in either direction (up or down), then ceiling and floor effects can be tested, and the researcher has a better chance of detecting meaningful change if present.

The ceiling effect occurs when you start with high scores, and the goal is to change scores and make them higher. The starting location of the scores (high) has an impact on the ability to measure change, even if the intervention were believed to be effective.

Figure 3.11: The ceiling effect

lan66845_03_c03_p063-100.indd 88 4/20/12 2:47 PM

89

CHAPTER 3Section 3.5 Special Issues Using Within Groups Designs

In an attempt to demonstrate that an intervention is effective, it is preferred to select participants neither at the floor nor at the ceiling, such as those highlighted in the figure, and attempt to show change. It is a fairer test of a hypothesis to be able to support or refute it; thus, avoiding potential floor and ceiling effects is preferred.

Figure 3.12: Avoiding floor and ceiling effects

Carryover Effects, Order Effects, and Counterbalancing

When using a within groups variable, each participant is exposed to every level of that independent variable. Having a participant complete multiple tasks can have some impli- cations as to how the completion of earlier tasks can influence the completion of later tasks. This general concern is typically labeled as a carryover effect—that is, the effect of one level of the independent variable can persist to influence another level of the inde- pendent variable (Goodlet, 2001). Carryover effects can lead to progressive error, which means that factors other than the independent variable are influencing the dependent variable over time. Two basic carryover effects are practice and fatigue (Goodlet, 2001; Hall, 1998). When performance on an earlier trial in the experiment positively influences later results (due to practice, experience, or familiarity), this is known as a practice effect, or positive progressive error. When an earlier trial negatively influences later results (due to fatigue, boredom, or inattention), this is known as a fatigue effect, or negative progres- sive error (Goodlet, 2001; Hall, 1998).

Let’s consider an example of a practice effect (positive progressive error)—hopefully this hasn’t happened to you too many times. A student will sometimes try very hard but may eventually fail a class. Often a student will retake the class with the same instruc- tor, because even though the student failed, he or she is familiar with the instructor’s lecture style, method of testing, classroom interactions, and so on. Think of this situation as a repeated measures design, where the first trial was the first time the student com- pleted the course and the second trial is the second time through the same course with the same instructor. When evaluating how much the student learned from the second time through the course, it is difficult to know this with certainty, because there is a carryover effect from the first time through the course. Certainly the student benefited from already knowing the instructor’s style of teaching, as well as having taken all the instructor’s tests before (even if the actual questions did change from semester to semester). Of course, in

lan66845_03_c03_p063-100.indd 89 4/20/12 2:47 PM

90

CHAPTER 3Section 3.5 Special Issues Using Within Groups Designs

this example, a practice effect is beneficial to the student, but from a researcher’s perspec- tive, this positive progressive error complicated our interpretation of just how much the student learned in the second completion of the course.

Repeated trials can also lead to fatigue effects, or negative progressive error. For example, you and your classmates might each develop a short survey in your Research Methods class, and ask a group of students to complete your surveys so that you can collect real data and apply statistical analyses to your data. If each student prepares a 10-item survey, and there are 20 students in your course, then you will be asking participants to complete 200 survey items. Think about that task—will the attention level of your participants be the same on survey item No. 10 as it is on survey item No. 190?. Participants might get fatigued by answering so many survey questions, or just get bored, and their attention may wander. So if your questions are at the end of survey, you may have less confidence that the participants answered your questions in a serious manner, which ultimately impacts the conclusions drawn.

So how do we minimize the impact of carryover effects? First, you should note that in all situations, we may not be able to reduce carryover effects. For instance, when a student takes a course and then retakes a course, since this is not an experiment, we lack control over the sequence of events. However, in our survey scenario, we could vary the order of presentation of the surveys such that the survey questions for Student 20 do not always occur last in the sequence, and we could order (or rearrange) the sequence so that Student 20 is not as concerned about fatigue effects. The technique that we use to minimize poten- tial carryover effects is called counterbalancing.

Although there are various approaches to counterbalancing, we will focus here on a general approach known as across-subjects counterbalancing (Goodlet, 2001), where dif- ferent participants in a within groups design receive different orders of the levels of the independent variable. The goal of counterbalancing is to vary the orders to such a degree than any order effects are dissipated, or balanced out. For example, for 20 stu- dents asking a 10-item survey each, Student 20 does not always have his or her questions appear last, but different students take turns with their survey questions occurring last. “The basic idea of counterbalancing is to spread any order effects evenly across experi- mental conditions so that order effects will not be confounded with experimental treat- ments” (Wuensch, 2007, ¶5). In complete counterbalancing, every possible combination of orders is presented to different groups of participants. As briefly mentioned earlier, I did research looking at the types of outlines students prefer with their textbooks (Lan- drum & Clark, 2006). In that study, we tested three types of outlines: (a) a traditional, Roman-numeral outline; (b) a graphical outline, using a concept map to connect related concepts; and (c) an alphabetical listing of the key terms from the chapter, not presented chronologically. At the end of each outline presentation, participants answered questions about their preference and use of the outline presented. This study used a counterbalanc- ing strategy known as complete counterbalancing, because we presented every possible order. With 3 levels of the within groups variable (traditional, graphical, alphabetical), there are 6 possible orders—to calculate the number of possible orders, use N!, where N is the number of levels (3), and the ! (factorial) symbol means that you multiply that whole number by every whole number less than that down to the number 1—such as in 3*2*1 = 6 orders. You can see the orders in Table 3.1.

lan66845_03_c03_p063-100.indd 90 4/20/12 2:47 PM

91

CHAPTER 3Section 3.6 Limitations of Experiments

Table 3.1: Outlines and possible orders

Order 1 Traditional Graphical Alphabetical

Order 2 Traditional Alphabetical Graphical

Order 3 Graphical Traditional Alphabetical

Order 4 Graphical Alphabetical Traditional

Order 5 Alphabetical Traditional Graphical

Order 6 Alphabetical Graphical Traditional

In the Landrum and Clark (2006) study, at least 24 participants completed each of the six orders, and we used an “order” independent variable to see if the order of presentation influenced the evaluation of the usefulness of the outlines. Sometimes, we discovered, order did matter! Complete counterbalancing can be useful with small numbers of levels of the within groups independent variable, but the numbers accelerate quickly. If you had 4 levels of the independent variable, it would take 24 different orders (4!). If you had 5 levels of the independent variable, it would take 120 different orders (5!). And, going back to our 20 students enrolled, if you wanted to present every possible order of 20 different surveys to participants, it would take 2.43 × 1018 different orders—not a task we would attempt. So when the total number of orderings is not possible or realistic, we utilize par- tial or incomplete counterbalancing approaches (Goodlet, 2001; Wuensch, 2007).

Within groups designs bring advantages as well as particular challenges, such as carry- over and order effects. Even with these creative workarounds, there are occasions when a within groups design is just not feasible. This occurs when one level of the within groups independent variable may lead to irreversible changes, such as learning (Hall, 1998). For example, what if you were to do a study on the different types of educational approaches used to teach someone how to use the latest version of Microsoft Word? You might have three different approaches, such as a paper-based tutorial, live classroom instruction, and a series of podcasts with instructions. You could pay attention to order effects and vary the order of presentation of the levels of the variable, but can you see how whatever level comes first may irreversibly change the participant? If you were to present a series of instructional podcasts about how to use the newest version of Microsoft Word, once you completed that training, you cannot undo what you have learned and start fresh with the paper-based tuto- rials—you are no longer a blank slate, and the learning from the first condition may carry over into the second condition. Sometimes, even with all the methodological controls that we use, a within groups design may not be the best choice, depending on your hypotheses of interest, and you may have to consider other designs, such as a between groups design.

3.6 Limitations of Experiments

Experiments, like those presented throughout this textbook, provide the strongest ability to draw cause-and-effect conclusions from research. This type of conclusion is very powerful; if we understand cause and effect, we can implement interven- tions and strategies to help positive events occur more often, as well as work to minimize

lan66845_03_c03_p063-100.indd 91 4/20/12 2:47 PM

92

CHAPTER 3Section 3.6 Limitations of Experiments

negative events. In the following chapters, we’ll address research scenarios where experi- mentation is not typically possible—quasi-experimental designs, single-subject designs, and surveys and questionnaires. In these latter situations, we can learn a great deal about human behavior, but the strength of our conclusions is typically not as strong as those conclusions drawn from experiments. However, as you will quickly see, not all situa- tions allow experimentation, nor random assignment to conditions, nor strict experimen- tal control of the situation, and so forth. Thus, we have multiple tools in the toolbox to address those situations.

Even though the experiment can yield valuable information, it too has limitations, as does any approach. In many ways, the experimental method is revered for elegance of its logic and the strength of conclusions drawn from it, but it is not the one-size-fits-all solution for research in psychology. Those psychologists who perform work in the field or clini-

cal studies are often unable to utilize the strict experimental controls of the laboratory (Levine, 1974). One of the limitations of the experiment is “that there is no such thing as a social vacuum” (Levine, 1974, p. 663). Just because a participant is brought into a laboratory setting does not mean that the participant is not influenced by the social context of the setting, the experimenters, the task, and so on. Levine expands on this theme when he reminds us that important human problems involve whole human events that occur in a his- torical and social context. What does this imply? Even though you may be conducting an experi- ment in a laboratory setting using adequate methodological controls, it does not mean that behavior observed in the laboratory is represen- tative of the actual behavior that would be seen outside of the laboratory.

Two other related concerns offered by Levine (1974) include that (a) researchers become a part of the phenomenon they are researching, and we are influenced by the processes we use, and (b) science is essentially a social enterprise, no matter to what extent we strive for objectivity. To summarize, what do these ideas mean in the midst of your applied project? These are the fun- damental ideas to keep in mind:

1. No method, not even an experiment, will yield the type of certain results that most scientists desire.

2. Some scientists overstate the importance of experiments and may not be aware of the social context that influences all that we do.

3. Many research situations exist (e.g., clinical work, fieldwork) that are not amena- ble to experiment/laboratory experimentation as we know it, yet these research situations have scientific merit.

Clinical and field studies are limited in their ability to control conditions. What is meant by the phrase “there is no such thing as a social vacuum”?

Photodisc/Thinkstock

lan66845_03_c03_p063-100.indd 92 4/20/12 2:47 PM

93

CHAPTER 3Chapter Summary

4. Although experiments may yield studies with high internal validity, the general- izability of the research (i.e., external validity) may be limited.

5. Researchers need an arsenal of tools (in addition to methodologies used in experi- ments) to study and understand attitudes and behaviors in the real world.

To achieve this last goal, the chapters that follow will explore quasi-experimental designs, where we apply the fundamental principles of experiments (as best we can) to partici- pants, topics, and scenarios that do not lend themselves well to experimentation. In another chapter you’ll learn about single-subject research, where much can be gained from the systematic study of one individual. A common research technique that will also be explored is the use of surveys and questionnaires, and how to adequately prepare such instruments to allow for meaningful interpretations. Rather than seeking a one-size- fits-all solution to research scenarios, the goal is to provide you with a versatile collection of tools so that you can apply appropriate methodological tools to appropriate research questions.

Chapter Summary

The basic building blocks for any experimental research include (a) whether or not the dependent variable is measured before and after the independent variable manipulation, or just after; (b) whether there is one independent variable or more than one (more than one independent variable receiving the label “factorial design”); (c) whether the primary question of interest is the difference between different groups of people (between groups design) or the change in the same people over time (within groups design); and (d) the primary method by which people are assigned to the condition of an experiment (e.g., matching or blocking). Although the effect of one independent variable can be instructive (a main effect), the complexity of human behavior is more likely to be captured by interactions; that is, when the levels of one independent variable combine with the levels of another independent variable, leading to a unique dependent variable measure. The potential deficiencies in a between groups design are often addressed by the strategies used to assign participants to conditions, such as double-blind studies to avoid demand characteristics and the placebo effect. Within groups research primarily addresses the potential changes in a person over time, such as before-and-after measure (called pretest-posttest). Different types of within groups designs (split plot, repeated mea- sures) are utilized to answer different types of research questions. Each design presents its own unique challenges, and ceiling, floor, carryover, and order effects are just some of the design issues that are addressed in within groups research. Ultimately, an experiment approach can be quite helpful in answering certain types of questions, but one size does not fit all. Other research approaches are necessary to study questions of interest where random assignment is not possible and/or is unethical.

lan66845_03_c03_p063-100.indd 93 4/20/12 2:47 PM

94

CHAPTER 3Concept Check

Research designs from this chapter

Type of Study Design Name

Symbolic Representation

X = treatment or intervention O = observation, dependent

variable measurement

Brief Features

Experimental or Quasi-Experimental

Pretest-Posttest O X O

Allows for measure of change over time. Pre-measure helpful to ensure equivalent groups. However, pre-measure also sensitizes participants to later measures.

Experimental or Quasi-Experimental

Posttest Only X O

Measures effect of treatment without sensitization. However, pre-existing group differences are not detectable without pretest.

Experimental or Quasi-Experimental

Pretest-Posttest Control Group

O X O O O

Enjoys the benefit of change over time, but control group adds a strong comparison. If time alone accounts for the change, the control group pretest-posttest will capture that effect.

Concept Check

1. Twenty-seven patients at a clinic with stage three lung cancer are randomly assigned to traditional medical treatment, homeopathic treatment, or a combina- tion of the two. Based on the literature, researchers predict that the combination will be most effective in reducing the size and number of existing tumors. In this study, what is the independent variable?

A. 27 patients at a clinic B. Lung cancer patients C. Size and number of existing tumors D. Medical, homeopathic, or combined treatment

2. Which of the following would most likely be a non-subject variable?

A. Test scores B. Personality C. Age D. Dental history

lan66845_03_c03_p063-100.indd 94 4/20/12 2:47 PM

95

CHAPTER 3Questions for Critical Thinking

3. Placebos

A. cannot be used in research involving social sciences. B. are people who act like participants but are in on the experiment. C. allow study conditions to be conducted as double-blind. D. are not permitted for human subjects research.

4. An advantage of matching participants is that it

A. decreases the number of variables needed. B. reduces the group variability. C. increases the sample size. D. highlights individual differences.

5. One method to control for floor and ceiling effects is to

A. use multiple independent variables. B. include a control group. C. use a pretest-posttest design. D. measure only one dependent variable.

Answers 1. D. Medical, homeopathic, or combined treatment. The answer can be found in Section 3.1.

2. A. Test scores. The answer can be found in Section 3.1.

3. C. Allow study conditions to be conducted as double-blind. The answer can be found in Section 3.3.

4. B. Reduces the group variability. The answer can be found in Section 3.4.

5. B. Include a control group. The answer can be found in Section 3.5.

Questions for Critical Thinking

1. Think about an aspect of psychology that you are extremely curious about. What type of between groups study would be useful in providing answers to your hypotheses? What would be the independent variables in your study, and what would be the dependent variables? Practice answering these key design ques- tions—would you ask questions pretest-posttest or posttest only? How would individuals be assigned to your levels of the independent variable?

2. Now consider the same scenario for a research question that would utilize a within groups approach. Is there a topic where you would be interested in change over time? How would you account for potential ceiling and floor effects? Is the topic you selected to study with a within groups approach one that could be converted to a between groups design or does it only work for a within groups design?

3. Experiments are valuable in that they can often lead to a cause-and-effect conclu- sion about how the levels of the independent variable bring about changes in the dependent variable measures. Can you generate some examples of when an experiment would be inappropriate? Brainstorm about other possible approaches to gaining information about these types of research scenarios, and then keep reading the next few chapters of this textbook for ideas on how to study phenom- ena that do not lend themselves to experimentation.

lan66845_03_c03_p063-100.indd 95 4/20/12 2:47 PM

96

CHAPTER 3Key Terms to Remember

Key Terms to Remember

ANOVA ANalysis Of VAriance A statisti- cal procedure that allows for the detection of differences when there are three or more levels of an independent variable, or two or more independent variables.

availability sampling When the individu- als selected to participate were conve- niently available to the research.

between groups design A method of study design is that intended to measure differences between separate groups of participants in a study. Ex: freshman, sophomores, juniors, and seniors.

blocking A process of data analysis that turns a potentially extraneous variable into an independent variable, which permits the examination of whether or not the vari- able interacts with the intended indepen- dent variable.

carryover effect The idea that the effect of one level of the independent variable can persist to influence another level of the independent variable.

ceiling effect When a test does not have the ability to identify performance accurately because of the lack of difficult test items.

cognitive dissonance theory A theory developed by Festinger and Carlsmith that occurs when a person privately holds an opinion but is pressured publicly to argue against the privately held opinion, and a form of discomfort or dissonance occurs because of the conflict.

counterbalancing A technique that is used to minimize potential carryover effects in an experiment.

degrees of freedom A statistical term that refers to the number of scores that are free to vary.

demand characteristics When experimen- tal participants try to figure out the nature of the research and “help” the researcher by giving into the perceived demands.

dependent variable The variable that is measured.

double-blind experimentation When neither the study participants nor the experimenter are aware of the conditions being administered during the course of an experiment in order to prevent bias.

factorial design The statistical experiment design in which more than one inde- pendent variable is being manipulated, controlled, or arranged. This enables the experimenter to understand interactions between multiple independent variables.

fatigue effect When an earlier trial negatively influences later results due to fatigue, boredom, or inattention. See nega- tive progressive error.

fixed-effect variable A variable assumed to be measured without error.

floor effect Occurs when you are work- ing with scores at the very low end of the distribution of scores that do not have the potential to go any lower. Thus, sub- sequent attempts yielding improvement may not be accurate because of the scores’ inability to decrease.

homogeneous Variables or conditions that are similar in nature.

lan66845_03_c03_p063-100.indd 96 4/20/12 2:47 PM

97

CHAPTER 3Key Terms to Remember

independent variable The variable that is manipulated, controlled, or arranged/ organized by the researcher.

interactions An effect that allows us to look at the combinations of the levels of the independent variables to examine if these combinations lead to different outcomes compared to other possible combinations.

main effects The overall effect of each of the independent variables considered individually.

matching The pairing of participants based on similar measures on a targeted variable.

mixed design When an experimenter includes both between groups and within groups design features into his or her research.

negative progressive error When an ear- lier trial negatively influences later results due to fatigue, boredom, or inattention. See fatigue effect.

non-subject variable When the value of the independent variable is not determined by the participant but rather by the researcher.

planned comparison When an experi- menter decides which comparisons to conduct when the experiment is being designed and develops questions relevant to that comparison.

positive progressive error When perfor- mance on an earlier trial in the experiment positively influences later results due to practice, experience, or familiarity. See practice effect.

post hoc analyses Analyzing the data after the experiment has been conducted to find patterns that were not outlined in the experiment development.

posttest only When the independent vari- able is measured only after the experimen- tal intervention has been administered.

practice effect When performance on an earlier trial in the experiment positively influences later results due to practice, experience, or familiarity. See positive pro- gressive error.

pretest-posttest When the independent variable is measured both before and after the experimental intervention has been administered.

progressive error When factors other than the independent variable are influencing the dependent variable over time.

qualitative variable A variable in which responses differ in kind or type. The outcomes of these variables are usually described in words.

quantitative variable A variable for which there is some known entity. The outcomes of these variables are usually described in numbers.

quota sampling When a particular makeup of participants in the sample is desired because of its similarity to the gen- eral population.

random assignment When participants are randomly assigned to a group or condition in an attempt to control for any significant differences among groups.

random-effect variable The condition where a sample is drawn from a popula- tion it hopes to represent, such as select- ing participants from a population into a sample.

randomization When individuals are assigned to a study group by chance and not in a predictable manner.

lan66845_03_c03_p063-100.indd 97 4/20/12 2:47 PM

98

CHAPTER 3Web Resources

Web Resources

Pretest-posttest design and explanation of different scenarios in which this experimental design is appropriate to use. Additionally, the challenges associated with this experimen- tal design are also outlined. http://www.experiment-resources.com/pretest-posttest-designs.html

Publication regarding the social psychology of the psychological experiment. It outlines a research study that examined the type of social interaction that psychological research causes and analyzes why these interactions are significant. http://www.psych.upenn.edu/history/orne/orne1962amerpsychol776783.html

randomized block design When partici- pants are grouped into blocks based on a determined variable and then the blocks are randomly split for assignment to an experimental or control group.

regression toward the mean When an experimenter sees a change in scores and thinks the independent variable is effec- tive, but this change in scores is due to something else.

repeated measures design When the experimenter wishes to examine change over time by administering a condition to the same participant over a period of time.

roughly equivalent groups Obtaining groups that are as similar to one another as possible through randomization or another technique because of the unlikelihood of obtaining exactly equivalent groups.

simple effects A statistical test that seeks to identify how the one condition is different from alternative combinations of other inde- pendent variables in the same experiment.

single-blind experiment When the par- ticipant is unaware of the experimental condition he or she is in.

snowball sampling The informal proce- dure where the researcher makes an initial round of contacts to solicit participants for a study, but then invites those contacts to invite others to participate.

sphericity The correlation between the multiple scores in a repeated measures design.

split plot When a variable is divided in multiple subplots.

split-plot design A type of mixed design, and a factorial design where experimen- tal conditions are grouped, such as study guide versus no study guide, and are then separately compared to different sub- groups, such as easy questions, medium questions, and hard questions, to accu- rately determine effectiveness.

subject variable A characteristic, such as GPA, that an experimenter cannot randomly assign because the participant already has that characteristic before par- ticipating in the study.

within groups design An experiment design that aims to measure the change within a participant over time.

lan66845_03_c03_p063-100.indd 98 4/20/12 2:47 PM

99

CHAPTER 3Web Resources

A sample chapter regarding experimental research and variable manipulation with a strong emphasis on test design and the different types of designs a researcher could choose to use. http://www.southalabama.edu/coe/bset/johnson/lectures/lec9.htm

This website details 2×2 factorial design by giving examples and visual representation of different design development strategies. http://web.mst.edu/~psyworld/mixed_designs.htm

lan66845_03_c03_p063-100.indd 99 4/20/12 2:47 PM

lan66845_03_c03_p063-100.indd 100 4/20/12 2:47 PM


Comments are closed.