Designing and Reporting Experiments in Psychology Peter Harris
     
 
 
 
Designing & Reporting Experiments in Psychology 3/e
 
  Buy this Book  
     
  A. Choosing a statistical test  
  B. Reporting specific inferential statistics  
  C. More on main effects, interactions and graphing interactions  
  D. Rules for writers  
  E. Reporting studies that include questionnaires  
  F. Experimental and nonexperimental data: Some things to watch out for  
  G. Some tips for advanced students to improve your experiments yet further  
  H. Some issues to consider in the RESULTS sections of your later reports and your projects  
  I. Final year projects  
     
 
Related Statistics Books
 
  Pallant, SPSS Survival Manual  
     
  Greene & D'Oliveira, Learning to Use Statistical Tests in Psychology  
     
   
Experimental and Nonexperimental Data

 

F Experimental and nonexperimental data: Some things to watch out for

Once you start to include more than one IV in your design, you need to establish whether the IVs are true IVs in the sense in which I defined these in Chapter 9 of the book. Knowing this will help you to make sensible inferences from any main effects or interactions involving these IVs. (To find out more about main effects and interactions, see Sections 13.4 and 13.5 of the book and Section C of this Web site.) Strictly speaking, you should also only refer to DVs (dependent variables) when these are being measured following true experimental manipulations. (In some places on this website you will therefore find occasions when I have referred to DVs when, strictly speaking, I should have referred to them in some other way.)

Data that are not experimental are all around you, everywhere you look. For a variety of practical and ethical reasons, the studies that you hear about in the media or in your psychology course are not all experimental. However, as I discuss in Chapter 9 of the book, you cannot infer causality unequivocally from data that are not experimental. However, it can be hard sometimes even to recognize that the data are not experimental. For often studies include variables that look like independent variables (IVs), but that are not really IVs. As I discuss in Section 13.8 of the book, sometimes experiments contain variables that look like IVs but that are not in fact “true” independent variables - things like sex (male or female), personality variables (e.g., optimists or pessimists; high-, medium- and low-anxiety groups), differences in attitudes or behaviour (e.g., being for or against abortion; a smoker or nonsmoker). The same caveats about inferring causality apply to these variables as to correlations. (For more on causality and correlation, see Section 9.2 of the book).

A variable is not truly an IV when, as experimenters, we do not have control over which level of the IV is allocated to which participant or which sequence of conditions is allocated to each participant. Because we cannot control this – because we cannot randomly allocate participants to the conditions or the sequence of conditions to the participant – then the levels of our IV are not truly independent. Under these circumstances, we have to be on our guard when we interpret the findings.

This does not mean that we should avoid testing for sex differences or for differences in response between people classified in other ways. On the contrary, we often want to know a great deal about what other responses and variables are associated with being a man or a woman, politically committed or apolitical, being of average or above average IQ, a smoker or a nonsmoker, pro or anti the use of genetically modified food, and so on. The important point is that, even though these variables look like IVs we must remember when reporting the study, and analysing and interpreting the data involving these variables, that they are not IVs in the strict sense and that our conclusions must therefore be qualified and cautious. That is, because they are not experimental in nature, we cannot assume that they are the causal variables when we find changes on the “DV” associated with levels of this “IV”.

You can find examples to illustrate this point in Section 13.8 of the book.

You will come across many examples of such variables in your reading and will use them in your own studies in psychology. It is perfectly appropriate to use such variables in studies and analyses, provided that you are circumspect in the conclusions that you reach. You can run studies with such variables in conjunction with true IVs (e.g., a study on the effects of alcohol and sex on driving, such as those described in Chapter 13 of the book). However, if you only have such variables in your design, make sure that you are aware that it is not an experiment. You therefore need to be able to differentiate a true IV from something that looks like one but is not. See section 13.8 of the book for more on this.

To summarize:

1

If you cannot randomly allocate participants to the levels of an IV or orders of conditions to participants, the IV is not truly an independent variable.

2 Thus we cannot assume that such variables are the causal variables when we find changes on the DV associated with the levels of such variables.
3 It is perfectly acceptable to use such variables alongside true IVs in an experiment. However, you must be circumspect when interpreting any effects involving such variables.
4 If you only have such variables in your study it is not an experiment (despite appearances to the contrary).

 

 

 

 

Open University PressMcGraw-Hill logo