Reliability vs. Validity
September 26, 2011
There’s only one way to find out…
Many people may argue that it is more important to have reliable results rather than valid results, and some may disagree. Or is it more important to have a good balance between the two?
Results are considered to be reliable if consistent results are collected from consistent conditions in an experiment. Achieving reliability can be considered easier than achieving validity as reliability is mostly a result of a well planned out experiment on behalf of the researchers. For example, most researchers can avoid achieving any extraneous variables that may affect the reliability of their results if they carefully plan and execute their research. This can be achieved if the researchers give the same instructions to each participant, that the researchers check that their equipment is working properly and fairly for each participant, and also any assistants used should be fully trained and prepared for the task. However many variables could still affect reliability, such as subject reliability. In observational studies, each researcher has to be fully briefed in detail of the criteria involved in the study. An example would be if a researcher was to look for ‘friendly’ behaviours or ‘aggressive’ behaviours, this must be consistent, as opinion may affect results if one researcher classes a behaviour as friendly and the other classes it as aggressive. Will this ever be fully achieved though? Is subjectivity too hard an issue to ever fully eliminate?
And in the other corner there is Validity. Validity is how appropriate the results are to what the researcher has claimed to have measured. Some may argue that Validity is more important as what would be the point in doing the research if you did not actually measure what you claim to have measured? Validity is not formed based on a single statistic, validity is formed based on a whole body of information. There are two types of validity; external and internal. Internal validity relates to the way in which the study has been conducted through its method and design, and also the extent to which the designers have considered alternative explanations for the results. External validity however relates to generalisability. Generalisability is the ability to apply the research findings to the rest of the target population, assuming based on the results that all members of this target group would achieve similar results. The issue with generalisability is whether or not research should be applied to the whole of the target population. It would be best to test each individual to get their own set of results, as the results gained from the experiment may not always be representative of the population as a whole. Can this ever be fully tested due to expenses and time constraints? Will our results ever be fully valid?
Overall reliability and validity are considered to be essential elements in psychological research, but whether one or the other is more important is questionable. Will we ever achieve fully reliable and valid results, or are there simply too many variables that will influence it?
Psychology Blog 
October 1, 2011 at 2:30 pm
I would argue that we have to be very careful how much emphasis we place upon issues of reliability and validity within research as it gives the perception that the only thing that matters is the end result rather than the participants involved. As you have suggested, there are ways of improving the levels of reliability and validity within studies but these values are impossible to fully achieve. In essence this is due to the subject matter of psychology, no one individual is the same and therefore no sample, irrespective of the size, can be seen as fully reliable or valid. Also, it is impossible to prevent both the participants and the researches from exerting their own influence on the results of the studies. (Rosenthal’s study showed that even the physical characteristics of the researcher, such as gender, influenced the results.)
Because, neither one can be fully achieved I think the debate between them is rather futile, as they are both much of the same.
October 3, 2011 at 1:06 pm
I really enjoyed reading it (: I think you have clearly stated the need for both reliability and validity during research. I would argue that in some respects reliability cannot be more important than validity as without validity there is not much meaning to the research and generalising the results becomes more difficult. However, I agree with your initial statement that perhaps a balance of the two makes for the best results. I believe that neither is more important than the other, as they are both really important in providing accurate and meaningful results. Also, I think subjectivity would be very hard to completely eliminate, particularly in observational studies as individual differences, such as interpretation of behaviour will always play a part in results.
October 3, 2011 at 2:25 pm
The effects of subjectivity in interpretations of observations may only be lessened if a number of researchers criticise and conclude the behaviour observed and the collection of data to produce the more frequent view, though there will always be individuality in results.
Studies are always going to be oversimplified in the generalising of results for external validity as, as brilliant as investigating every individual would be both time and resources could never match such a request or their needs. Nor is it necessarily true that these same individuals would remain consistent through experiences and time in their lifetime (Andy Field 2011). I believe human behaviour is far too complicated to ever be fully converted into statistics that are both reliable and valid, but it doesn’t mean we can’t try. Through practice our abilities and understanding into design will increase validity and reliability and therefore the quality of our knowledge of psychology.
Reference:
Andy Field (2011). Discovering Statistics Using SPSS 3rd Ed.
October 3, 2011 at 2:58 pm
As you pointed out, and emj6 further qualified, objective reliability seems an almost impossible task to achieve. I did some reading into internal reliability and came across these two measures to help maximise reliability.
Test-retest reliability’s main weakness lies in the length of time between tests, which if left too long allows the opportunity for the characteristics of the participant to change or if too short, then may result in a learning effect or memory contamination clouding subsequent tests.
Alternate-forms reliability expresses more of an objective attitude in the cross comparison of two different methods of measuring a phenomena. It then seeks a correllation to suggest that both are (as far as possible) measuring the same thing, yielding maximum reliability.
An interesting study that compares different models of depression severity assessment can be found here:
http://0-www.sciencedirect.com.unicat.bangor.ac.uk/science/article/pii/0022395688900222
October 3, 2011 at 9:38 pm
I found your blog really interesting 🙂 I believe that it is important to aim for reliability and validity in psychological research, despite the fact that it is impossible to ever fully achieve either. Where validity is concerned, I agree that it would be too costly and time-consuming to gather results from every member of the target population, which is why researchers use samples. As you mentioned, it is difficult to obtain valid results from a sample that is considerably smaller than the target population; particularly when individual differences act as an uncontrollable extraneous variable. However, the overall impact of individual differences can be limited depending on the type of sample used. For example, a random sample is considered to be more representative than an opportunity sample because everyone in the target population has an equal chance of being selected. Taking this into consideration, I believe that there are plenty of ways in which researchers can strive to achieve valid results.
October 4, 2011 at 6:17 pm
I agree that reliability and validity are important factors in experiments, in order for them to be accepted and consequently used to support or diprove scientific findings. They also allow the general public to trust the resultss and listen and take notice of the findings. However, as mentioned previously, it is impossible for any findings to be truly reliable or valid, as there would always be other factors affecting the results. I agree that reliability and validity are equally important, as without one the experiment wouldn’t be accurate and therefore wouldn’t provide much use.