External validity is one of the main goals of researchers who are trying to attain trustworthy cause-and-effect relationships in qualitative studies.
In fact, if a study has external validity, it means that the results of the research can be generalized to other populations, situations, or settings.
Therefore, external validity is an integral part of psychology studies that are conducted outside the lab setting. Why? Because without external validity, research can’t be generalized and researchers can’t apply the outcome of studies to the real world.
Still, sometimes they prefer to study causal relationships between variables instead of being able to generalize the results. In this article, we’ll explore the meaning of external validity and discuss why it’s so important for producing valid findings.
We’ll also review the 3 types of external validity and the factors that improve or decrease it. In the end, we’ll compare external validity to internal validity.
What is External Validity?
In general, validity refers to the extent to which a psychological instrument assesses exactly what it claims to assess. And the main value of validity is that if a tool has validity, then the results gained with it can be considered accurate and can be interpreted.
The American Psychological Association (APA) defines validity as “the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of conclusions drawn from some form of assessment”.
Nowadays, researchers try to achieve multiple forms of validity, such as internal validity, construct validity, or face validity. External validity is one of them.
External validity is a characteristic of experiments or studies conducted in a natural setting and is related to the generalizability of findings.
While studies conducted in the laboratory setting focus on revealing cause-and-effect relationships by using strong research methods, studies with high external validity are as close to real-life situations as possible.
As a result, the findings can be generalized to real populations.
And generalizability indicates how useful the study is for border situations and wider types of people.
If the results can be generalized, it means that the sample is representative of the entire population.
Consequently, the researcher can claim that their study was actually worth spending so much effort, time, and resources as the results will be used in real-life situations.
That’s how “generalizability” and “representativeness” are related to external validity (Kukull & Ganguli, 2012).
Why Is External Validity Important?
Generally speaking, what is the main aim of any type of psychological research?
In simple words, the purpose of conducting research is to extend knowledge in a specific field, find real-life solutions to certain problems or get results that are useful to help people understand their problems or improve their quality of life.
No matter whether you conduct an educational study, a medication study, a product study, or some sort of peer-reviewed studies, external validity is necessary to ensure that the results can be applied to the real world.
Usually, researchers who are concerned about external validity choose to conduct a field study because field studies are accompanied by higher external validity than laboratory studies.
However, sometimes it’s also possible to manipulate the situation and conduct an experiment with high external validity.
Today, there is a lot of scientific literature that covers the topic of the importance of external validity for studies (e.g., Steckler & McLeroy, 2008).
And indeed, if the researcher aims to use their results in real life, it’s an obligation for them to prove that the study has external validity and can be generalized to wider situations and settings.
Otherwise, the research can’t be replicated in other situations, and the value of the study will be questioned.
3 Types of External Validity With Examples
Based on the focus of the specific study, researchers usually try to attain different types of validity.
Similar to internal, construct, and other types of validity, external validity has its own subtypes.
Generally, there are 3 main types of external validity: 1. population validity; 2. ecological validity; and 3. temporal validity.
We’ll discuss each of them below and provide some examples for better understanding.
Population validity
Population validity is a type of external validity that assesses the degree to which the findings of the study can be generalized to a larger population.
In this case, population refers to the group of people about whom a researcher is attempting to draw conclusions.
Unlike this, a sample is a specific group that takes part in the research.
If the results received on the sample can be applied to a larger population, then it means that the study has broad population validity.
Examples of Population validity
You want to assess the hypothesis about the relationship between exercise and sleep. You predict that regularly taking part in physician exercises improves the quality of sleep in adults.
Your target population is American adults, but your sample consists of about 300 university students.
Even though all of them are adults, in this case, it could be hard to ensure the population validity because the sampling model of students doesn’t represent the entire population of adults in the US.
Therefore, your study has low population validity, and the results of your study can’t be generalized to the larger population.
Ecological validity
Ecological validity is also a form of external validity that represents the degree to which you can accurately generalize the results of the research in different contexts.
In simple words, ecological validity is related to the possibility of applying your findings to the real world.
Therefore, if a study has high ecological validity, the results can be generalized in real-world settings.
However, low ecological validity means that the results can’t be applied outside of the experimental situation.
Examples of Ecological validity
The Milgram Experiment is a classical example of low ecological validity.
In the 1960s, social psychologist Stanley Milgram conducted a series of experiments in order to examine the concept of obedience to authority.
He randomly chose the participants and instructed them to use increasingly high-voltage shocks to punish the actors who reported wrong answers to their questions.
Although the shock wasn’t real and the victims’ reactions were also faked, the study showed a high degree of obedience to authority.
This study has revolutionary findings for social psychology. However, it’s often criticized due to its low ecological validity.
In fact, the situation Milgram created was different from real-life situations.
In the experiment, he created a situation where participants couldn’t avoid obedience to authority. But the real-life situation could be different.
Temporal validity
Other than population and confounding factors, time is also an important factor when it comes to determining external validity.
Temporal validity is related to the progression of time regarding findings. Particularly, this type of validity refers to the degree to which the results of a study can be generalized to other period.
Thus, high temporal validity means that research findings can be accurately applied to various time periods and the variables will still be relevant in the future.
Examples of Temporal validity
Imagine you’re Solomon Asch and you’re conducting a study about conformity.
You discovered that the social pressure from the majority group significantly affects the decisions of the minority. As a result, individuals act conformably.
Even though Asch conducted this research back in the 1950s, the results are still applicable in today’s real-world situations.
Therefore, this study has temporal validity even after almost a century.
List of Threats to External Validity
As we said, achieving external validity isn’t easy because there are several potential threats that hinder external validity.
Various situational factors, sample features, pre-and post-test effects, or testing effects are some of the potential factors that can be perceived as threats to external validity.
Thus, a great researcher always pays attention to the following factors to ensure the external validity of a study.
Pre- and post-test effects
Pre- and post-test effects are some of the most common factors that significantly affect external validity.
A pretest is a survey, questionnaire, or another type of research instrument that is used before the actual process of conducting research begins.
A post-test is an endline of research that is done after some time has passed since the research was conducted to ensure the results are still applicable.
Unfortunately, pre- and post-tests often interfere with the generalizability of the results.
The reason is that added tests eliminate the effect of a causal relationship between variables, and as a result, they pose a threat to external validity.
Sample features
The degree of external validity significantly depends on the type of sampling. The sample is part of the target population.
In fact, the results should be generalized to the population based on the sample features, which means that choosing participants reasonably is one of the most important factors that affect the external validity of the study.
While non-probability sampling methods where participants aren’t chosen randomly and thus don’t represent the population pose a threat to external validity, probability sampling counters selection bias and ensures that each member of the population has the same chance of being selected in the sample.
Selection bias
Selection bias is another threat to external validity, which refers to the error in selecting the participants that take part in the research.
In order to ensure external validity, it’s important for the subjects to have similar features to one another and represent the larger population.
Otherwise, their results can’t be generalized. That’s why researchers try to correct the weighting of factors and control them.
But sometimes it’s hard to avoid selection bias, especially when the participants represent specific groups such as clinical patients or criminals.
In these cases, often the research methodology doesn’t allow the researchers to ensure external validity.
Situational factors
A wide range of situational factors affects the external validity of the study.
In particular, confounding variables such as location, time of the day, noise, temperature, and even researcher characteristics significantly affect the way participants respond to the research manipulations.
That’s why it’s important to control these situational effects as much as possible to avoid biases and ensure external validity.
Hawthorne effect
One specific factor that affects external validity is the so-called Hawthorne effect.
It’s a common concept in social psychology and represents a tendency of research participants to change their behaviors in order to meet the researchers’ requirements.
The reason for this kind of behavior is the natural need to appear socially desirable. As a result, the participants act in a pleasing way, which affects the results and damages the accuracy of the findings.
Aptitude-treatment interaction
Aptitude-treatment interaction (ATI) means that interventions in the study match the attributions and characteristics of a participant.
The reason why this factor affects the external validity is that interactions between participants’ characteristics influence the dependent variable and don’t allow the researcher to effectively control or manipulate it.
Consequently, aptitude treatment is another important threat to external validity.
Factors That Improve External Validity
Considering that sometimes the accuracy of the study findings almost completely depends on the external validity, researchers often try to look for ways to increase the external validity.
Replicating the study, considering psychological realism, trying field experiments or increasing randomization are some of the factors that help improve external validity.
Consider psychological realism
At first glance, the phrase “psychological realism” might sound a bit complicated, but actually, it means that participants in the study perceive the experimental manipulations as real events.
For this, sometimes it’s necessary to cover the story and not reveal the real purpose of the study until the research is finished.
Otherwise, participants might try to appear socially desirable and behave differently.
So, considering psychological realism is one way to improve external validity and generalize the results in real-life settings.
Do reprocessing or calibration
Researchers often use various statistical methods in order to ensure or improve external validity.
For instance, reprocessing data using calibration tables allows researchers to go back and reprocess the data in order to avoid errors while interpreting the results.
This might be the best possible option when there are some issues in the data, such as having uneven groups or having different characteristics.
Replicate
Replication is one of the most frequently used methods when the findings don’t show external validity.
In order to increase validity, researchers often use this method, which means using different samples from the same target population. If they get the same results, then it means that the study has high external validity.
But if the results are different from the previous sample group after the replication, then their results can’t be generalized to the population.
Try field experiments
Another way to ensure external validity is to conduct an experiment outside the lab setting.
Conducting a field study in a natural setting instead of a lab setting where the researcher has to manipulate all the variables themselves is a proven way to improve external validity.
Still, based on the study’s aims and specificity, it’s not always possible to try field experiments.
Use inclusion and exclusion criteria
Using inclusion and exclusion criteria while choosing participants for your sample from the target population makes sure that you have defined the population accurately and that the sample is representative of the population.
As a result, the likelihood of producing reliable and valid results increases in accordance with the degree of external validity.
Increase randomization
Randomization is the process in which participants are assigned to trials completely by chance.
In this case, neither the participant nor the researcher knows which group of participants belongs to which.
As a result, increasing randomization might affect the external validity and improve its degree.
The reason is that randomization increases the generalizability of your results.
External Vs. Internal Validity
Internal validity is another type of validity that’s almost the complete opposite of external validity.
In fact, internal validity is the degree to which the causal relationship that is being assessed is reliable and other factors do not influence the variables.
Internal validity measures the internal consistency of a research instrument or method that is used rather than the generalizability of results.
The main difference between external and internal validity is that internal validity is related to the internal structure of the research methods, while external validity tries to prove that the findings are universal.
Furthermore, in order to achieve internal validity, researchers usually use strong research methods and strictly control extraneous factors.
On the other hand, high external validity means that the results can be applied to practical situations and different contexts.
Respectively, the findings of the studies that have high internal validity prove that a causal relationship between variables is trustworthy, but it doesn’t say anything about the possibility of generalizing the results.
On the contrary, if a study has external validity, the results can be applied to the entire population.
External Validition FAQ
What is an example of external validity?
Paul is a researcher, and currently, he’s taking part in conducting cancer studies to improve pharmacological treatments.
Although his research methods and hypotheses predict valuable insights about the disease, he worries that he might not be able to generalize the administration of treatment to different groups of patients.
The reason is that his study doesn’t have high external validity and represents only the specific type of people that take part in his research.
What is good external validity?
The main purpose of external validity is to help the researcher ensure that their findings can be used in real situations and generalized to other people and settings.
Therefore, the study has good external validity when it’s highly likely to replicate the results in other situations.
The possibility to generalize the findings to other people, times, and settings is what makes external validity good.
How is external validity achieved?
In order to achieve external validity, a researcher should make sure that their conclusions and findings can be generalized to the larger population accurately.
First of all, a representative sample is necessary for external validity. But also, there are several methods to increase external validity.
For instance, by strictly controlling variables, increasing the randomization of the sample, and adding control groups, external validity can be maximized.
How do you determine the external validity of a study?
Determining the external validity of a study is a hard process, but it’s vital for the further use of the findings.
In order to determine external validity, researchers compare the results they receive to other relevant data. Besides, they can repeat the study with other samples of the target population.
However, if the study does not have external validity, this method might be impractical.
