conclusions in observational studies versus experiments

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

Statistics and probability

Course: statistics and probability   >   unit 6.

• Types of statistical studies
• Worked example identifying experiment
• Worked example identifying observational study
• Worked example identifying sample study

Observational studies and experiments

• Appropriate statistical study example
• In an observational study, we measure or survey members of a sample without trying to affect them.
• In a controlled experiment, we assign people or things to groups and apply some treatment to one of the groups, while the other group does not receive the treatment.

Problem 1: Drinking tea before bedtime

• (Choice A)   Observational study A Observational study
• (Choice B)   Experiment B Experiment

Problem 2: Social media and happiness

• One group was directed to use social media sites as they usually do.
• One group was blocked from social media sites.

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

Experiment vs Observational Study: Similarities & Differences

An experiment involves the deliberate manipulation of variables to observe their effect, while an observational study involves collecting data without interfering with the subjects or variables under study.

This article will explore both, but let’s start with some quick explanations:

• Experimental Study : An experiment is a research design wherein an investigator manipulates one or more variables to establish a cause-effect relationship (Tan, 2022). For example, a pharmaceutical company may conduct an experiment to find out if a new medicine for diabetes is effective by administering it to a selected group (experimental group), while not administering it to another group (control group).
• Observational Study : An observational study is a type of research wherein the researcher observes characteristics and measures variables of interest in a subset of a population, but does not manipulate or intervene (Atkinson et al., 2021). An example may be a sociologist who conducts a cross-sectional survey of the population to determine health disparities across different income groups. 

Experiment vs Observational Study

1. experiment.

An experiment is a research method characterized by a high degree of experimental control exerted by the researcher. In the context of academia, it allows for the testing of causal hypotheses (Privitera, 2022).

When conducting an experiment, the researcher first formulates a hypothesis , which is a predictive statement about the potential relationship between at least two variables.

For instance, a psychologist may want to test the hypothesis that participation in physical exercise ( independent variable ) improves the cognitive abilities (dependent variable) of the elderly.

In an experiment, the researcher manipulates the independent variable(s) and then observes the effects on the dependent variable(s). This method of research involves two or more comparison groups—an experimental group that is subjected to the variable being tested and a control group that is not (Sampselle, 2012).

For instance, in the physical exercise study noted above, the psychologist would administer a physical exercise regime to an experimental group of elderly people, while a control group would continue with their usual lifestyle activities .

One of the unique features of an experiment is random assignment . Participants are randomly allocated to either the experimental or control groups to ensure that every participant has an equal chance of being in either group. This reduces the risk of confounding variables and increases the likelihood that the results are attributable to the independent variable rather than another factor (Eich, 2014).

For instance, in the physical exercise example, the psychologist would randomly assign participants to the experimental or control group to reduce the potential impact of external variables such as diet or sleep patterns.

1. Impacts of Films on Happiness: A psychologist might create an experimental study where she shows participants either a happy, sad, or neutral film (independent variable) then measures their mood afterward (dependent variable). Participants would be randomly assigned to one of the three film conditions.

2. Impacts of Exercise on Weight Loss: In a fitness study, a trainer could investigate the impact of a high-intensity interval training (HIIT) program on weight loss. Half of the participants in the study are randomly selected to follow the HIIT program (experimental group), while the others follow a standard exercise routine (control group).

3. Impacts of Veganism on Cholesterol Levels: A nutritional experimenter could study the effects of a particular diet, such as veganism, on cholesterol levels. The chosen population gets assigned either to adopt a vegan diet (experimental group) or stick to their usual diet (control group) for a specific period, after which cholesterol levels are measured.

Read More: Examples of Random Assignment

Strengths and Weaknesses

Read More: Experimental Research Examples

2. Observational Study

Observational research is a non-experimental research method in which the researcher merely observes the subjects and notes behaviors or responses that occur (Ary et al., 2018).

This approach is unintrusive in that there is no manipulation or control exerted by the researcher. For instance, a researcher could study the relationships between traffic congestion and road rage by just observing and recording behaviors at a set of busy traffic lights, without applying any control or altering any variables.

In observational studies, the researcher distinguishes variables and measures their values as they naturally occur. The goal is to capture naturally occurring behaviors , conditions, or events (Ary et al., 2018).

For example, a sociologist might sit in a cafe to observe and record interactions between staff and customers in order to examine social and occupational roles .

There is a significant advantage of observational research in that it provides a high level of ecological validity – the extent to which the data collected reflects real-world situations – as the behaviors and responses are observed in a natural setting without experimenter interference (Holleman et al., 2020)

However, the inability to control various factors that might influence the observations may expose these studies to potential confounding bias , a consideration researchers must take into account (Schober & Vetter, 2020).

1. Behavior of Animals in the Wild: Zoologists often use observational studies to understand the behaviors and interactions of animals in their natural habitats. For instance, a researcher could document the social structure and mating behaviors of a wolf pack over a period of time.

2. Impact of Office Layout on Productivity: A researcher in organizational psychology might observe how different office layouts affect staff productivity and collaboration. This involves the observation and recording of staff interactions and work output without altering the office setting.

3. Foot Traffic and Retail Sales: A market researcher might conduct an observational study on how foot traffic (the number of people passing by a store) impacts retail sales. This could involve observing and documenting the number of walk-ins, time spent in-store, and purchase behaviors.

Read More: Observational Research Examples

Experimental and Observational Study Similarities and Differences

Experimental and observational research both have their place – one is right for one situation, another for the next.

Experimental research is best employed when the aim of the study is to establish cause-and-effect relationships between variables – that is, when there is a need to determine the impact of specific changes on the outcome (Walker & Myrick, 2016).

One of the standout features of experimental research is the control it gives to the researcher, who dictates how variables should be changed and assigns participants to different conditions (Privitera, 2022). This makes it an excellent choice for medical or pharmaceutical studies, behavioral interventions, and any research where hypotheses concerning influence and change need to be tested.

For example, a company might use experimental research to understand the effects of staff training on job satisfaction and productivity.

Observational research , on the other hand, serves best when it’s vital to capture the phenomena in their natural state, without intervention, or when ethical or practical considerations prevent the researcher from manipulating the variables of interest (Creswell & Poth, 2018).

It is the method of choice when the interest of the research lies in describing what is, rather than altering a situation to see what could be (Atkinson et al., 2021).

This approach might be utilized in studies that aim to describe patterns of social interaction, daily routines, user experiences, and so on. A real-world example of observational research could be a study examining the interactions and learning behaviors of students in a classroom setting.

I’ve demonstrated their similarities and differences a little more in the table below:

Experimental and observational research each have their place, depending upon the study. Importantly, when selecting your approach, you need to reflect upon your research goals and objectives, and select from the vast range of research methodologies , which you can read up on in my next article, the 15 types of research designs .

Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to research in education . London: Cengage Learning.

Atkinson, P., Delamont, S., Cernat, A., Sakshaug, J. W., & Williams, R. A. (2021). SAGE research methods foundations . New York: SAGE Publications Ltd.

Creswell, J.W., and Poth, C.N. (2018). Qualitative Inquiry and Research Design: Choosing among Five Approaches . New York: Sage Publications.

Eich, E. (2014). Business Research Methods: A Radically Open Approach . Frontiers Media SA.

Holleman, G. A., Hooge, I. T., Kemner, C., & Hessels, R. S. (2020). The ‘real-world approach’and its problems: A critique of the term ecological validity. Frontiers in Psychology , 11 , 721. doi: https://doi.org/10.3389/fpsyg.2020.00721  

Privitera, G. J. (2022). Research methods for the behavioral sciences . Sage Publications.

Sampselle, C. M. (2012). The Science and Art of Nursing Research . South University Online Press.

Schober, P., & Vetter, T. R. (2020). Confounding in observational research. Anesthesia & Analgesia , 130 (3), 635.

Tan, W. C. K. (2022). Research methods: A practical guide for students and researchers . World Scientific.

Walker, D., and Myrick, F. (2016). Grounded Theory: An Exploration of Process and Procedure . New York: Qualitative Health Research.

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

• Chris Drew (PhD) https://helpfulprofessor.com/author/admin/ Montessori vs Reggio Emilia vs Steiner-Waldorf vs Froebel
• Chris Drew (PhD) https://helpfulprofessor.com/author/admin/ 15 Meritocracy Examples
• Chris Drew (PhD) https://helpfulprofessor.com/author/admin/ 21 Types of Teaching Styles
• Chris Drew (PhD) https://helpfulprofessor.com/author/admin/ 5 Best Laminators for Teachers, Reviewed!

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

User Preferences

Content preview.

Arcu felis bibendum ut tristique et egestas quis:

• Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident

Keyboard Shortcuts

3.4 - experimental and observational studies.

Now that Jaylen can weigh the different sampling strategies, he might want to consider the type of study he is conduction. As a note, for students interested in research designs, please consult STAT 503 for a much more in-depth discussion. However, for this example, we will simply distinguish between experimental and observational studies.

Now that we know how to collect data, the next step is to determine the type of study. The type of study will determine what type of relationship we can conclude.

There are predominantly two different types of studies: 

Let's say that there is an option to take quizzes throughout this class. In an  observational study , we may find that better students tend to take the quizzes and do better on exams. Consequently, we might conclude that there may be a relationship between quizzes and exam scores.

In an experimental study , we would randomly assign quizzes to specific students to look for improvements. In other words, we would look to see whether taking quizzes causes higher exam scores.

Causation Section  

It is very important to distinguish between observational and experimental studies since one has to be very skeptical about drawing cause and effect conclusions using observational studies. The use of random assignment of treatments (i.e. what distinguishes an experimental study from an observational study) allows one to employ cause and effect conclusions.

Ethics is an important aspect of experimental design to keep in mind. For example, the original relationship between smoking and lung cancer was based on an observational study and not an assignment of smoking behavior.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is an Observational Study? | Guide & Examples

What Is an Observational Study? | Guide & Examples

Published on March 31, 2022 by Tegan George . Revised on June 22, 2023.

An observational study is used to answer a research question based purely on what the researcher observes. There is no interference or manipulation of the research subjects, and no control and treatment groups .

These studies are often qualitative in nature and can be used for both exploratory and explanatory research purposes. While quantitative observational studies exist, they are less common.

Observational studies are generally used in hard science, medical, and social science fields. This is often due to ethical or practical concerns that prevent the researcher from conducting a traditional experiment . However, the lack of control and treatment groups means that forming inferences is difficult, and there is a risk of confounding variables and observer bias impacting your analysis.

Table of contents

Types of observation, types of observational studies, observational study example, advantages and disadvantages of observational studies, observational study vs. experiment, other interesting articles, frequently asked questions.

There are many types of observation, and it can be challenging to tell the difference between them. Here are some of the most common types to help you choose the best one for your observational study.

A faster, more affordable way to improve your paper

Scribbr’s new AI Proofreader checks your document and corrects spelling, grammar, and punctuation mistakes with near-human accuracy and the efficiency of AI!

Proofread my paper

There are three main types of observational studies: cohort studies, case–control studies, and cross-sectional studies .

Cohort studies

Cohort studies are more longitudinal in nature, as they follow a group of participants over a period of time. Members of the cohort are selected because of a shared characteristic, such as smoking, and they are often observed over a period of years.

Case–control studies

Case–control studies bring together two groups, a case study group and a control group . The case study group has a particular attribute while the control group does not. The two groups are then compared, to see if the case group exhibits a particular characteristic more than the control group.

For example, if you compared smokers (the case study group) with non-smokers (the control group), you could observe whether the smokers had more instances of lung disease than the non-smokers.

Cross-sectional studies

Cross-sectional studies analyze a population of study at a specific point in time.

This often involves narrowing previously collected data to one point in time to test the prevalence of a theory—for example, analyzing how many people were diagnosed with lung disease in March of a given year. It can also be a one-time observation, such as spending one day in the lung disease wing of a hospital.

Observational studies are usually quite straightforward to design and conduct. Sometimes all you need is a notebook and pen! As you design your study, you can follow these steps.

Step 1: Identify your research topic and objectives

The first step is to determine what you’re interested in observing and why. Observational studies are a great fit if you are unable to do an experiment for practical or ethical reasons , or if your research topic hinges on natural behaviors.

Step 2: Choose your observation type and technique

In terms of technique, there are a few things to consider:

• Are you determining what you want to observe beforehand, or going in open-minded?
• Is there another research method that would make sense in tandem with an observational study?
• If yes, make sure you conduct a covert observation.
• If not, think about whether observing from afar or actively participating in your observation is a better fit.
• How can you preempt confounding variables that could impact your analysis?
• You could observe the children playing at the playground in a naturalistic observation.
• You could spend a month at a day care in your town conducting participant observation, immersing yourself in the day-to-day life of the children.
• You could conduct covert observation behind a wall or glass, where the children can’t see you.

Overall, it is crucial to stay organized. Devise a shorthand for your notes, or perhaps design templates that you can fill in. Since these observations occur in real time, you won’t get a second chance with the same data.

Step 3: Set up your observational study

Before conducting your observations, there are a few things to attend to:

• Plan ahead: If you’re interested in day cares, you’ll need to call a few in your area to plan a visit. They may not all allow observation, or consent from parents may be needed, so give yourself enough time to set everything up.
• Determine your note-taking method: Observational studies often rely on note-taking because other methods, like video or audio recording, run the risk of changing participant behavior.
• Get informed consent from your participants (or their parents) if you want to record:  Ultimately, even though it may make your analysis easier, the challenges posed by recording participants often make pen-and-paper a better choice.

Step 4: Conduct your observation

After you’ve chosen a type of observation, decided on your technique, and chosen a time and place, it’s time to conduct your observation.

Here, you can split them into case and control groups. The children with siblings have a characteristic you are interested in (siblings), while the children in the control group do not.

When conducting observational studies, be very careful of confounding or “lurking” variables. In the example above, you observed children as they were dropped off, gauging whether or not they were upset. However, there are a variety of other factors that could be at play here (e.g., illness).

Step 5: Analyze your data

After you finish your observation, immediately record your initial thoughts and impressions, as well as follow-up questions or any issues you perceived during the observation. If you audio- or video-recorded your observations, you can transcribe them.

Your analysis can take an inductive  or deductive approach :

• If you conducted your observations in a more open-ended way, an inductive approach allows your data to determine your themes.
• If you had specific hypotheses prior to conducting your observations, a deductive approach analyzes whether your data confirm those themes or ideas you had previously.

Next, you can conduct your thematic or content analysis . Due to the open-ended nature of observational studies, the best fit is likely thematic analysis .

Step 6: Discuss avenues for future research

Observational studies are generally exploratory in nature, and they often aren’t strong enough to yield standalone conclusions due to their very high susceptibility to observer bias and confounding variables. For this reason, observational studies can only show association, not causation .

If you are excited about the preliminary conclusions you’ve drawn and wish to proceed with your topic, you may need to change to a different research method , such as an experiment.

• Observational studies can provide information about difficult-to-analyze topics in a low-cost, efficient manner.
• They allow you to study subjects that cannot be randomized safely, efficiently, or ethically .
• They are often quite straightforward to conduct, since you just observe participant behavior as it happens or utilize preexisting data.
• They’re often invaluable in informing later, larger-scale clinical trials or experimental designs.

Disadvantages

• Observational studies struggle to stand on their own as a reliable research method. There is a high risk of observer bias and undetected confounding variables or omitted variables .
• They lack conclusive results, typically are not externally valid or generalizable, and can usually only form a basis for further research.
• They cannot make statements about the safety or efficacy of the intervention or treatment they study, only observe reactions to it. Therefore, they offer less satisfying results than other methods.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

The key difference between observational studies and experiments is that a properly conducted observational study will never attempt to influence responses, while experimental designs by definition have some sort of treatment condition applied to a portion of participants.

However, there may be times when it’s impossible, dangerous, or impractical to influence the behavior of your participants. This can be the case in medical studies, where it is unethical or cruel to withhold potentially life-saving intervention, or in longitudinal analyses where you don’t have the ability to follow your group over the course of their lifetime.

An observational study may be the right fit for your research if random assignment of participants to control and treatment groups is impossible or highly difficult. However, the issues observational studies raise in terms of validity , confounding variables, and conclusiveness can mean that an experiment is more reliable.

If you’re able to randomize your participants safely and your research question is definitely causal in nature, consider using an experiment.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

An observational study is a great choice for you if your research question is based purely on observations. If there are ethical, logistical, or practical concerns that prevent you from conducting a traditional experiment , an observational study may be a good choice. In an observational study, there is no interference or manipulation of the research subjects, as well as no control or treatment groups .

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned.

Exploratory research aims to explore the main aspects of an under-researched problem, while explanatory research aims to explain the causes and consequences of a well-defined problem.

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is an Observational Study? | Guide & Examples. Scribbr. Retrieved November 19, 2023, from https://www.scribbr.com/methodology/observational-study/

Is this article helpful?

Tegan George

Other students also liked, what is a research design | types, guide & examples, guide to experimental design | overview, steps, & examples, naturalistic observation | definition, guide & examples, what is your plagiarism score.

An official website of the United States government

Here's how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

•   Pinterest

Latest Earthquakes |    Live WebChat   Share Social Media  

Ways of learning: Observational studies versus experiments

Manipulative experimentation that features random assignment of treatments, replication, and controls is an effective way to determine causal relationships. Wildlife ecologists, however, often must take a more passive approach to investigating causality. Their observational studies lack one or more of the 3 cornerstones of experimentation: controls, randomization, and replication. Although an observational study can be analyzed similarly to an experiment, one is less certain that the presumed treatment actually caused the observed response. Because the investigator does not actively manipulate the system, the chance that something other than the treatment caused the observed results is increased. We reviewed observational studies and contrasted them with experiments and, to a lesser extent, sample surveys. We identified features that distinguish each method of learning and illustrate or discuss some complications that may arise when analyzing results of observational studies. Findings from observational studies are prone to bias. Investigators can reduce the chance of reaching erroneous conclusions by formulating a priori hypotheses that can be pursued multiple ways and by evaluating the sensitivity of study conclusions to biases of various magnitudes. In the end, however, professional judgment that considers all available evidence is necessary to render a decision regarding causality based on observational studies.

Citation Information

Related content, douglas johnson, research statistician emeritus.

Encyclopedia of Gerontology and Population Aging pp 1748–1756 Cite as

Experimental Studies and Observational Studies

• Martin Pinquart 3  
• Reference work entry
• First Online: 01 January 2022

473 Accesses

Experimental studies: Experiments, Randomized controlled trials (RCTs) ; Observational studies: Non-experimental studies, Non-manipulation studies, Naturalistic studies

Definitions

The experimental study is a powerful methodology for testing causal relations between one or more explanatory variables (i.e., independent variables) and one or more outcome variables (i.e., dependent variable). In order to accomplish this goal, experiments have to meet three basic criteria: (a) experimental manipulation (variation) of the independent variable(s), (b) randomization – the participants are randomly assigned to one of the experimental conditions, and (c) experimental control for the effect of third variables by eliminating them or keeping them constant.

In observational studies, investigators observe or assess individuals without manipulation or intervention. Observational studies are used for assessing the mean levels, the natural variation, and the structure of variables, as well as...

This is a preview of subscription content, access via your institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Atalay K, Barrett GF (2015) The impact of age pension eligibility age on retirement and program dependence: evidence from an Australian experiment. Rev Econ Stat 97:71–87. https://doi.org/10.1162/REST_a_00443

CrossRef   Google Scholar  

Bergeman L, Boker SM (eds) (2016) Methodological issues in aging research. Psychology Press, Hove

Google Scholar  

Byrkes CR, Bielak AMA (under review) Evaluation of publication bias and statistical power in gerontological psychology. Manuscript submitted for publication

Campbell DT, Stanley JC (1966) Experimental and quasi-experimental designs for research. Rand-McNally, Chicago

Carpenter D (2010) Reputation and power: organizational image and pharmaceutical regulation at the FDA. Princeton University Press, Princeton

Cavanaugh JC, Blanchard-Fields F (2019) Adult development and aging, 8th edn. Cengage, Boston

Fölster M, Hess U, Hühnel I et al (2015) Age-related response bias in the decoding of sad facial expressions. Behav Sci 5:443–460. https://doi.org/10.3390/bs5040443

Freund AM, Isaacowitz DM (2013) Beyond age comparisons: a plea for the use of a modified Brunswikian approach to experimental designs in the study of adult development and aging. Hum Dev 56:351–371. https://doi.org/10.1159/000357177

Haslam C, Morton TA, Haslam A et al (2012) “When the age is in, the wit is out”: age-related self-categorization and deficit expectations reduce performance on clinical tests used in dementia assessment. Psychol Aging 27:778–784. https://doi.org/10.1037/a0027754

Institute for Social Research (2018) The health and retirement study. Aging in the 21st century: Challenges and opportunities for americans. Survey Research Center, University of Michigan

Jung J (1971) The experimenter’s dilemma. Harper & Row, New York

Leary MR (2001) Introduction to behavioral research methods, 3rd edn. Allyn & Bacon, Boston

Lindenberger U, Scherer H, Baltes PB (2001) The strong connection between sensory and cognitive performance in old age: not due to sensory acuity reductions operating during cognitive assessment. Psychol Aging 16:196–205. https://doi.org/10.1037//0882-7974.16.2.196

Löckenhoff CE, Carstensen LL (2004) Socioemotional selectivity theory, aging, and health: the increasingly delicate balance between regulating emotions and making tough choices. J Pers 72:1395–1424. https://doi.org/10.1111/j.1467-6494.2004.00301.x

Maxwell SE (2015) Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? Am Psychol 70:487–498. https://doi.org/10.1037/a0039400

Menard S (2002) Longitudinal research (2nd ed.). Sage, Thousand Oaks, CA

Mitchell SJ, Scheibye-Knudsen M, Longo DL et al (2015) Animal models of aging research: implications for human aging and age-related diseases. Ann Rev Anim Biosci 3:283–303. https://doi.org/10.1146/annurev-animal-022114-110829

Moher D (1998) CONSORT: an evolving tool to help improve the quality of reports of randomized controlled trials. JAMA 279:1489–1491. https://doi.org/10.1001/jama.279.18.1489

Oxford Centre for Evidence-Based Medicine (2011) OCEBM levels of evidence working group. The Oxford Levels of Evidence 2. Available at: https://www.cebm.net/category/ebm-resources/loe/ . Retrieved 2018-12-12

Patten ML, Newhart M (2018) Understanding research methods: an overview of the essentials, 10th edn. Routledge, New York

Piccinin AM, Muniz G, Sparks C et al (2011) An evaluation of analytical approaches for understanding change in cognition in the context of aging and health. J Geront 66B(S1):i36–i49. https://doi.org/10.1093/geronb/gbr038

Pinquart M, Silbereisen RK (2006) Socioemotional selectivity in cancer patients. Psychol Aging 21:419–423. https://doi.org/10.1037/0882-7974.21.2.419

Redman LM, Ravussin E (2011) Caloric restriction in humans: impact on physiological, psychological, and behavioral outcomes. Antioxid Redox Signal 14:275–287. https://doi.org/10.1089/ars.2010.3253

Rutter M (2007) Proceeding from observed correlation to causal inference: the use of natural experiments. Perspect Psychol Sci 2:377–395. https://doi.org/10.1111/j.1745-6916.2007.00050.x

Schaie W, Caskle CI (2005) Methodological issues in aging research. In: Teti D (ed) Handbook of research methods in developmental science. Blackwell, Malden, pp 21–39

Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin, Boston

Sonnega A, Faul JD, Ofstedal MB et al (2014) Cohort profile: the health and retirement study (HRS). Int J Epidemiol 43:576–585. https://doi.org/10.1093/ije/dyu067

Weil J (2017) Research design in aging and social gerontology: quantitative, qualitative, and mixed methods. Routledge, New York

Download references

Author information

Authors and affiliations.

Psychology, Philipps University, Marburg, Germany

Martin Pinquart

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Martin Pinquart .

Editor information

Editors and affiliations.

Population Division, Department of Economics and Social Affairs, United Nations, New York, NY, USA

Department of Population Health Sciences, Department of Sociology, Duke University, Durham, NC, USA

Matthew E. Dupre

Section Editor information

Department of Sociology and Center for Population Health and Aging, Duke University, Durham, NC, USA

Kenneth C. Land

Department of Sociology, University of Kentucky, Lexington, KY, USA

Anthony R. Bardo

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Cite this entry.

Pinquart, M. (2021). Experimental Studies and Observational Studies. In: Gu, D., Dupre, M.E. (eds) Encyclopedia of Gerontology and Population Aging. Springer, Cham. https://doi.org/10.1007/978-3-030-22009-9_573

Download citation

DOI : https://doi.org/10.1007/978-3-030-22009-9_573

Published : 24 May 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-22008-2

Online ISBN : 978-3-030-22009-9

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us

Module 4: Probability and Statistics

4.7 observational studies and experiments, learning outcomes.

• Identify the differences between observational studies and experiments

Read with a pencil in hand

Continue to read these sections with a pencil in hand. Make note of definitions, but also make note of keywords that will help you to identify an observational study versus an experiment. Work the TRY IT and EXAMPLES out by hand to gain the experience you’ll need to correctly identify these on a test.

What you’ll learn to do: Examine the methods for sampling and experimentation and how bias can affect the results

As we mentioned previously, the first thing we should do before conducting a survey is to identify the population that we want to study. In this lesson, we will show you examples of how to identify the population in a study, and determine whether or not the study actually represents the intended population. We will discuss different techniques for random sampling that are intended to ensure a population is well represented in a sample.

We will also identify the difference between an observational study and an experiment, and ways experiments can be conducted. By the end of this lesson, we hope that you will also be confident in identifying when an experiment may have been affected by confounding or the placebo effect, and the methods that are employed to avoid them.

Observing vs. Acting

So far, we have primarily discussed observational studies – studies in which conclusions would be drawn from observations of a sample or the population. In some cases these observations might be unsolicited, such as studying the percentage of cars that turn right at a red light even when there is a “no turn on red” sign. In other cases the observations are solicited, like in a survey or a poll.

In contrast, it is common to use experiments when exploring how subjects react to an outside influence. In an experiment, some kind of treatment is applied to the subjects and the results are measured and recorded.

Observational studies and experiments

• An observational study is a study based on observations or measurements
• An experiment is a study in which the effects of a treatment are measured

Here are some examples of experiments:

A pharmaceutical company tests a new medicine for treating Alzheimer’s disease by administering the drug to 50 elderly patients with recent diagnoses. The treatment here is the new drug.

A gym tests out a new weight loss program by enlisting 30 volunteers to try out the program. The treatment here is the new program.

You test a new kitchen cleaner by buying a bottle and cleaning your kitchen. The new cleaner is the treatment.

A psychology researcher explores the effect of music on temperament by measuring people’s temperament while listening to different types of music. The music is the treatment.

These examples are discussed further in the following video.

Is each scenario describing an observational study or an experiment?

a. The weights of 30 randomly selected people are measured

b. Subjects are asked to do 20 jumping jacks, and then their heart rates are measured

c. Twenty coffee drinkers and twenty tea drinkers are given a concentration test

a. Observational study

b. Experiment; the treatment is the jumping jacks

c. Experiment; the treatments are coffee and tea

This is the end of the section. Close this tab and proceed to the corresponding assignment.

• Revision and Adaptation. Provided by : Lumen Learning. License : CC BY: Attribution
• Experiments. Authored by : David Lippman. Located at : http://www.opentextbookstore.com/mathinsociety/ . Project : Math in Society. License : CC BY-SA: Attribution-ShareAlike
• lab-research-chemistry-test. Authored by : PublicDomainPictures. Located at : https://pixabay.com/en/lab-research-chemistry-test-217041/ . License : CC0: No Rights Reserved
• Experiments. Authored by : OCLPhase2's channel. Located at : https://youtu.be/HSTTKzsdHEw . License : CC BY: Attribution
• Confounding. Authored by : OCLPhase2's channel. Located at : https://youtu.be/SrCm12HZay0 . License : CC BY: Attribution
• Controlled Experiments. Authored by : OCLPhase2's channel. Located at : https://youtu.be/UkCHUeqMb5Y . License : CC BY: Attribution
• Blind Experiments. Authored by : OCLPhase2's channel. Located at : https://youtu.be/7BFZVGCxeYc . License : CC BY: Attribution
• Question ID 6736, 6728, 6914. Authored by : Lippman, David. License : CC BY: Attribution . License Terms : IMathAS Community License CC-BY + GPL

Section 1.2: Observational Studies versus Designed Experiments

• 1.1 Introduction to the Practice of Statistics
• 1.2 Observational Studies versus Designed Experiments
• 1.3 Simple Random Sampling
• 1.4 Other Effective Sampling Methods
• 1.5 Bias in Sampling
• 1.6 The Design of Experiments

By the end of this lesson, you will be able to...

• distinguish between an observational study and a designed experiment
• identify possible lurking variables
• explain the various types of observational studies

For a quick overview of this section, watch this short video summary:

To begin, we're going to discuss some of the ways to collect data. In general, there are a few standards:

• existing sources
• survey sampling
• designed experiments

Most of us associate the word census with the U.S. Census, but it actually has a broader definition. Here's typical definition:

A census is a list of all individuals in a population along with certain characteristics of each individual.

The nice part about a census is that it gives us all the information we want. Of course, it's usually impossible to get - imagine trying to interview every single ECC student . That'd be over 10,000 interviews!

So if we can't get a census, what do we do? A great source of data is other studies that have already been completed. If you're trying to answer a particular question, look to see if someone else has already collected data about that population. The moral of the story is this: Don't collect data that have already been collected!

Observational Studies versus Designed Experiments

Now to one of the main objectives for this section. Two other very common sources of data are observational studies and designed experiments . We're going to take some time here to describe them and distinguish between them - you'll be expected to be able to do the same in homework and on your first exam.

The easiest examples of observational studies are surveys. No attempt is made to influence anything - just ask questions and record the responses. By definition,

An observational study measures the characteristics of a population by studying individuals in a sample, but does not attempt to manipulate or influence the variables of interest.

For a good example, try visiting the Pew Research Center . Just click on any article and you'll see an example of an observational study. They just sample a particular group and ask them questions.

In contrast, designed experiments explicitly do attempt to influence results. They try to determine what affect a particular treatment has on an outcome.

A designed experiment applies a treatment to individuals (referred to as experimental units or subjects ) and attempts to isolate the effects of the treatment on a response variable .

For a nice example of a designed experiment, check out this article from National Public Radio about the effect of exercise on fitness.

So let's look at a couple examples.

Visit this link from Science Daily , from July 8th, 2008. It talks about the relationship between Post-Traumatic Stress Disorder (PTSD) and heart disease. After reading the article carefully, try to decide whether it was an observational study or a designed experiment

What was it?

This was a tricky one. It was actually an observational study . The key is that the researchers didn't force the veterans to have PTSD, they simply observed the rate of heart disease for those soldiers who have PTSD and the rate for those who do not.

Visit this link from the Gallup Organization , from June 17th, 2008. It looks at what Americans' top concerns were at that point. Read carefully and think of the how the data were collected. Do you think this was an observational study or a designed experiment? Why?

Think carefully about which you think it was, and just as important - why? When you're ready, click the link below.

If you were thinking that this was an observational study , you were right!The key here is that the individuals sampled were just asked what was important to them. The study didn't try to impose certain conditions on people for a set amount of time and see if those conditions affected their responses.

This last example is regarding the "low-carb" Atkins diet, and how it compares with other diets. Read through this summary of a report in the New England Journal of Medicine and see if you can figure out whether it's an observational study or a designed experiment.

As expected, this was a designed experiment , but do you know why? The key here is they forced individuals to maintain a certain diet, and then compared the participants' health at the end.

Probably the biggest difference between observational studies and designed experiments is the issue of association versus causation . Since observational studies don't control any variables, the results can only be associations . Because variables are controlled in a designed experiment, we can have conclusions of causation .

Look back over the three examples linked above and see if all three reported their results correctly. You'll often find articles in newspapers or online claiming one variable caused a certain response in another, when really all they had was an association from doing an observational study.

The discussion of the differences between observational studies and designed experiments may bring up an interesting question - why are we worried so much about the difference?

We already mentioned the key at the end of the previous page, but it bears repeating here:

Observational studies only allow us to claim association ,not causation .

The primary reason behind this is something called a lurking variable (sometimes also termed a confounding factor , among other similar terms).

A lurking variable is a variable that affects both of the variables of interest, but is either not known or is not acknowledged.

Consider the following example, from The Washington Post:

Coffee may have health benefits and may not pose health risks for many people

By Carolyn Butler Tuesday, December 22, 2009

Of all the relationships in my life, by far the most on-again, off-again has been with coffee: From that initial, tentative dalliance in college to a serious commitment during my first real reporting job to breaking up altogether when I got pregnant, only to fail miserably at quitting my daily latte the second time I was expecting. More recently the relationship has turned into full-blown obsession and, ironically, I often fall asleep at night dreaming of the delicious, satisfying cup of joe that awaits, come morning.

[...] Rest assured: Not only has current research shown that moderate coffee consumption isn't likely to hurt you, it may actually have significant health benefits. "Coffee is generally associated with a less health-conscious lifestyle -- people who don't sleep much, drink coffee, smoke, drink alcohol," explains Rob van Dam, an assistant professor in the departments of nutrition and epidemiology at the Harvard School of Public Health. He points out that early studies failed to account for such issues and thus found a link between drinking coffee and such conditions as heart disease and cancer, a link that has contributed to java's lingering bad rep. "But as more studies have been conducted-- larger and better studies that controlled for healthy lifestyle issues --the totality of efforts suggests that coffee is a good beverage choice."

Source: Washington Post

What is this article telling us? If you look at the parts in bold, you can see that Professor van Dam is describing a lurking variable: lifestyle. In past studies, this variable wasn't accounted for. Researchers in the past saw the relationship between coffee and heart disease, and came to the conclusion that the coffee was causing the heart disease.

But since those were only observational studies, the researchers could only claim an association . In that example, the lifestyle choices of individuals was affecting both their coffee use and other risks leading to heart disease. So "lifestyle" would be an example of a lurking variable in that example.

For more on lurking variables, check out this link from The Math Forum and this one from The Psychology Wiki . Both give further examples and illustrations.

With all the problems of lurking variables, there are many good reasons to do an observational study. For one, a designed experiment may be impractical or even unethical (imagine a designed experiment regarding the risks of smoking). Observational studies also tend to cost much less than designed experiments, and it's often possible to obtain a much larger data set than you would with a designed experiment. Still, it's always important to remember the difference in what we can claim as a result of observational studies versus designed experiments.

Types of Observational Studies

There are three major types of observational studies, and they're listed in your text: cross-sectional studies, case-control studies, and cohort studies.

Cross-sectional Studies

This first type of observational study involves collecting data about individuals at a certain point in time. A researcher concerned about the effect of working with asbestos might compare the cancer rate of those who work with asbestos versus those who do not.

Cross-sectional studies are cheap and easy to do, but they don't give very strong results. In our quick example, we can't be sure that those working with asbestos who don't report cancer won't eventually develop it. This type of study only gives a bit of the picture, so it is rarely used by itself. Researchers tend to use a cross-sectional study to first determine if their might be a link, and then later do another study (like one of the following) to further investigate.

Case-control Studies

Case-control studies are frequently used in the medical community to compare individuals with a particular characteristic (this group is the case )with individuals who do not have that characteristic (this group is the control ). Researchers attempt to select homogeneous groups, so that on average, all other characteristics of the individuals will be similar, with only the characteristic in question differing.

One of the most famous examples of this type of study is the early research on the link between smoking and lung cancer in the United Kingdom by Richard Doll and A. Bradford Hill. In the 1950's, almost 80% of adults in the UK were smokers, and the connection between smoking and lung cancer had not yet been established. Doll and Hill interviewed about 700 lung cancer patients to try to determine a possible cause.

This type of study is retrospective ,because it asks the individuals to look back and describe their habits(regarding smoking, in this case). There are clear weaknesses in a study like this, because it expects individuals to not only have an accurate memory, but also to respond honestly. (Think about a study concerning drug use and cognitive impairment.) Not only that, we discussed previously that such a study may prove association , but it cannot prove causation .

Cohort Studies

A cohort describes a group of individuals, and so a cohort study is one in which a group of individuals is selected to participate in a study. The group is then observed over a period of time to determine if particular characteristics affect a response variable.

Based on their earlier research, Doll and Hill began one of the largest cohort studies in 1951. The study was again regarding the link between smoking and lung cancer. The study began with 34,439 male British doctors, and followed them for over 50 years. Doll and Hill first reported findings in 1954 in the British Medical Journal , and then continued to report their findings periodically afterward. Their last report was in 2004,again published in the British Medical Journal . This last report reflected on 50 years of observational data from the cohort.

This last type of study is called prospective , because it begins with the group and then collects data over time. Cohort studies are definitely the most powerful of the observational studies,particularly with the quantity and quality of data in a study like the previous one.

Let's look at some examples.

A recent article in the BBC News Health section described a study concerning dementia and "mid-life ills". According to the article, researches followed more than 11,000 people over a period of 12-14 years. They found that smoking, diabetes, and high blood pressure were all factors in the onset of dementia.

What type of observational study was this? Cross-sectional, case-control,or cohort?

Because the researchers tracked the 11,000 participants, this is a cohort study .

In 1993, the National Institute of Environmental Health Sciences funded a study in Iowa regarding the possible relationship between radon levels and the incidence of cancer. The study gathered information from 413 participants who had developed lung cancer and compared those results with 614 participants who did not have lung cancer.

What type of study was this?

This study was retrospective - gathering information about the group of interest (those with cancer) and comparing them with a control group(those without cancer). This is an example of a case-control study .

Thought his may seem similar to a cross-sectional study, it differs in that the individuals are "matched" (with cancer vs. without cancer)and the individuals are expected to look back in time and describe their time spent in the home to determine their radon exposure.

In 2004, researchers published an article in the New England Journal of Medicine regarding the relationship between the mental health of soldiers exposed to combat stress. The study collected information from soldiers in four combat infantry units either before their deployment to Iraq or three to four months after their return from combat duty.

Since this was simply a survey given over a short period of time to try to examine the effect of combat duty, this was a cross-sectional study. Unlike the previous example, it did not ask the participants to delve into their history, nor did it explicitly "match" soldiers with a particular characteristic.

<< previous section | next section >>

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Ways of learning: Observational studies versus experiments

• More information: Publisher Index Page (via DOI)
• Download citation as: RIS | Dublin Core

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

1.4 Designed Experiments

Observational studies vs. experiments.

Ignoring anecdotal evidence, there are two primary types of data collection: observational studies and controlled (designed) experiments .  Remember, we typically cannot make claims of causality from observation studies because of the potential presence of confounding factors.  However, making causal conclusions based on experiments is often reasonable by controlling for those factors. Consider the following example:

Suppose you want to investigate the effectiveness of vitamin D in preventing disease. You recruit a group of subjects and ask them if they regularly take vitamin D. You notice that the subjects who take vitamin D exhibit better health on average than those who do not. Does this prove that vitamin D is effective in disease prevention? It does not. There are many differences between the two groups compared in addition to vitamin D consumption. People who take vitamin D regularly often take other steps to improve their health: exercise, diet, other vitamin supplements, choosing not to smoke. Any one of these factors could be influencing health. As described, this study does not necessarily prove that vitamin D is the key to disease prevention.

Experiments ultimately provide evidence to make decisions, so how could we narrow our focus and make claims of causality? In this section, you will learn important aspects of experimental design.

Designed Experiments

The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory variable . The affected variable is called the response variable . In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The different values of the explanatory variable may be called treatments . An experimental unit is a single object or individual to be measured. 

The main principles we want to follow in experimental design are:

Randomization

Replication.

In order to provide evidence that the explanatory variable is indeed causing the changes in the response variable, it is necessary to isolate the explanatory variable. The researcher must design their experiment in such a way that there is only one difference between groups being compared: the planned treatments. This is accomplished by randomization of experimental units to treatment groups. When subjects are assigned treatments randomly, all of the potential lurking variables are spread equally among the groups. At this point the only difference between groups is the one imposed by the researcher. Different outcomes measured in the response variable, therefore, must be a direct result of the different treatments. In this way, an experiment can show an apparent cause-and-effect connection between the explanatory and response variables.

Recall our previous example of investigating the effectiveness of vitamin D in preventing disease. Individuals in our trial could be randomly assigned, perhaps by flipping a coin, into one of two groups:  The control group (no treatment) and the second group receives extra doses of Vitamin D.

The more cases researchers observe, the more accurately they can estimate the effect of the explanatory variable on the response. In a single study, we replicate by collecting a sufficiently large sample. Additionally, a group of scientists may replicate an entire study to verify an earlier finding.  Having individuals experience a treatment more than once, called repeated measures is often helpful as well.

The power of suggestion can have an important influence on the outcome of an experiment. Studies have shown that the expectation of the study participant can be as important as the actual medication. In one study of performance-enhancing drugs, researchers noted:

Results showed that believing one had taken the substance resulted in [ performance ] times almost as fast as those associated with consuming the drug itself. In contrast, taking the drug without knowledge yielded no significant performance increment. [1]

It is often difficult to isolate the effects of the explanatory variable. To counter the power of suggestion, researchers set aside one treatment group as a control group . This group is given a placebo treatment–a treatment that cannot influence the response variable. The control group helps researchers balance the effects of being in an experiment with the effects of the active treatments. Of course, if you are participating in a study and you know that you are receiving a pill which contains no actual medication, then the power of suggestion is no longer a factor. Blinding in a randomized experiment preserves the power of suggestion. When a person involved in a research study is blinded, he does not know who is receiving the active treatment(s) and who is receiving the placebo treatment. A double-blind experiment is one in which both the subjects and the researchers involved with the subjects are blinded.

Randomized experiments are an essential tool in research. The US Food and Drug Administration typically requires that a new drug can only be marketed after two independently conducted randomized trials confirm its safety and efficacy; the European Medicines Agency has a similar policy. Large randomized experiments in medicine have provided the basis for major public health initiatives. In 1954 approximately 750,000 children participated in a randomized study comparing the polio vaccine with a placebo. In the United States, the results of the study quickly led to the widespread and successful use of the vaccine for polio prevention.

How does sleep deprivation affect your ability to drive? A recent study measured the effects on 19 professional drivers. Each driver participated in two experimental sessions: one after normal sleep and one after 27 hours of total sleep deprivation. The treatments were assigned in random order. In each session, performance was measured on a variety of tasks including a driving simulation.

The Smell & Taste Treatment and Research Foundation conducted a study to investigate whether smell can affect learning. Subjects completed mazes multiple times while wearing masks. They completed the pencil and paper mazes three times wearing floral-scented masks, and three times with unscented masks. Participants were assigned at random to wear the floral mask during the first three trials or during the last three trials. For each trial, researchers recorded the time it took to complete the maze and the subject’s impression of the mask’s scent: positive, negative, or neutral.

More Experimental Design

There are many different experimental designs from the most basic, a single treatment and control group, to some very complicated designs.  In an experimental design setting, when working with more than one variable, or treatment, they are often called factors , especially if it is categorical .  The values of factors are are often called levels .  When there are multiple factors, the combinations of each of the levels are called treatment combinations , or interactions.  Some basic ones you may see are:

• Completely randomized
• Block design
• Matched pairs design

Completely Randomized

While very important and an essential research tool, not much explanation is needed for this design.  It involves figuring out how many treatments will be administered and randomly assigning participants to their respective groups.

Block Design 

Researchers sometimes know or suspect that variables, other than the treatment, influence the response. Under these circumstances, they may first group individuals based on this variable into blocks and then randomize cases within each block to the treatment groups. This strategy is often referred to as blocking. For instance, if we are looking at the effect of a drug on heart attacks, we might first split patients in the study into low-risk and high-risk blocks, then randomly assign half the patients from each block to the control group and the other half to the treatment group, as shown in the figure below. This strategy ensures each treatment group has an equal number of low-risk and high-risk patients.

Matched Pairs

A matched pairs design is one where we have very similar individuals (or even the same individual) receiving different two treatments (or treatment vs. control), then comparing their results.  This design is very powerful, however, it can be hard to find many like individuals to match up.  Some common ways of creating a matched pairs design are twin studies, before and after measurements,  pre and post test situations, or crossover studies.  Consider the following example:

In the 2000 Olympics, was the use of a new wetsuit design responsible for an observed increase in swim velocities? In a matched pairs study designed to investigate this question, twelve competitive swimmers swam 1500 meters at maximal speed, once wearing a wetsuit and once wearing a regular swimsuit. The order of wetsuit versus swimsuit was randomized for each of the 12 swimmers. Figure 1.6 shows the average velocity recorded for each swimmer, measured in meters per second (m/s).

Notice in this data, two sets of observations are uniquely paired so that an observation in one set matches an observation in the other; in this case, each swimmer has two measured velocities, one with a wetsuit and one with a swimsuit. A natural measure of the effect of the wetsuit on swim velocity is the difference between the measured maximum velocities (velocity.diff = wet.suit.velocity- swim.suit.velocity).  Even though there are two measurements per individual, using the difference in observations as the variable of interest allows for the problem to be analyzed.

A new windshield treatment claims to repel water more effectively. Ten windshields are tested by simulating rain without the new treatment. The same windshields are then treated, and the experiment is run again.  What experiment design is being implemented here?

A new medicine is said to help improve sleep. Eight subjects are picked at random and given the medicine. The means hours slept for each person were recorded before starting the medication and after. What experiment design is being implemented here?

Image References

Figure 1.5: Kindred Grey (2020). “Block Design.” CC BY-SA 4.0. Retrieved from https://commons.wikimedia.org/wiki/File:Block_Design.png

• McClung, M. Collins, D. “Because I know it will!”: placebo effects of an ergogenic aid on athletic performance. Journal of Sport & Exercise Psychology. 2007 Jun. 29(3):382-94. Web. April 30, 2013. ↵

Data collection where no variables are manipulated

Type of experiment where variables are manipulated; data is collected in a controlled setting

The independent variable in an experiment; the value controlled by researchers

The dependent variable in an experiment; the value that is measured for change at the end of an experiment

Different values or components of the explanatory variable applied in an experiment

Any individual or object to be measured

When an individual goes through a single treatment more than once

A group in a randomized experiment that receives no (or an inactive) treatment but is otherwise managed exactly as the other groups

An inactive treatment that has no real effect on the explanatory variable

Not telling participants which treatment they are receiving

The act of blinding both the subjects of an experiment and the researchers who work with the subjects

Variables in an experiment

Certain values of variables in an experiment

Combinations of levels of variables in an experiment

Dividing participants into treatment groups randomly

Grouping individuals based on a variable into "blocks" and then randomizing cases within each block to the treatment groups

Very similar individuals (or even the same individual) receive two different two treatments (or treatment vs. control) then the difference in results are compared

Significant Statistics Copyright © 2020 by John Morgan Russell, OpenStaxCollege, OpenIntro is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Advanced Search
• Journal List
• Aust Prescr
• v.41(3); 2018 Jun

Observational studies and their utility for practice

Julia fm gilmartin-thomas.

2 Research Department of Practice and Policy, University College London, School of Pharmacy, London

1 Department of Epidemiology and Preventive Medicine, School of Public Health and Preventive Medicine, Monash University, Melbourne

Ingrid Hopper

Randomised controlled clinical trials are the best source of evidence for assessing the efficacy of drugs. Observational studies provide critical descriptive data and information on long-term efficacy and safety that clinical trials cannot provide, at generally much less expense.

Observational studies include case reports and case series, ecological studies, cross-sectional studies, case-control studies and cohort studies. New and ongoing developments in data and analytical technology, such as data linkage and propensity score matching, offer a promising future for observational studies. However, no study design or statistical method can account for confounders and bias in the way that randomised controlled trials can.

Clinical registries are gaining importance as a method to monitor and improve the quality of care in Australia. Although registries are a form of cohort study, clinical trials can be incorporated into them to exploit the routine follow-up of patients to capture relevant outcomes.

Introduction

Observational studies involve the study of participants without any forced change to their circumstances, that is, without any intervention. 1 Although the participants’ behaviour may change under observation, the intent of observational studies is to investigate the ‘natural’ state of risk factors, diseases or outcomes. For drug therapy, a group of people taking the drug can be compared to people not taking the drug.

The main types of observational studies used in health research, their purpose and main strengths and limitations are shown in the Table . 2 - 8

Their purpose may be descriptive, analytical or both.

• Descriptive studies are primarily designed to describe the characteristics of a studied population.
• Analytical studies seek to address cause-and-effect questions.

Case reports and case series

Case reports and case series are strictly speaking not studies. However, they serve a useful role in describing new or notable events in detail. These events often warrant further formal investigation. Examples include reports of unexpected benefits or adverse events, such as a case report describing the use of high-dose quetiapine in treatment-resistant schizophrenia after intolerance to clozapine developed 9 and a case report of a medication error involving lookalike packaging. 10

Ecological studies

Ecological studies are based on analysis of aggregated data at group levels (for example populations), and do not involve data on individuals. These data can be analysed descriptively, but not definitively for causation. Typical examples include studies that examine patterns of drug use over time. One example is the comparison of the use of non-steroidal anti-inflammatory drugs and COX-2 inhibitors in Australia and Canada. 11 Sometimes ecological studies describe associations between drugs and outcomes, such as changes in the rates of upper gastrointestinal haemorrhage after the introduction of COX-2 inhibitors. 12 However, because individual-level data are not presented, causality is at best only implied in ecological studies. The 'ecological fallacy' refers to the error of assuming that associations observed in ecological studies are causal when they are not.

Cross-sectional studies

Cross-sectional studies collect data at a single point in time for each single individual, but the actual data collection may take place over a period of time or on more than one occasion. There is no longitudinal follow-up of individuals. Cross-sectional studies represent the archetypal descriptive study. 1 Typically, they provide a profile of a population of interest, which may be broad, like the Australian Health Survey undertaken intermittently by the Australian Bureau of Statistics, 13 or focused on specific populations, such as older Australians. 14

Case-control studies

Case-control studies focus on determining risk factors for an outcome of interest (such as a disease or a drug’s adverse effect) that has already occurred. 5

• those who already have the outcome (cases)
• those who do not have the outcome (controls), who are often matched to the cases to make them similar and reduce bias.

Second, data on previous exposure to selected risk factors are collected and compared to see if these risk factors are more (or less) common among cases versus controls. Case-control studies are useful for studying the risk factors of rare outcomes, as there is no need to wait for these to occur. Multiple risk factors can be studied, but each case-control study can involve only one outcome. 5 One example explored the relationship between the use of antiplatelet and anticoagulant drugs (risk factor) and the risk of hospitalisation for bleeding (outcome) in older people with a history of stroke. 15 Another case-control study explored the risk factors for the development of flucloxacillin-associated jaundice (outcome). 16

Cohort studies

Cohort studies compare outcomes between or among subgroups of participants defined on the basis of whether or not they are exposed to a particular risk or protective factor (defined as an exposure). They provide information on how these exposures are associated with changes in the risk of particular downstream outcomes. Compared to case-control studies, cohort studies take individuals with exposures and look for outcomes, rather than taking those with outcomes and looking for exposures. Cohort studies are longitudinal, that is they involve follow-up of a cohort of participants over time. This follow-up can be prospective or retrospective. Retrospective cohort studies are those for which follow-up has already occurred. They are typically used to estimate the incidence of outcomes of interest, including the adverse effects of drugs.

Cohort studies provide a higher level of evidence of causality than case-control studies because temporality (the explicit time relationship between exposures and outcomes) is preserved. They also have the advantage of not being limited to a single outcome of interest. Their main disadvantage, compared to case-control studies, has been that longitudinal data are more expensive and time-consuming to collect. However, with the availability of electronic data, it has become easier to collect longitudinal data.

One prospective cohort study explored the relationship between the continuous use of antipsychotic drugs (exposure) and mortality (outcome) and hospitalisation (outcome) in older people. 17 In another older cohort, a retrospective study was used to explore the relationship between long-term treatment adherence (exposure) and hospital readmission (outcome). 18

Observational studies versus randomised controlled trials

Compared to randomised controlled trials, observational studies are relatively quick, inexpensive and easy to undertake. Observational studies can be much larger than randomised controlled trials so they can explore a rare outcome. They can be undertaken when a randomised controlled trial would be unethical. However, observational studies cannot control for bias and confounding to the extent that clinical trials can. Randomisation in clinical trials remains the best way to control for confounding by ensuring that potential confounders (such as age, sex and comorbidities) are evenly matched between the groups being compared. In observational studies, adjustment for potential confounders can be undertaken, but only for a limited number of confounders, and only those that are known. Randomisation in clinical trials also minimises selection bias, while blinding (masking) controls for information bias. Hence, for questions regarding drug efficacy, randomised controlled trials provide the most robust evidence.

New and upcoming developments

New methods of analysis and advances in technology are changing the way observational studies are performed.

Clinical registries

Clinical registries are essentially cohort studies, and are gaining importance as a method to monitor and improve the quality of care. 19 These registries systematically collect a uniform longitudinal dataset to evaluate specific outcomes for a population that is identified by a specific disease, condition or exposure. This allows for the identification of variations in clinical practice 20 and benchmarking across practitioners or institutions. These data can then be used to develop initiatives to improve evidence-based care and patient outcomes. 21

An example of a clinical registry in Australia is the Australian Rheumatology Association Database, 22 which collects data on the biologic disease-modifying antirheumatic drugs used for inflammatory arthritis. Clinical data from treating specialists are combined with patient-reported quality of life data and linked to national databases such as Medicare and the National Death Index. This registry has provided insight into the safety and efficacy of drugs and their effect on quality of life. It was used by the Pharmaceutical Benefits Advisory Committee to assess cost-effectiveness of these drugs. 23

Another example is the Haemostasis Registry. It was used to determine the thromboembolic adverse effects of off-label use of recombinant factor VII. 24

Clinical registries can also be used to undertake clinical trials which are nested within the registry architecture. Patients within a registry are randomised to interventions and comparators of interest. Their outcome data are then collected as part of the routine operation of the registry. The key advantages are convenience, reduced costs and greater representativeness of registry populations as opposed to those of traditional clinical trials.

One of the first registry-based trials was nested within the SWEDEHEART registry. 25 This prospectively examined manual aspiration of thrombus at the time of percutaneous coronary intervention in over 7000 patients. 26 The primary endpoint of all-cause mortality was ascertained through linkage to another Swedish registry. The cost of the trial was estimated to be US$400 000, which was a fraction of the many millions that a randomised controlled trial would have cost.

Propensity score matching

Even without randomising people within cohorts, methods have emerged in recent years that allow for less biased comparisons of two or more subgroups. Propensity score matching is a way to assemble two or more groups for comparison so that they appear like they had been randomised to an intervention or a comparator. 27 In short, the method involves logistic regression analyses to determine the likelihood (propensity) of each person within a cohort being on the intervention, and then matching people who were on the intervention to those who were not on the basis of propensity scores. Outcomes are then compared between the groups. Propensity score analysis of a large cohort of patients with relapsing remitting multiple sclerosis found that natalizumab was superior to interferon beta and glatiramer acetate in terms of improved outcomes. 28

Data technology

Increasing sophistication in techniques for data collection will lead to ongoing improvements in the capacity to undertake observational studies (and also clinical trials). Data linkage already offers a convenient way to capture outcomes, including retrospectively. However, ethical considerations must be taken into account, such as the possibility that informed consent might be required before linking data. Machine learning will soon allow for easy analyses of unstructured text (such as free text entries in an electronic prescription). 29 Patient-reported outcome measures are important and in future will be greatly facilitated by standardised, secure hardware and software platforms that allow for their capture, processing and analyses.

While clinical trials remain the best source of evidence regarding the efficacy of drugs, observational studies provide critical descriptive data. Observational studies can also provide information on long-term efficacy and safety that is usually lacking in clinical trials. New and ongoing developments in data and analytical technology offer a promising future for observational studies in pharmaceutical research.

Conflict of interest: Julia Gilmartin-Thomas is a Dementia research development fellow with the National Health and Medical Research Council (NHMRC) - Australian Research Council (ARC). Ingrid Hopper is supported by an NHMRC Early Career Fellowship.

Inferential Statistics

• Inferential Statistics – Definition, Types, Examples, Formulas

Observational Studies and Experiments

Sample and population.

• Sampling Bias
• Sampling Methods
• Confounding Variables
• Causal Conclusions
• Independent and Paired Samples
• Control and Placebo Groups
• Population Distribution, Sample Distribution and Sampling Distribution
• Central Limit Theorem
• Point Estimates
• Confidence Intervals
• Introduction to Bootstrapping
• Bootstrap Confidence Interval
• Paired Samples
• Impact of Sample Size on Confidence Intervals
• Introduction to Hypothesis Testing
• Writing Hypotheses
• Hypotheses Test Examples
• Randomization Procedures
• Type I and Type II Errors
• P-value Significance Level
• Issues with Multiple Testing
• Confidence Intervals and Hypothesis Testing
• One Sample Proportion
• One Sample Mean & t Distribution
• Inference for Paired Means
• Inference for Two Independent Proportions
• Inference for Two Independent Means
• Introduction to the F Distribution
• One-way ANOVA hypothesis test
• Two-Way ANOVA
• Chi-Square Goodness of Fit Test
• Chi-Square Test of Independence

Observational vs. Experimental Studies

Observational studies and experiments are two common types of research designs used in many fields, including medicine, psychology, and social sciences.

Observational studies are research designs where the researcher observes and records data without intervening or manipulating any variables. The goal of an observational study is to describe and understand relationships between variables or to identify possible causes and effects. Examples of observational studies include case-control studies, cohort studies, and cross-sectional studies.

Experiments , on the other hand, are research designs where the researcher manipulates one or more variables to observe the effect on another variable. The goal of an experiment is to establish cause-and-effect relationships between variables. Examples of experiments include randomized controlled trials, quasi-experiments, and natural experiments.

Content Overview

Observational Studies

In an observational study investigators observe subjects and measure variables of interest without assigning treatments to the subjects. The treatment that each subject receives is determined beyond the control of the investigator.

• In observational study researchers collect data in a way that does not directly interfere how the data arise. It is merely observed.
• From observation study, we can only establish correlation or association between explanatory and response variable.

Different Types of Observational Studies

There are several types of observational studies, including:

Cross-Sectional studies

These studies involve collecting data on a population or group of people at a single point in time. This type of study is useful for understanding the prevalence of a disease or condition in a population.

Cohort Studies

In cohort studies, a group of individuals is followed over time to observe changes in health outcomes. This type of study is useful for investigating the causes of diseases or conditions.

Case-control Studies

These studies compare individuals with a certain health outcome (the cases) to individuals without the health outcome (the controls) to identify potential risk factors for the disease or condition.

Ecological Studies

In ecological studies, data is collected at the population level rather than the individual level. This type of study is useful for investigating environmental or social factors that may impact health outcomes.

Longitudinal Studies

In longitudinal studies, individuals are followed over a period of time to observe changes in health outcomes. This type of study is useful for investigating the natural history of a disease or condition.

Cross-sequential Studies

These studies combine elements of cross-sectional and longitudinal studies by comparing different cohorts at different points in time. This type of study is useful for investigating how factors change over time and how they affect different generations.

Example of Observational Studies

For example, suppose we want to study the effect of smoking on lung capacity in women.

• Find 100 women age 30 of which 50 have been smoking a pack a day for 10 years while the other 50 have been smoke free for 10 years.
• Measure lung capacity for each of the 100 women.
• Analyze, interpret, and draw conclusions from data.

Retrospective Study

If an observational study uses it data from past then it is called retrospective study.

Prospective Study

If data is collected throughout the study then it is called prospective study.

Confounding Variable

A confounding variable is related both to group membership and to the outcome of interest. It is extraneous variable that affect both the explanatory and response variable and that make it seem like there is a relationship between them.  Its presence makes it hard to establish the outcome as being a direct consequence of group membership.

Example of Confounding Variable

Let’s think eating breakfast makes people slim. It means people who eat breakfast regularly are slim. For this case, there might be three explanations.

1. Eating breakfast make people slim.

2. Being slim cause people to eat breakfast.

3. There might be some third variable that might be responsible for both being slim and eating breakfast. Generally, people who are really health conscious they are slim and starts their day with breakfast.

This third variable is called confounding variable.

Experiments

In an experiment investigators apply treatments to experimental units (people, animals, plots of land, etc.) and then proceed to observe the effect of the treatments on the experimental units.

In a randomized experiment investigators control the assignment of treatments to experimental units using a chance mechanism.

Different Types Experimental Studies

Experimental studies can be classified into different types based on the design and the way the participants are assigned to the groups.

Randomized controlled trials (RCTs)

In RCTs, participants are randomly assigned to either a treatment group or a control group. The treatment group receives the intervention being tested, while the control group does not. RCTs are considered the gold standard for evaluating the effectiveness of treatments and interventions.

Quasi-experimental studies

Quasi-experimental studies are similar to RCTs but do not involve randomization. Participants are assigned to groups based on existing characteristics or criteria. Quasi-experimental studies are useful when randomization is not possible or ethical.

Single-blind studies

In single-blind studies, participants are unaware of whether they are in the treatment group or the control group. This helps to reduce bias and ensure that the results are not influenced by participants’ expectations.

Double-blind studies

In double-blind studies, both the participants and the researchers are unaware of which group the participants are assigned to. This helps to further reduce bias and ensure that the results are not influenced by the expectations of either the participants or the researchers.

Crossover studies

In crossover studies, participants receive both the treatment and the control intervention in a random order, with a washout period in between. This type of study is useful for evaluating the immediate effects of an intervention.

Factorial studies

Factorial studies involve testing the effects of multiple interventions at the same time. This type of study is useful for investigating how different interventions interact with each other.

Example of Experimental Studies

In an experiment as researchers randomly assign subjects to various treatments and therefore it establishes causal connection between explanatory and response variable.

• Find 100 women age 20 who do not currently smoke
• Randomly assign 50 of the 100 women to the smoking treatment and the other 50 to the no smoking treatment
• Those in the smoking group smoke a pack a day for 10 years while those in the control group remain smoke free for 10 years.
• Analyze, interpret, and draw conclusions from data

Principle of Experimental Design

In the design of experiments, treatments are applied to experimental units in the treatment group(s).In comparative experiments, members of the complementary group, the control group, receive either no treatment or a standard treatment.

From a statistician’s perspective, an experiment is performed to decide

1. whether the observed differences among the treatments (or sets of experimental conditions) included in the experiment are due only to change, and

2. whether the size of these differences is of practical importance.

Three Principles of Experimental Design

Statistical inference reaches above decisions by comparing the variation in response among those experimental units exposed to the same treatment (experimental error) with that variation among experimental units exposed to different treatments (treatment effect).

Thus, the three principles of experimental design are:

Compare the treatment of interest to a control group to reduce experimental error by making the experiment more efficient.

Randomly assign subject to treatments to ensure that this estimate is statistically valid.

Collect a sufficiently large sample or replicate the entire study to provide an estimate of experimental error.

In summary, observational studies are used to observe and measure variables of interest without manipulating them, while experiments are used to manipulate one or more variables to observe the effect on another variable.

The type of study design chosen should be appropriate for the research question being investigated. Observational studies are useful when it’s not ethical or practical to carry out a randomized controlled trial. However, the results of observational studies can be open to bias and confounding factors. Cohort studies are useful for investigating how a problem develops over time, while case-control studies are efficient when dealing with rare conditions. The randomized controlled trial is still the gold standard for reliable evidence, but there are limitations, including cost, time, and participant restrictions. It’s important to consider the strengths and weaknesses of each study design and choose the most appropriate one to answer the research question.

Observational Study vs Experiments: Difference and Comparison

The collection of data varies in different types and kinds of studies. In some studies, it is done spatially, whereas in others, it is done statically. In some studies, the researcher must conduct experiments to derive a conclusion.

Key Takeaways Observational studies are non-interventional research methods, while experiments are research methods where researchers manipulate variables to observe the effect on the outcome. Observational studies are generally used to explore a phenomenon or association, while experiments are used to test a hypothesis. Observational studies have limited control over confounding variables, while experiments can control confounding variables to a great extent.

Observational Study vs Experiments

In an observational study, study or research is done. There is no experimental or practical work. An observational study can be done on the collected data. The conclusion is based on observation in an observational study. In experiments, practical work is done. In experiments, researchers or scientists can try different studies or methods. 

The study in which observations are counted is called an observational study. The way the observations are noted makes it different from another kind of study called an experimental study.

The study in which more importance is given to conducting experiments is called an experimental study. As the name suggests, the conclusion solely depends upon the data collected primarily after the experiments.

Education Quiz

Test your knowledge about topics related to education

What is the name of the famous Greek philosopher who taught Alexander the Great?

The purpose of the evaluation is to make a judgment about educational...

Who is the author of the famous novel "Pride and Prejudice"?

What is the study of history called?

Which branch of mathematics deals with the study of shapes and sizes of objects?

What is the highest degree that can be earned in a university?

What is the study of languages called?

Which of the following is NOT one of the Seven Wonders of the Ancient World?

Which of the following is NOT a type of writing?

Who wrote the novel "Great Expectations"?

Your score is

Restart quiz

Comparison Table

What is observational study.

In the case of an observational study, it is not at all mandatory for the researcher to do any experiments. Still, he or she needs to make a list of observations and then can arrive at a conclusion. Observational study abstains from conducting any experiments.

Although there are many examples of observational studies, we will determine a relationship between the happening of lung cancer in humans and smoking. Thus, data were collected from those who smoke regularly and those who do not smoke.

What are Experiments?

In the case of experiments, the researcher must conduct experiments and then draw observations from them. In this type of study, manipulation can be done by the researcher in almost every aspect to conclude.

Hawthorne’s studies best set an example for an experimental study. This study was conducted in the Western Electric Company and the Hawthorne plant.

Main Differences Between Observational Study and Experiments

• In Observational Studies, observations are counted. On the other hand, in experiments or experimental studies, more emphasis is given to experiments.
• An observational study differs from an experiment or experimental study in how the observations are taken or noted.
• In an observational study, the person undergoing the study emphasizes making an observation, and the conclusion is drawn from it. On the contrary, more emphasis is on experiments, not just observations in experiments or experimental studies.
• In an Observational study, experiments are not done, and the researcher has to rely on the collected data. On the other hand, in the case of experiments or experimental studies, the researcher has to observe various things through different websites or studies.
• The observational study does not include human intervention. On the other hand, in the case of experiments, human intervention is quite common.
• An example of an observational study includes the relationship between lung cancer and smoking. On the other hand, an example of experiments includes Hawthorne studies.

• https://onlinelibrary.wiley.com/doi/abs/10.1002/0470013192.bsa454
• https://www.metheval.uni-jena.de/projekte/symposium2006/material/cook_shadish_and_wong.pdf

I’ve put so much effort writing this blog post to provide value to you. It’ll be very helpful for me, if you consider sharing it on social media or with your friends/family. SHARING IS ♥️

Emma Smith holds an MA degree in English from Irvine Valley College. She has been a Journalist since 2002, writing articles on the English language, Sports, and Law. Read more about me on her bio page .

Related Articles

• Longitudinal vs Cross Sectional Study: Difference and Comparison
• Study vs Experiment: Difference and Comparison

Share this post!

Want to save this article for later? Click the heart in the bottom right corner to save to your own articles box!

Ads Blocker Detected!!!

We have detected that you are using extensions to block ads. Please support us by disabling these ads blocker.

Friend's Email Address

Your Email Address