Joshua Corona

‘Fly-on-the-Wall Studies’ capture randomly assigned or as-if randomly assigned treatment and control conditions and have a few identifying characteristics:

1. They leverage unobtrusive measures of individual-level behavior for maximum psychological and experimental realism and satisfy the assumptions of traditional experiments regarding randomization, measurement, and analysis. 
2. Though treatment assignment mechanisms are known, they are not implemented by the researcher. 
3. The namesake of a fly-on-the-wall study indicates that the researcher(s) are meant to be mostly invisible as observers of social phenomena. 
4. As a mode of discovery, fly-on-the-wall studies draw testable hypotheses from the field site itself. 

There are a few key challenges that scholars must grapple with in order to achieve the causal inference most typically accredited to the best randomized controlled trials in the social sciences. One primary benefit of this method, however, is the ability to capture human behavior in its most natural state, while also allowing scholars to test hypotheses through minimally invasive and ethically sound ways once a protocol is established. 

Scholars attempting to use natural experiments as a method must ensure randomization (Sekhon and Titiunik, 2012). Fly-on-the-wall studies remedy this concern by taking advantage of readily observable behaviors in key arenas of daily social life, establish falsifiable treatment and control conditions, and perform tests for statistical independence between study subjects. Random (or as-if random) social interactions are captured without imposing manipulations or employing confederates/research assistants. Enough observations are collected to satisfy the law of large numbers. The protocol established by the researcher lays the foundation for knowledge creation. Fly-on-the-wall studies test the internal validity of a research design through statistical independence tests between subjects and show balance on observables. This strategy provides a research design with maximum external validity.

Titiunik (2021) indicates that if treatment assignment mechanisms are known, then even when the researcher does not control treatment assignment, well-defined treatment effects are identifiable, inference methods for the analysis of randomized experiments are ensured to be valid, and assumptions are falsifiable. How the treatment is assigned determines the credibility of the causal inference that can be drawn from a natural experiment (Seawright 2016). What matters is that the treatment as a group must be the true causal counterfactual for the control as a group. (Seawright 2016). The treatment group and the control group serve as each other’s counterfactuals, and causal inference at the group level is justified (Seawright, 2016). 

Fly-on-the-wall studies allow researchers examining identities to observe social phenomena through iterative processes of systematic data collection that, when done correctly, are natural experiments because they capture justifiably random encounters in human environments without direct intervention. It is the researcher’s responsibility to find these natural experiments at field sites of interest. Fly-on-the-wall studies are reflexive when they respond to research questions and field settings to formulate testable hypotheses. Importantly, the process of data collection cannot contribute an outstanding treatment effect at the field site. This condition demands high ethical standards from researchers and a commitment to minimal interventions and unobtrusive measures^[1] that are purely observational in nature.

^[1] See: Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. (1966/1999). Unobtrusive measures.