Beyond Checking A Box: A Lack Of Authentically Inclusive Representation Has Costs At The Box Office

Death On The Nile - 20th Century Studios

As fans of cinema, that title may feel like an understatement at times. When characters collide with our minds like comets, impacting the constructs nestled deep inside them. Hero, immigrant, etc. - Fandom seems an inadequate qualifier in these instances. A central belief of the Center for Scholars & Storytellers at UCLA is that “The power of stories to shape the lives and identities of young people in meaningful ways is real.” We think harder about a character's decisions and share more profoundly in their experience if we can identify with them, making movies an important part of understanding our world and triangulating our place within it. This institution values representation in film, and is conducted “to further explore whether authentically inclusive representation (AIR) in character portrayals and storylines can affect the success of a film (financially and critically), particularly when considering factors like production budget”. CS&S, in partnership with the Creative Artists Agency (CAA) and the Full Story Initiative (FSI), hopes to plant a flag where all production studios can see it, by annotating how much it costs to forgo diversity. Contributing author Dr. Yalda T Uhls tells A Hot Set “We’ve confirmed - you have to make content that feels authentic and inclusive if you want to make money.”

You can defer to the paper itself for details on the methodology, but essentially, Lee Lazar, Gerald D. Higginbotham, Jenna Signorelli, Christy Wang, Jamie Azar, and Dr. Uhls have identified a positive correlation between films that are generally more authentically inclusive in their representation of people of color, and the subsequent performance of those films at the box office. The question that we’re all responsible for asking, as enthusiasts of scientific rigor, is whether or not this is a true causal relationship. If the answer is undeniable yes, then the money-motivated Hollywood machine will need to recalibrate and begin representing these groups until they’ve claimed the profit that is being left on the table. But, what does ‘undeniably’ look like by the standards of the scientific community? Perhaps it is useful to compare this year’s publication to the study’s previous version, published in 2020. I remember feeling disheartened after reading the first one, quickly recognizing in the methodology the kinds of flaws that truly erode the integrity of findings. After reaching out to CS&S for an interview about the research, they sent me the updated version. I was eager to read; were any methodological improvements made? Do they firm up their discoveries as, indeed, undeniable?

Firstly, when analyzing the data, we’re responsible for wondering if the source of these diversity quotients is an arbitrary one. In the 1st version, it is completely arbitrary. I was prepared to accept the argument that it needed to be; is there really any way to account for authentically inclusive diversity using data points, like screen time or a number of lines? In an interview with A Hot Set, Dr. Uhls points out that the second time around, “the methodology for measuring AIR is different.” The report sums up this method in the following way: “An AIR-rated film is a film that was tagged by machine learning as having a storyline relevant to marginalized communities, further confirmed as a ‘relevant’ film by FSI, and then passed through an additional qualifier.” Determining the AIR coefficient for each film is clearly a keystone in research like this, and CS&S treats it that way. They involved several groups that they feel can sufficiently assess a film on behalf of those within this group’s protected class. The 1IN4 Coalition, for instance, weighed in on the People with Disabilities component of this AIR score, just as Color of Change was the guarantor of Black and African American representation, and the list goes on. A completely objective AIR score will always elude researchers, but as long as it needs to be subjective, this strikes me as the strongest approach to assigning reliably accurate scores. The only way to consult a higher authority on the matter would be by appealing to the individual, and polling communities in hopes of collecting aggregate scores would open the door to a host of other complications.

There is one question, however, that even the second version failed to answer definitively, and it’s an important one. Findings #1 and #2 make the claim that films with higher AIR ratings were both more successful at the box office and by the lights of audiences/critics (referencing Rotten Tomatoes scores, among others). Since the only variable that is being controlled for is budget, the research did leave some room for skepticism. After all, it feels plausible that films performing better at the box office may have done so because they were, in the eyes of audiences and critics, better films. It’s reasonable to assume that the merit of a film might impact its opening weekend numbers. It would be intellectually dishonest to pretend that this doesn’t call into question the causality implied by the study. When asked about teasing out the ‘merit’ variable, Dr. Uhls says it’s “kind of impossible. But the reality is you’ll just make a better movie if it feels authentically inclusive.” To be clear, the causal relationship in which AIR is responsible for box office success may be very strong. Unfortunately, it hasn’t been precisely determined. Additionally, I feel as though the 136 films examined and their respective AIR scores should be listed in this report. Disclosing the sample seems the bare minimum for being forthcoming about the ‘how’ of any claim. A trustworthy methodology is a transparent one. Besides, in light of these data, shouldn’t we be annotating both good and bad examples of representation as such, so that executives and filmmakers alike have an understanding about what to strive for, and what to avoid?

The most influential version of this study is the version where these breaches are shored up. If we encourage scientific rigor because the findings it yields are impossible to ignore. Nonetheless, this report boasts a strong case that time has only strengthened. You can encourage this mission by sharing the study, and if you like the work done by the Center for Scholars & Storytellers, you can donate here. In the meantime, we’ll see if this is a wind strong enough to turn the mighty sails of Hollywood.