Random Assignment Got Us the Vaccines

But We Need Deeper Evidence to Reduce Violence

As 2020 comes to its welcome end, rescue by vaccine from the COVID-19 pandemic has begun. When Pfizer CEO Dr. Albert Bourla announced the first vaccine to have been proven effective with randomized trials, he described the achievement as “likely the most significant medical advance in the last 100 years.” A second vaccine, this one from Moderna, is about to come online. More vaccines, also shown by randomized trials to be highly effective in combatting the worldwide epidemic, are expected to follow.

Since 1747, when ship surgeon James Lind used a randomized experiment to discover that citrus fruit could prevent thousands of seafaring men from dying of scurvy, urgent problems by the multitudes have been solved by randomized trials. The trick is simple: you find what works by comparing results for half the participants who got the intervention against the randomly selected other half that got the placebo. We have randomized controlled trials (RCTs) to thank for the antibiotic Streptomycin, found in 1948 to be successful in treating tuberculosis. By 1985 randomized tests made it possible to replace most radical mastectomies with less intrusive breast cancer treatments.

It was not long before RCTs moved from medicine to social policy. In the last several decades troubled decision-makers in foundations as well as in the Office of Management and Budget and other federal agencies were seizing on randomized trials as a way of avoiding the messiness of politics and ideology in making decisions ranging from welfare reform to early childhood education. With funding from foundations and federal contracts, a substantial evaluation research industry grew. Web-based clearinghouses featured lists of interventions certified effective by experimental trials. Findings became more certain as they answered narrower questions.

The widespread temptation to rely on RCTs to guide the search for solutions to every sort of urgent problem revealed the fly in the ointment. There were urgent problems which could not be understood with randomized trials. They required more complex methods than RCTs to gather the essential evidence.

One such urgent problem, the subject of deepening and widening concern, is the high incidence of violence found throughout the US. After World War II, violent crime increased dramatically in the United States, nearly quadrupled between 1960, and reached its peak in 1991. Since then, contrary to common misconception, violent crime in the United States has declined fairly steadily, from 1990 (730 per 100,000 of the population) to 367 in 2019. However, when it comes to violence, the US is still an outlier among high income countries. In 2015, the last year for which comparative data are available, the homicide rate in the US was 7.5 times higher than the homicide rate in the other high-income countries combined, and the firearm homicide rate that was 24.9 times higher. Data from the World Health Organization revealed that 83.7 percent of all the world’s firearm deaths, 91.6 percent of women killed by guns, and 96.7 percent of all children aged 0–4 years killed by guns were from the US.

2020 was the year that brought these high rates of violence to an unprecedented level of national attention as they converged with the mishandling of the COVID pandemic, with its disparate damage to communities of color, and the particularly brutal killings of African Americans at the hands of the police. The demand to recognize the damage that violence was wreaking — especially on minority communities — and to find solutions has suddenly become more intense.

Here I explore four strategies to bring down high rates of violence. Despite their promise these strategies have not received the attention they deserve, in part because the experimental, randomized studies that evaluators prioritize are a poor fit with the wider and deeper array of evidence needed to guide effective action.

The four strategies that are promising, if not proven, to bring down high rates of violence would do so by accomplishing the following:

· Strengthening community values, trust, and cohesion;

· Restructuring police functions;

· Creating a supportive welfare state; and

· Focusing deterrence where it will matter the most.

Strengthening community values, trust, and cohesion. A 2013 proposal to reduce violence by strengthening community values deserves particular attention because it is the work of Secretary of the Treasury nominee, Janet Yellen, and her Nobel Prize winning husband, George A. Akerlof. They describe the upsurge in the murder and aggravated assault rates that occurred between the 1960s and early 1990s, and ask whether the response in the form of increased law enforcement activity, with the incarceration rate more than doubling, was the correct response. They contend that the “bricks-and-sticks” approach to crime ignored more promising policy alternatives, particularly the possibility that changing community attitudes toward crime and law enforcement could play a major role in reducing violent crime.

Yellen and Akerlof built an analytic model to explore a range of strategies to deter crime, and concluded that manipulation of social values is as important in the control of crime — and may be more effective — as harsh punishments and high public expenditures for police. They argued that the prevailing approaches to crime control may prove counterproductive in the long run precisely because they risk undermining community values. Instead, Yellen and Akerlof would promote efforts to legitimize the judicial system and improve trust between police and local community members, to spread social programs aimed at strengthening such value-building community institutions as churches and parent support groups, and to promote community grass roots efforts to organize citizen patrols and neighborhood cleanups.

In the years since economists Yellen and Akerlof called attention to the promise of social programs aimed at strengthening value-building community institutions, academics from other disciplines have found further evidence to support their conclusions. Princeton’s Professor of Sociology and Public Affairs, Patrick Sharkey and colleagues point out that the literature on the causes of the increases and reductions in violent crime has largely overlooked how “violence is regulated through informal sources of social control internal to communities.” The National Academy of Sciences 2017 report, Communities in Action: Pathways to Health Equity, cites several such community organizations. The Dudley Street Neighborhood Initiative in Boston is one example, which succeeded in building affordable homes, placed young people in jobs, built community centers and was able to stop outsiders from dumping trash on its streets. Such factors as the level of social cohesion within a neighborhood and the ability of communities to solve common problems may be crucial but tend to be neglected by many researchers because analyzing them and their impacts seems somehow “unscientific” — they are difficult to define and quantify, and the effectiveness of efforts to bring about change in these domains are almost always impossible to subject to randomized experiments.

Sharkey believes that the success achieved by community residents’ organizations in controlling violence by retaking public spaces and creating stronger neighborhoods has not been recognized and built upon because this kind of work is less amenable to stricter forms of evaluation, and because they so frequently operate without enough resources to do their job effectively.

Racial discrimination is the other critical element of building or undermining community values, cohesion and trust. Consider race-restrictive housing covenants, banks discriminating against people of color seeking to borrow money or buy homes, and the strategic siting of interstate highways and public housing that have kept Black and white neighborhoods separate and unequal. These historical inequities continue to be reinforced in the form of gated communities, restrictive land-use regulations, and schools that widen rather than reduce race-based achievement gaps. All of these factors can be counted, but the effectiveness of efforts to change these impediments to cohesion and trusting respectful relationships are extremely difficult to measure — and certainly cannot be assessed through randomized trials.

It would be a lot easier to persuade the public of the benefits of acting on these understandings if it were possible to offer proof. But the proof from randomized trials that vaccines work as intended is not available to legitimate the complex interventions that may be powerful, but are not limited to assessing the impact on individuals. (Randomized trials can’t even prove the efficacy of wearing masks. As Zeynep Tufekci explains in The New York Times, “Randomized trials look at the benefits of the intervention only for the person who is enrolled in the trial. It’s not possible for them to conclude how effectively masks prevent community spread to others — people who are not enrolled in the trial.”)

Restructuring the policing function. A second set of strategies that could reduce violence is based on the reality of how frequently the police are performing functions for which most are neither well suited nor well trained. Police practices in allocating resources, time and energy are only weakly connected to controlling violence and maintaining law and order. The police have somehow become our default mechanism to keep the peace and to address problems of public health, safety and social welfare. The police are expected to keep neighborhoods safe by preventing and responding to criminal activity while simultaneously acting as mediators in domestic and neighborhood disputes. They transport more persons to hospital emergency rooms than ambulances do — in many places because they are better funded to respond 24/7. They respond as best they can to calls to deal with the homeless, the mentally ill, the evicted, and the stressed-out families with hungry children. Not only is there a mismatch between functions and training, but the functions that police are expected to perform are a poor fit with a police culture based on the use of force.

Professor Barry Friedman, founding director of the Policing Project at the New York University School of Law, works at the intersection of law, politics, history, and social change. He urges that we recognize that “Policing imposes serious and extensive harm, from shootings and non-lethal uses of force, to stops, searches, arrests, and incarceration, all of which come with pervasive racial disparities.” Friedman parts from many of his colleagues when he contends that harm-reduction efforts are unlikely to be successful, however, because, he says, “Harm is not collateral to policing, it is innate to it.”

Crimefighting turns out to be only a small part of what police do every day, and their actual work would benefit from an entirely different range of skills, including mediation skills to address conflict, social work skills to help people arrive at the long-term solutions they need, enough clinical skills to judge the extent of a medical emergency, the knowledge and resources to provide victim-assistance, and interviewing and investigative skills to solve crimes. Yet, police are barely trained in any of these skills. They are trained to be warriors. So, it is no surprise that harm is the result. Friedman believes that to reduce the harms of policing, we need to stop sending armed people who see their mission as bringing force to deal with myriad problems not susceptible to this solution. Rather, he says we need to reimagine public safety from the ground up.

We would begin by differentiating functions and reducing altogether the footprint of force and law where it doesn’t belong. We would boost the capacity of health, mental health, family support, housing, and education agencies to respond to needs that don’t require an armed response. Leaders and staff of these agencies, long accustomed to working in impermeable silos, would strengthen their connections to become more effective in offering help. Generalists would be trained to become first responders, equipped with the expertise to handle the first moments of a problem, to direct the right resources and information to the situation, ultimately handing it off to health, social service or — when appropriate — criminal justice agencies.

Patrick Sharkey urges that we look beyond the question of who answer calls for help. We should be asking who is out in public space, making sure everybody is welcomed within that community, making sure everybody is cared for, making sure no one slips through the cracks.

To contemplate a restructuring of police functions that would strengthen and expand the public role in responding to calls for help, would require that we understand that how such a strategy would be implemented will differ markedly from one jurisdiction to another. Together with its complexity and the many formal and informal entities which must be involved, this lack of a standardized intervention model means — once again — that no one can count on experimental methods of figuring out the ultimate impact of this enhanced intervention.

Creating a supportive welfare state. Today’s deliberations among advocates, community leaders, residents, and scholars about reducing violence are more intense, better informed, and go deeper than they have in the past. Increasingly, they don’t stop with the need for change in policing or the judicial system. They extend to structural inequality and to the meagerness of public investment in housing, jobs, health care, education, and public transportation.

The argument is that US communities would experience less violence by shifting resources from surveillance and punishment toward fostering equitable, healthy, cohesive, and safe communities. This argument is another that is supported by extensive data, but not experimental evidence. Much of the data that this argument is based on is comparisons with the experience of other nations.

Our social welfare spending is shockingly low when compared to other industrialized countries. The US rank in social welfare spending as a percentage of GDP is twenty-second among industrialized countries.

Partly as a result of our weak social welfare state, the inequality between rich and poor is greater in the US than it is in any of the G7 nations, and the wealth gap between America’s richest and poorer families more than doubled between 1989 and 2016. Not co-incidentally, our nation has the highest rate of incarceration in the world. With less than 5 percent of the world’s population, the US has almost 25 percent of the world’s total prison population.

David Garland, NYU professor of law and sociology, widely considered one of the world’s experts on crime and punishment, explains that if we are to deal with violent crime and related social problems, we must construct a more extensive welfare state and build more inclusive forms of citizenship.

Garland attributes the ability of other prosperous nations to rely less on incarceration than does the US, to the fact that the social and economic conditions of poor people in other countries are much less harsh and much more supportive of community and family life. He has assembled data to show that where there is greater equity and inclusivity, there is less need for penal control. He has been able to show that where welfare states function to ensure that families, schools, public health, affordable housing and employment are supported and social exclusion combatted, it is possible to deal with problems like homelessness, mental illness, drug addiction and prisoner re-entry in a “non-penal manner.” At bottom, he says, because we have such a weak safety net, “American criminal justice is charged with tasks that other nations allocate to social service agencies.”

Michelle Alexander, author of The New Jim Crow: Mass Incarceration in the Age of Colorblindness, sounds a note of optimism, when she suggests that today’s newly energized protests for racial justice may provide the setting in which it will be possible to design and implement a long-overdue paradigm shift. We may be on the verge of adopting alternative approaches to race and criminal justice, poverty, drug abuse, mental illness, trauma and violence that would do less harm than police, prisons, jails, and lifelong criminal records.

This strategy stands out not only because of the hope it would bring to many who have almost given up hoping, but also because the evidence structure on which it is based is an understanding of how other societies are dealing more successfully than we are with challenges that we have in common.

Focusing deterrence on where it matters. The last of the four promising strategies to bring down high rates of violence has come to be known as “focused deterrence.” It’s a strategy that got its start from the designers of a new approach to violence who knew that the evidence of its success, should it come, would likely be dismissed. It took more than a decade of perseverance for its triumphs to be recognized and widely learned from.

Boston’s Operation Ceasefire was the creation of a handful of adventurous scholars from the Harvard Kennedy School, a few brave Boston police, and some persevering community leaders who together had identified a serious problem (a horrifying rise in youth gunfire deaths occurring in Boston beginning in the late 1980s). The working group set out to solve the problem by gathering and learning from a wide range of diverse research and experience. Their grant proposal to the National Institute of Justice (NIJ) was initially turned down for its “lack of a powerful experimental design,” but was ultimately funded. It turned out that Jeremy Travis, then director of the NIJ, made the decision because he was intrigued by the Harvard partnership with the Boston Police Department, the focus on a critical issue, and the application of a new and potentially useful analytic framework.

In their search for new ideas the group talked and listened to the police, street workers, gang members, and anyone else who could provide useful insights. They moved continuously between data analysis and street-level inquiry, each investigation informing, enriching and redirecting the others.

In his book describing the work, Don’t Shoot: One Man, A Street Fellowship, and the End of Violence in Inner City America, David Kennedy writes of their astonishment when they realized how much information the police already had. “They knew the kids who were getting killed, and almost always who did it. They were able to identify Boston’s youth violence problem: a few score groups, already well known, made up of shockingly active offenders, many of them already well known, hurting each other.”

As the pieces of the puzzle came into focus, the team went to work on crafting a strategy that they hypothesized would directly reduce youth homicide in Boston. The small numbers of highly active offenders would receive a single message of targeted deterrence, with heavy enforcement announced ahead of time, face-to face. The signal sent, as described by Kennedy, would be direct, concrete, literal, rapid and explicit: If you do this (engage in gun violence), we will do that (throw the book at you — applying sanctions from a full menu of possible law enforcement actions). The deterrence, which became known as “pulling levers,” would be aimed not at individuals, but at the group. It was a clear message: Put your guns down! If anyone in the gang shoots, you’re all in trouble.

The threats would be coupled with offers of help: Streetworkers (a coalition of Boston social service workers), probation and parole officers, churches, and other community groups would offer gang members the services and supports they would need to change their lives.

To send the message, Kennedy and colleagues would assemble targeted gang members and identified troublemakers in group forums which they called “call-ins.” To their surprise, they obtained the blessing and participation of the Boston police command structure.

At the first call-in on May 15, 1996, the officials in the front of the room faced an audience that was mostly looking at the floor, looking at the ceiling, looking at nothing, slumped and their legs spread. But slowly, they began to listen. Gradually, most of the attitude was gone. By the time Streetworker Tracy Litthcut spoke, they were leaning forward, focused, paying attention, taking it in. “We’re tired of Black kids dying, Black kids killing each other,” he said. “It makes us sick. We know you’re hurting. We know you’re caught up in something you can’t control. We know it’s dangerous out there. And we’ll help in any way we can. If you need protection from your enemies, if you want a job, if your mom needs treatment, if you want back into school, tell us — here’s my phone number. But the violence has to stop.”

It was after this call-in that Boston began to go quiet. Kennedy reported his big insight: The gangs were rational. They listened, they learned, and they changed — in response to both formal and informal sanctions.

NIJ statisticians documented the impact of Operation Ceasefire between 1996 and 1999:

· A 63 percent decrease in monthly youth homicides;

· A 32 percent decrease in monthly shots-fired calls;

· A 25 percent decrease in monthly citywide gun assault incidents, and

· A 44 percent decrease in monthly youth gun assault incidents in a particularly chaotic district.

The streets essentially remained quiet through 1999. It may be that the working group’s largest aspiration — to create a “firebreak” across the cycle of youth violence — was actually attained.

But it didn’t last. Key police personnel changed. The collaborations necessary to legitimize, fund, equip, and operate the complex strategies most likely to succeed were difficult to sustain amid changes in priorities. Support from the Black churches ended as key ministers were plagued by rivalries in claiming credit for the changes.

Almost from the beginning, and well before there was any indication that the sharp reduction in Boston’s youth homicides would end after four years, Boston Ceasefire became famous as the Boston Miracle. Kennedy said he hated that name, because “it wasn’t a miracle, it was hard damned work.”

After the New York Times described and lauded the Boston intervention in July of 1996, journalists and other observers focused on the question of what, exactly, was the winning strategy, with its many moving parts. Kennedy complained that most descriptions ignored their core insight, that they had put a full range of moving parts together and fuse them in an entirely new way.

The hardest thing to convey was that Boston Ceasefire was not a program but a theory of deterrence; it was a theory of human behavior. The theory could be applied to a range of problems. Its guiding principles were very transportable — but had to be mapped onto unique settings. Kennedy says “You don’t patent this kind of work.”

My own conclusion was that the pieces that together made up Boston Ceasefire do not constitute a replicable model, but do contain a set of principles and approaches that are daily informing some of America’s most effective efforts to reduce youth violence.

Over the ensuing two decades, Kennedy and his colleagues responded to invitations to work with communities in every part of the U.S. In 2009, to capitalize on the lessons learned, Jeremy Travis and David Kennedy joined to found the National Network for Safe Communities (NNSC). A project of the John Jay College of Justice, NNSC supports communities implementing strategic interventions to reduce violence and community disorder, minimize arrests and incarceration, enhance police legitimacy, and rebuild relationships between law enforcement and distressed communities.

The reach of the Network continues to expand, while additional cities are implementing targeted deterrence on their own. The Network serves as a clearinghouse for lessons learned throughout the country. These lessons prompted Jeremy Travis to tell a 2008 Senate Judiciary Committee hearing that enough was now known about what it takes “to achieve four ambitious goals: to simultaneously reduce violence, abate drug markets, reduce our reliance on incarceration and promote better relationships between the police and minority communities.”

The Network’s strategies have been deployed with powerful impact in more than 70 cities over nearly two decades. Nevertheless, there has been a vigorous debate in the academic literature about whether the claimed results are real. Malcolm K. Sparrow, professor of the Practice of Public Management at Harvard’s Kennedy School of Government, is concerned that the absence of Boston Ceasefire’s formal experimental structure may leave the Evidence Based Practice community wondering whether Operation Ceasefire really worked, or whether its success was some kind of fluke. He worries that Ceasefire’s work will be devalued “just because this partnership used nothing from the evaluation toolbox of elite science.“

In a 2009 article about Boston Ceasefire for The New Yorker, author John Seabrook reported that Franklin Zimring, professor of law at the University of California at Berkeley and a leading criminologist, told him “We just don’t have the evidence,” to assess how effective Ceasefire really is. He said that “It’s odd that no one has ever said, ‘O.K., here are the youths who were not part of the Ceasefire program in Boston, let’s compare them to the youths who were. And no one has followed up with any long-range studies of the criminal behavior of the group that was in the program, either.”

When Seabrook asked Kennedy about Zimring’s comments, Kennedy said “Frank still doesn’t get it. There’s plenty of research, but it’s not focused on the impact on the people in the call-ins, because the strategy isn’t just about the people in the room.” He explained, “When you have a couple of meetings and homicide city-wide goes down 40 percent, it’s not because the 40 guys you’ve talked to have turned their lives around. There are a thousand guys on the street you haven’t talked to. But the 40 get the word out to the thousand — which ruins them as controls for the kind of evaluation that Frank’s talking about.”

But word of the successes being achieved by targeted deterrence got around — despite the lack of the evaluation tools of elite science.

In September 2018, John Seabrook noted in The New Yorker that Kennedy’s work had “led to a paradigm shift in urban law enforcement.”

By 2019 Thomas Abt’s important book, Bleeding Out, couldn’t have been clearer: “In the United States, nothing works as well to reduce urban violence as focused deterrence. It does not work perfectly, it does not work every time, but it works better, on average, than anything else out there.” And Abt was able to quote from a 2018 National Academy of Science’s report, “Focused deterrence programs show consistent crime-control impacts in reducing gang violence, street crime driven by drug markets, and repeat individual offending.” The footnote to the Academy report was quite remarkable: “While there have been no randomized experiments, and only a few of the quasi-experimental designs are rigorous, the programs from the stronger (as well as the weaker) designs show consistent outcomes.”

We have seen that each of the four strategies explored here are supported by evidence that makes them highly promising — without being proven. That’s not because the work to prove them effective has not been done. It’s because they all involve complex interventions that are targeted at communities and groups rather than individuals, and prevailing methods of mobilizing the evidence of their promise that would be persuasive to skeptics, are still underdeveloped.

The enormous current interest in identifying and implementing the strategies that could reduce violent crime in the US should provide the impetus for a major thrust to develop and use these deeper approaches to documenting “what works” among complex interventions.