All merger reviews come with uncertainty, but the culture among courts, consultants, and regulators is to pretend that sophisticated modeling can eliminate uncertainty, and thus any uncertainty reflects poor econometric analysis. Creating standards of uncertainty would produce more honest analysis and better competition outcomes, writes Bart Lahcen.
For those familiar with antitrust practice, it is somewhat of an open secret that in a contested merger, those who pay are able to produce the more sophisticated estimates of a merger’s impact, and thus are more likely to persuade the judge. These estimates often come from economic consultants, and the price tag for them can run in the millions. While fees are often not disclosed, ProPublica reported in 2016 that Dennis Carlton, an economist retained by AT&T and working through Compass Lexecon, charged at least $1,350 an hour. The publication estimated that he had amassed roughly $100 million from consulting work.
It is also an open secret that although economic consultants are paid to produce refined estimates, an honest analysis of a merger’s effect on competition often comes in a range bracketed by uncertainty. This is not a radical premise, and yet regulators, courts, and economic consultants continue to operate as if any analysis can and should produce a conclusive number summarizing a merger’s impact on price or output. For the consultants of the defendants, the incentives to produce specific numbers defending the welfare outcome of a merger is obvious. For the regulators refuting the defense of a merger, the incentives of their economic consultants to produce an equally convincing number is likewise clear. Judges must decide between these two figures and their underlying cases. Uncertain rulings are believed to reflect poor acumen rather than reality. Yet, giving a more prominent place to uncertainty would lead to more honest merger review deliberations and, in the aggregate, a more competitive economy.
More than two decades ago, three of the most senior United States antitrust enforcement economists made this argument publicly. In 2004, the then-Director of the Federal Trade Commission’s Bureau of Economics, Luke Froeb, argued in a presentation that economic models are too easily misused, with assumptions that are more restrictive than they appear. Froeb’s examples were concrete: in an upstream merger, whether retail prices rise depends on whether the model assumes downstream retailers pass wholesale cost increases on to consumers or absorb them. The curvature of a demand curve or the chosen demand elasticities can similarly affect the outcomes. Therefore, he argued that assumptions must be supported by evidence or accompanied by a sensitivity analysis that studies how modeled outcomes change as specifications change.
One year earlier, Froeb’s predecessor, David Scheffman, made a similar plea in a presentation at the Conference Board Antitrust Conference. He argued that sound analysis should stand up to plausible testing of assumptions. The concern was put even more bluntly in 2004 by Gregory J. Werden, then senior economic counsel at the Antitrust Division. In a workshop presentation on applying Rule 702, which maintains the requirement and standards for a judge to admit expert testimony in federal courts, he stated that though merger simulations are theoretically sound, their empirical soundness, i.e. their predictive accuracy, is unknown. But what is often omitted, he said, is that the predictive accuracy of every alternative for merger simulation is also unknown.
The verdict from academic literature is harsher still. A study by Christopher Knittel and Konstantinos Metaxoglou showed how modeled post-merger prices shift in a hypothetical merger in the U.S. automobile and cereal industries as assumed values or optimization algorithms change even with the same model and underlying data. It is not generally possible to determine which starting values or algorithms are best equipped to structure the model. Oliver Budzinski has elaborated that for a given merger, it can be theoretically impossible to identify the best of two simulation models because both can fit a case equally well, even if they have incompatible predictions. Economists call this observational equivalence. As a result of observational equivalence, merger review cannot fully prevent parties from advancing economically selective evidence, even though that evidence may be the result of a model no better or worse than other models that produce varied or conflicting evidence.
If it truly is the case that an economic consultant’s estimates are distorted by the incentives of those paying them and ranges better reflect the inherent uncertainty, a key question is where that leaves the judges relying on the provided evidence to approve or reject a merger.
Uncertainty on trial
Many academic scholars and regulators have criticized the economic consulting industry, whose estimates, on their account, underwrote what they describe as decades of permissive merger review running from the 1980s to the 2010s. However, these consultants did not create a standard demanding certainty, though they may have benefited from it. Rather, the demand for certainty was built into the procedure: a regulator carrying a burden of proof can’t discharge it with a range of possible outcomes. An adversarial hearing tends to reward the expert who sounds the least hedged.
The Daubert standard and Rule 702 make judges gatekeepers who decide whether expert testimony rests on sound and reliably applied methodology. However, neither the Daubert standard nor Rule 702 requires certainty and, if applied properly, they would expose the uncertainty inherent to all these models. That may be why economists are challenged so often in this regard. Edoardo Peruzzi found that economists faced more frequent Daubert challenges in antitrust cases than in other legal fields, such as patent and labor law, between 1993 and 2021. His dataset captured 286 Daubert challenges, covering more than 200 expert witnesses across 182 federal cases before 174 judges, with their opinions being excluded in part or in full 36 percent of the time, mostly in private damages actions.
However, the expectation of uncertainty is already prevalent in merger review. A legalist will point out that model specifications affect the weight of a testimony, not their admissibility. There is no need to create an expectation of calculations admitting uncertainty in order to be admitted as evidence when the uncertainty is acknowledged during the trial. Experts and cross-examinations from the opposing side debate calculations and models to demonstrate the extent to which results depend on particular assumptions. The judge then discounts the explanatory power of the models as he or she sees fit. In other words, the legal system already has a way of handling uncertainty.
Yet, that position has proved pragmatically insufficient, which is why the Advisory Committee on Evidence Rules moved against it. Since December 2023, Rule 702 requires the party relying on an expert to show, more likely than not, that the testimony rests on sufficient facts and reliable methods. The Committee’s stated reason was that too many courts have treated exactly these questions as going to weight rather than admissibility. In a fair number of cases, courts admitted expert testimony without the expert’s party first demonstrating, by a preponderance of the evidence, that the reliability requirements were satisfied. This left the court factfinder to evaluate opinions whose foundation had never even been established.
The concern with the status quo is that judges who have been told a number is contested are expected to discount it to the right degree but there is no guidance on how much to discount. Instead, they tend to rely on whichever figures appear most certain. For instance, in the U.S. federal government’s 2018 challenge to AT&T’s purchase of Time Warner, Carl Shapiro’s bargaining model put the harm to consumers at roughly 45 cents per subscriber per month, or $436 million a year. However, Shapiro, the government’s expert witness and a University of California, Berkeley professor, had presented a range, and 45 cents sat near the top. Under cross-examination, he conceded that newer profit-margin data from AT&T, supplied late to the case, dropped the range’s low end from 27 cents to 13 cents. The presiding judge did not settle somewhere in the initial or updated range but instead wondered aloud whether the model’s complexity made it a Rube Goldberg contraption. He then declined to rely on it and credited the defense economist’s finding that three earlier vertical mergers in the industry had produced no statistically significant increase in content prices. The judge approved the merger without conditions, holding that the government had failed to meet its burden of proof. Eight years on, whether Shapiro was right is still not settled, as the only published assessments of his model’s accuracy were written by economists who had testified for the opposing side. Then in 2022, AT&T divested Time Warner, rendering further analysis unavailable. The merger’s eventual failure was not something Shapiro’s model had anticipated.
In the end, the judge interpreted Shapiro’s range not as a statement about an honest analysis of the merger’s impact, but rather treated it as evidence that his data were stale and his estimate inflated. Instead, the judge accepted the defense’s argument that previous vertical mergers had not negatively impacted consumer welfare. Uncertainty may arise during cross-examination, but the convention is to treat it with suspicion, rather than to treat it as a strength of an analysis. Surely, uncertainty can reflect an analysis’ weakness, and this can be uncovered through cross-examination, but uncertainty can also reflect the analysis’ strength. There is always uncertainty, and any analysis should be admitted acknowledging this. Cross-examination can then debate the rigor of the uncertainty.
Upgrading uncertainty
No analysis is possible without assumptions about the values going into models. These assumptions are inherently uncertain. To make this uncertainty visible, the burden should be on the party that is best positioned to analyze and expose that uncertainty, whether due to funding or more sophisticated data. In practice, that usually means the party that developed or ran the model. Instead of putting forward a single number, a table should be the main output. This table should show how far the conclusions travel as the modeling choices move. This would include alternative specifications, such as leading competing demand forms, market definitions proposed by the other side, extended and truncated sample periods, and outliers in and out. This is what has been called sensitivity analysis.
There are two main objections against putting the uncertainty on center stage. The first is cost. Merger reviews run against statutory clocks and a finite staff while sensitivity analysis is not free. The silver lining is that much of the required work often already exists, since the economist who opted for one demand form generally knows what the rejected one produced since he or she also ran it. Not everyone would benefit from this, as small complainants and third parties can be left worse off since they now have a fuller submission to address.
The second one is more subtle and, unfortunately, does not seem to have a clear answer. If the submitting expert chooses the alternative specifications, then the incentive that led to a single skewed value can simply evolve to producing a skewed set of alternatives. Transparency alone does not fix this since a fully disclosed model still reports only the specifications the author chose to run. The key question is who picks the alternatives. Letting the other side propose specifications, as suggested above, closes part of the gap but not all of it. An opponent can only ask for the alternatives it has thought of, and in most reviews there is no opponent with the resources to think of many.
One option would be to have the regulator standardize the set of specifications. For example, the agency could require that any simulation be reported under a fixed menu: results under at least two demand systems rather than one, since the assumed demand form is itself known to drive the predicted price effect; results under alternative assumptions about pass-through; diversion ratios estimated from more than one source, such as survey evidence alongside scanner or switching data; and results with and without the claimed efficiencies. However, this could lead to the standard becoming a lobbying target. Standards are the kind of technical instruments that are easiest to capture, because the technical working groups that make them are usually attended by private-sector parties that can afford to designate staff to take part in their construction.
Two ideas to hold lightly
Where does that leave things? Two procedural ideas can be put forward with an important caveat. First, court-appointed neutral experts could be part of the solution if the standing objection against them, namely that they might produce a third contestable number with unearned uncertainty, is dealt with appropriately. A way to do this is to redefine their role: let the neutral experts instead audit the sensitivity analyses, confirm whether the reported alternatives were run correctly, and that nothing was omitted. This assignment would be narrower and more technical than the original one but whether this would resolve the initial criticism remains to be seen.
Another idea builds on “hot-tubbing” or concurrent evidence, a practice whereby two experts are put in the same room to discuss a case under oath with a judge to narrow what is truly in dispute. But it can entrench positions rather than softening them since experts publicly challenged have reasons to defend their work rather than concede. If instead experts were asked to identify which assumptions drive the gap between results, the exercise would become more tractable. That said, here too it remains an empirical question whether this setup would hold in the face of adversarial incentives.
Uncertainty is not enough
Making uncertainty visible may produce more honest analysis, but that won’t change how merger reviews are decided. In a merger challenge, uncertainty favors the side that doesn’t have to prove anything. Regulators carry the initial burden, and any uncertainty currently weakens their case against merging parties. Hence, if the concern among many antitrust scholars and regulators is that they already have to clear a high bar of proof to persuade a judge, uncertainty would need to count as a reason to doubt a merger, not as a hole in the case against it. That would require a regulatory or legal update, meaning that uncertainty can be a first step in the right direction, but on its own it will not be enough.
Author’s Disclaimer: The views expressed here are his own and do not represent the position of the European Commission or Hasselt University.
Author’s Disclosure: The author reports no conflicts of interest. You can read our disclosure policy here.
Articles represent the opinions of their writers, not necessarily those of the University of Chicago, the Booth School of Business, or its faculty.
Subscribe here for ProMarket’s weekly newsletter, Special Interest, to stay up to date on ProMarket’s coverage of the political economy and other content from the Stigler Center.
