On August 6, 2001, George W. Bush was given what may be the most infamous daily intelligence brief ever received by a US president. It was titled “Bin Ladin Determined to Strike in US,” and it included details on the activities of al-Qaeda operatives in the US, including threats to hijack US aircraft. In response, Bush did virtually nothing. And then, a little over a month later, those predictions came stunningly true with the 9/11 attacks.

Bush would not be the last leader to ignore such a warning. Decades into the “war on terror,” it’s clear that political leaders, as well as some of the world’s most powerful militaries and intelligence, still underestimate the ability and ambition of extremist militant groups to carry out large-scale attacks. There have now been three major instances of such failures in the past decade.

Multiple news outlets, including the New York Times and Haaretz, have now reported that Israeli intelligence agencies had provided officials with extraordinary details about the plans for what became the October 7 attack more than a year before it was carried out. Just a day before the attack, the CIA reported unusual activity by Hamas in Gaza, suggesting an imminent military operation. Hamas militants reportedly trained for the attack in all but plain sight less than a mile from the Israeli border.

Yet Israeli officials appear to have dismissed these warnings, believing the group had scaled back its military ambitions to the occasional rocket barrage. Instead of reinforcing the border with Gaza, they chose to focus Israel’s military assets on other threats, including Hezbollah in Lebanon and the increasingly restive West Bank. Ultimately, some 1,200 Israelis would pay for this miscalculation with their lives, as would thousands more Palestinian civilians in the war that has followed.

But the October 7 attack was far from the first instance of this kind of strategic surprise from an extremist militant group in recent years. In 2014, the terrorist group then known as the Islamic State in Iraq and Syria (ISIS) began taking over towns in western Iraq. Even after the group had seized the city of Fallujah, the site of the bloodiest battles of the Iraq War, President Barack Obama felt comfortable using a flip sports metaphor to dismiss ISIS as an amateur imitation of al-Qaeda. “If a jayvee team puts on Lakers uniforms that doesn’t make them Kobe Bryant,” he told the New Yorker that January.

Six months later, the “jayvees” had taken over Mosul, Iraq’s largest city, were threatening Baghdad, and had proclaimed the establishment of a “caliphate” that, at its height, would control a territory the size of Great Britain. The US launched a military intervention to defeat the group in Iraq and eventually in Syria as well. US troops remain in both countries today — as does ISIS, which is still active, albeit much diminished.

In August 2021, the world was stunned again as the Taliban marched into Kabul, meeting almost no resistance as an Afghan state propped up with tens of billions of dollars in American funding collapsed and US personnel — along with thousands of Afghans — scrambled to evacuate.

While an eventual Taliban victory was not shocking in itself — the group had been steadily gaining territory in the lead-up to a planned US troop withdrawal — few expected it to happen so rapidly. A month before the withdrawal, President Joe Biden told reporters at the White House that he trusted “the capacity of the Afghan military” and said he believed that the Afghan leaders “clearly have the capacity to sustain the government in place.” He made these statements despite increasingly dire warnings from US intelligence agencies that the Afghan military was likely to collapse. While senior administration officials later said these assessments were made with a low degree of confidence, the reality is that the evacuation of Kabul would descend into chaos, including a bombing that killed 13 US service members and more than 160 Afghans.

These are three very different examples involving very different countries and militant groups. But in all these cases, extremist militant groups demonstrated previously unseen ambition and destructive capabilities. And in all three cases, governments ignored or dismissed warning signs of an impending catastrophe until it was too late.

Analysts and former intelligence agents who spoke with Vox say that a combination of cognitive biases, cultural prejudices, and bureaucratic inertia cause such warnings to be ignored time and again. But just because these problems are well-known does not mean they are easy to address. And given the terrible toll from fighting terror over the past two decades — in lives, dollars, and lost civil liberties, especially for Muslims — the price of success may be just as high as the cost of failure.

The signal and the noise

The problem of surprise isn’t a new one in intelligence-gathering, nor one unique to extremist groups. In 1962, RAND Corporation analyst Roberta Wohlstetter wrote a classic account of the intelligence failure leading up to the Pearl Harbor attack in 1941. It concluded that, though the US military had collected abundant information suggesting an impending attack — including intercepts of decoded Japanese diplomatic cables that indicated preparations for a major rupture in US-Japan relations — they were hampered by diplomatic inertia and a failure to detect the relevant signals within all the noise.

Los Angeles Times via Getty Images

“If our intelligence systems and all our other channels of information failed to produce an accurate image of Japanese intentions and capabilities, it was not for want of the relevant materials,” Wohlstetter wrote. “Never before have we had so complete an intelligence picture of the enemy.”

That scenario — too much information, not enough understanding of which bits are actually important — has occurred over and over again in the history of intelligence failures, from the outbreak of the Korean War to the Tet Offensive in the Vietnam War to 9/11. Erik Dahl, a former naval intelligence officer and a professor at the Naval Postgraduate School, said that the high-tech surveillance tools available to today’s spies have, in some ways, only made the problem worse. According to a 2019 estimate, the National Security Agency intercepted and stored an average of 1.7 billion emails, phone calls, and other communications every day.

“We have too much information and not enough understanding of what’s going on in the world,” Dahl told Vox. As an example, he pointed to the failed 2009 Christmas Day “underwear bombing” attack. A White House review later concluded that intelligence analysts had collected enough data to disrupt the plot, but did not act on it because the information was “fragmentary and embedded in a large volume of other data.” The plot only failed because the bomb failed to detonate.

The problem is compounded when the information counteracts a government’s political preferences. Famously, the George W. Bush administration cherry-picked only the information that supported its preferred narrative when it was building the case for invading Iraq. That was a case of political leaders hearing what they want in order to hype up a nonexistent threat. But political preferences can also cause them to ignore a real one.

Obama had been elected in large part because of his pledge to take troops out of Iraq. Biden had vowed to bring America’s long and frustrating war in Afghanistan to a close. Benjamin Netanyahu’s government had argued that the security threat from Hamas militants in Gaza was contained and the country could focus on other political priorities. Some Israeli officials even believed there was an advantage to having a group like Hamas in power, as it reduced pressure on Israel to negotiate over the establishment of a Palestinian state. In each case, warnings of an impending attack were highly inconvenient for the government’s preferred course of action — and thus, tended to be downplayed.

“Human beings are really, really good at shaping the facts to support their own opinions,” said Emily Harding, a former CIA analyst now at the Center for Strategic and International Studies.

The militant challenge

Religiously motivated, underground, non-state militant groups also pose a particular challenge when state-run intelligence agencies try to analyze their intentions.

“It is difficult for modern Western intelligence services and national security organizations, which are created and established largely to track other entities that look like themselves, to get a handle on the problem of subnation states or non-state actors,” said Dahl.

Shiraz Maher, co-director of the International Centre for the Study of Radicalisation at King’s College London, pointed out that religiously motivated groups often act with a different set of cost-benefit concerns than secular ones. Hamas, for instance, was willing to carry out its attacks despite every indication that it would ultimately result in massive casualties for both its own fighters and Palestinian civilians. (Israel claims to have killed 5,000 of Hamas’s fighters, while estimates are that more than twice as many Palestinian civilians have been killed. Even if the number of Hamas casualties has been exaggerated, it’s safe to say the group’s military losses have been substantial.)

“These are rational groups,” Maher said. “But at the same time, they’re rational from a completely different epistemic, sociological, philosophical, ideological premise.”

Extremist militant groups, by definition underground organizations that emphasize secrecy and demand fierce adherence to their cause, are also notoriously difficult for intelligence services to infiltrate, compared with traditional national governments. They also tend to issue a lot of threats, making it difficult to know which plots they actually intend on, or are capable of, carrying out.

Getty Images

Harding, who served at the CIA during the rise of ISIS, says that because extremist groups don’t look like traditional militaries, it’s easy to believe they lack the ability to carry out grandiose plans. “The intent is always there with these groups. Its capability is what you have to measure,” she said. “They look like a ragtag group of misfits. They don’t look like what we think of as a very capable fighting force.”

Overestimating your allies

In each of these cases, the failure was not only in predicting the actions of the terrorist group, but in the ability of the terrorists’ opponents to meet the threat once it materialized. In Iraq and in Afghanistan, the United States overestimated the ability of the militaries they had spent years and billions of dollars training and arming, and, perhaps even more important, the willingness of those militaries to fight back.

“The psychology and motivation of a military force: That’s the most difficult thing to assess in intelligence,” said Robert Grenier, former director of the CIA’s Counterterrorism Center. “Anybody who tries to make a prediction about that isn’t very likely to be successful.”

Conversely, in the runup to the war in Ukraine, US intelligence agencies gathered and publicized what turned out to be stunningly accurate information about Russian intentions and battle plans, but failed to predict the capability and resolve of Ukrainian forces in meeting that threat.

The October 7 Hamas attacks were different, in part because Hamas is a very different group. In addition to being a militant organization, it’s a governing authority that administers services for more than half a million people and has relations with multiple other states. It’s also engaged in what it views as a nationalist resistance fight, unlike groups such as al-Qaeda or ISIS. Yet Israel was caught off guard by Hamas’s assault, in part because in the months before October 7, Israel relocated military forces away from Gaza, counting on electronic surveillance, autonomous systems, and relatively small groups of troops to deal with whatever threats might emanate from the strip. The Israelis clearly viewed Hamas as a manageable problem, rather than the impending catastrophe it has turned out to be.

“I think it shows there’s clearly a kind of arrogance on the part of the state,” said Aki Peritz, a former CIA counterterrorism analyst now at American University.

An inconvenient threat

In an essay for Foreign Affairs written before the October 7 attacks — but published shortly after — National Security Adviser Jake Sullivan described the Middle East as “quieter than it has been for decades.” It’s a line likely to age about as well as Obama’s junior varsity team quip about ISIS.

Sullivan’s description might be wishful thinking, but the desire to focus on other priorities is hardly irrational. In recent years, leaders of both parties in the United States have argued that, with the threat from groups like al-Qaeda and ISIS vastly diminished, the country should shift the focus of its national security policy to “great power competition” with countries like Russia and China.

But some terrorism analysts say the attack by Hamas, a group largely written off as a serious threat by both Israel and the United States, shows that we may have been premature in dismissing the ability of groups that have threatened major attacks to actually turn those plans into reality.

“I do a lot of work with the US government, including with the intelligence community,” said Colin Clarke, a terrorism analyst with the Soufan Group, a security consultancy. “We’ve talked a lot about ISIS, about al-Qaeda and its respective affiliates, about far-right groups, QAnon, you name it. I hadn’t heard anybody mention the name ‘Hamas’ in about 10 years. It wasn’t on anybody’s radar.”

The October 7 attack, Clarke said, was a “paradigm shift in how we think about the capabilities of these groups. He added: “We’ve kind of written these guys off, because the big names are gone,” referring to globally famous terrorist leaders like Osama Bin Laden and Ayman al-Zawahiri.

Extremist militant groups certainly didn’t disappear with the decimation of al-Qaeda’s senior leadership and the destruction of ISIS’s caliphate. In many parts of the world, they are growing rapidly. West Africa, for instance, saw more than 1,800 terrorist attacks resulting in nearly 4,600 deaths in just the first half of this year. But the prevailing assumption has been that these groups are mainly concerned with local conflicts rather than transnational attacks.

History has shown, however, that local threats don’t always stay local. Al-Qaeda in the Arabian Peninsula, the group’s Yemeni offshoot, was largely thought of as a local group until it very nearly pulled off the failed “underwear bomber” plot of 2009. ISIS, at its height, carried out numerous high-profile and deadly attacks in Europe.

“We have batted zero in terms of correctly predicting when a group decides to shift from local to global,” said Katherine Zimmerman, a terrorism analyst at the American Enterprise Institute.

Going forward, Israel has vowed that its massive military response to October 7 will wipe out Hamas as a military threat, but these groups can almost never be eliminated entirely. And senior officials, including Gen. Charles Q. Brown, America’s new top military commander, have expressed concerns that the scale and brutality of the operation could end up creating more terrorists than it eliminates. Discussing the war in Gaza in a recent CNN interview, Democratic Congressman Seth Moulton cited a study commissioned by retired US Gen. Stanley McChrystal which found that for every civilian killed, about 10 future fighters are recruited.

Whatever soul searching goes on in intelligence agencies in the wake of these failures, the answer is clearly not to go straight back to a September 12, 2001, mindset. Western governments, including the United States, have very good reasons for wanting to turn the page on the war on terror. Over the past two decades, the desire to eliminate terrorist threats has led the US into long, bloody, and frustrating wars and a troubling expansion of the surveillance state and discrimination against Muslim Americans. And it’s worth remembering that for every successful attack or military offensive by an extremist militant group, there are many more that never come to fruition. It’s simply neither possible nor desirable for a society to be on full alert at all times. No one wants to go back to the days of color-coded terror alerts.

But after three significant failures in the past decade to predict a major attack by an extremist militant group, it’s clearly a mistake to assume that these groups are no longer capable of surprising and outwitting the world’s most powerful and technologically advanced states, particularly when, as in the Israeli case, the preparations for that attack are happening in plain view of the state. And there’s no reason to assume it won’t happen again.

“The curse of the intelligence officer is that we constantly get blamed for bringing the doom and gloom,” said Harding. “There’s a piece, I think, of human psychology, where you do not want to believe that something truly bad will happen.” That is one mistake that governments can’t afford to keep making.