The U.S. military is retreating from Afghanistan as the Taliban make rapid gains across the country. It recently slipped away from the key Bagram airbase near Kabul in the dead of night, without informing its Afghan allies. The strange thing about this grim endgame to the forever war is that every aspect of it was predictable from the beginning.

Yet false assumptions and a lack of awareness still fueled a ruinous undertaking that cost innumerable lives and hundreds of billions of dollars, and arguably left Afghanistan worse off than before.

The U.S. and its allies had to respond forcefully to a regime that had directly or indirectly enabled the terrorist atrocity of 9/11. But a military-intelligence operation aimed at the perpetrators of 9/11 and their colluders would have served the demands of both justice and vengeance while sending a message of deterrence to all political players in Afghanistan, better than a full-scale invasion.

Instead, the Bush administration opted for a colossal military and political re-engineering of an entire country. It was supported in this patently hopeless endeavor by a quick and eager consensus across the political and media establishment.

For a few months, all seemed to go well. Those clamoring to oust the Taliban regime felt vindicated when dancing and cheering crowds in Kabul welcomed their Western liberators.

But those of us who had known Afghanistan outside Kabul, and before 9/11, knew that the Taliban themselves had been welcomed as liberators in large parts of the country, especially the south, where the country’s Pashtun majority lives.

The Taliban had, in fact, emerged in the mid-1990s to rid the country of warlords, many of them opium traffickers, and almost all of them specialists in murder, torture and rape. By 2001, many Afghans had tired of the Taliban’s own arbitrary brutality, especially in Kabul, where Soviet-supported communist rulers had expanded educational opportunities and modern social freedoms. Women in the capital city and provinces dominated by ethnic minorities had come to despise the Taliban’s merciless strictures on dress and deportment.

One did not have to invoke the inaccurate cliche of Afghanistan as the graveyard of empires to understand that the Taliban were a resilient and mercurial force, as capable of persuading opponents to switch sides (a common Afghan maneuver) as of disappearing, when necessary, from the battlefield.

They drew their strength from the Pashtun countryside, not to mention from sympathetic Pashtuns and military and intelligence officers in Pakistan, who saw the Taliban as their hedge against Western and Indian influence in Afghanistan.

Almost everyone whose opinion I respected in Pakistan and Afghanistan back in the early 2000s was convinced that the U.S. was doomed to fail. To talk to Western diplomats, military officials and journalists, however, was to encounter a fantasy — that Western military and economic assistance would help remake Afghanistan into a modern democracy.

The Soviet Union and its proxies in Kabul had brutally tried to modernize and centralize a country of many linguistic and ethnic communities living in remote areas on a subsistence economy and with some extremely poor physical and social infrastructure.

Why should the U.S. have succeeded where the communists had failed?

Pankaj Mishra is a Bloomberg Opinion columnist.