The unprecedented banishment of Prince Andrew to the outer reaches of the royal firm, without a useful future role or official engagements, free – presumably – to spend more time with his golf clubs, comes at a critical time for the family.

After the calamities of the 1990s and the patient rebuilding of reputation that followed, all seemed set fair only 18 months ago. The Queen was still soldiering stoically on into her 90s, working on her red boxes of official papers every day as she had done for the previous seven decades and attending 293 engagements during the year. Prince Harry had just married Meghan Markle to popular acclaim, seemingly inaugurating a new era for the royal family. And the succession was secure to three generations, if they played their cards right.

How easily destabilising uncertainties can intrude. From the Duke of Edinburgh’s car crash in January, through to Harry and Meghan’s evident unhappiness with their royal role and their fissure with Prince William and Duchess Kate, the great hopes of the royal family, it has been a difficult year, if not yet another annus horribilis like 1992. On top of that came Andrew’s self-inflicted crisis, which has angered other members of the family, especially Prince Charles, reopening old sores and throwing the smooth running of the institution into chaos.

Andrew’s fall from grace has had a slow motion inevitability. His relationship with the paedophile American financier Jeffrey Epstein had been known since the pair were pictured strolling together in Central Park in 2011, but was ignored, especially once a Florida judge ruled that the allegations involving the prince were unnecessary to know. But following the latest complaints about Epstein and his suicide in prison in August, Andrew’s association with him was bound to float into view once more, even though no criminal allegations have been laid against him and he denies any personal wrongdoing.

His ill-judged television interview last weekend precipitated his fall, highlighting as it did well documented flaws: his sense of entitlement, freeloading, obtuseness and insensitivity. The British public can bear a lot from its royal family but resent extravagance and arrogance, especially over the expenditure of money and particularly from someone whose dedication to the common good has been sometimes hard to detect.

The prince’s timing was also at fault: distracting attention from the general election (which royals are not supposed to do) and, from Charles’s point of view, deflecting more positive publicity from the Waleses’ visit to New Zealand – not that that was generating much excitement anyway.

Making matters worse: Andrew did not have the excuse that the interview was sprung on him. It was his decision to take part, evidently without proper briefing, rehearsal or forethought. Perhaps he believed he could wing it, despite his ill-concealed disdain for the media and the warnings of his newly appointed communications adviser Jason Stein, who lasted four weeks before leaving for the City PR firm Finsbury. The future of his private secretary, Amanda Thirsk, who lobbied in favour of the interview as a clear-the-air exercise must now be in doubt.

The royal family operates to a complex grid system, not only in terms of official visits (“whose turn is it to go to the west country? ... when was the last visit to Leicester?”) but also in their association with particular interests: William and Kate with the arts, Camilla, duchess of Cornwall with health and literacy issues.

Andrew’s turf was business. For a time he relished the title of British trade ambassador, giving him scope to travel in some comfort and no little expense by chartered jet to often agreeable parts of the world, a role he continued to carry out until this month, though he lost the title some time ago because of his association with Epstein. If the trade results were nebulous – did foreigners really sign deals because Prince Andrew had turned up at a British embassy reception? – the contacts did him no harm at all. One such is Timur Kulibayev, the son-in-law of the former dictator of Kazakhstan, who bought the Yorks’ derelict former marital home near Ascot for £15m – £3m above the asking price – in 2008.

Years of the media ridiculing the cost of his trips – last year’s best effort was £22,208 for a three-day visit to Hungary – had no effect whatsoever. “The trade role was perfect for him because he felt he was having an impact,” a former palace official told me. “The problem with Yorkie is he can be very arrogant and petulant. I think that’s down to insecurity.”

The writing was really on the wall for Andrew’s future when companies started questioning their sponsorship of his Pitch@Palace initiative, a sort of Dragon’s Den with chandeliers, where young entrepreneurs once a year attend Buckingham Palace and present their ideas to business leaders.

They would scarcely want to be associated with someone tarred by friendship with a notorious sex offender. Andrew also has links to more than 200 charities, 33 of which are involved with education and young people. He won’t immediately shed his honorary military titles, as an admiral (except perhaps as admiral of the Sea Cadet Corps), honorary colonel of several regiments and an air commodore in the RAF, but they won’t like the taint either and he may have to stand down for the good of the services. Talk that Andrew’s purdah will only last a few months is wishful thinking: Epstein’s ghost will always hover behind him now.

Andrew’s absence will leave a hole in the royal grid: who will fulfil the 394 engagements he carried out last year? Who will step in to his role with business, particularly with Harry and Meghan also taking a back seat? The ambition of Prince Charles and some advisers to slim down the working membership of the family to a smaller active core is being achieved by natural wastage, not design.

Age is also having its effect: the 98 year-old Duke of Edinburgh has retired, the Queen herself is slowing down at 93, Charles is 71 and even William is nearly 40: “He’s just another bald bloke in a suit,” one royal photographer lamented. The latest poll this week, YouGov for the Times, was ominous: 54% supporting the sovereign, 20% not and 25% indifferent. The strategy will need a rethink if the royals are, in the Queen’s words, needing to be seen to be believed.

• Stephen Bates is the author of Royalty Inc: Britain’s Best-Known Brand (Aurum)