Rumours of tension between staff and members of the royal family have been circulating for some time.
In 2017, it was reported that over 14 members of the royal family’s kitchen staff made the decision to leave their jobs.
Recently, the Duchess of Sussex was highly criticised by royal insiders who said she "difficult" and "demanding," after it was rumoured her staff left their duties in tears.
"Nobody gets time off to see their families," a source told The Sun. "They’ve concluded it’s not worth it anymore."
The Duchess of Cambridge's aide reportedly quit to spend more time with partner after they got engaged.
However, the former protection officer says the perks of working for the royal family outweigh the downfalls.
"Work is demanding but rewards are high. The lure of rent-free accommodation for household employees, both during employment and on retirement, is reason enough not to fail," he said.
Buckingham Palace has 188 bedrooms for servants, with more than 1000 royal staff running the household.
"There were difficult times of course, and mistakes invariably made but personally, I never felt burnt out. Staffers, from my experience, were there for life unless dismissed," said Ken - who was first employed during Princess Diana's time.
Ken said a great deal has changed since he was employed by Scotland Yard, stating that previously protection officers and staff had good relationships with the royal family.
"Of course this develops over time and Diana was, without doubt, the moderniser in the palace."
The SAS trained officer also said he was privy to some of the late Princess Diana's innermost secrets, recalling when the Princess of Wales confided in him about Prince Charles’ relationship with Camilla Parker Bowles.
"William was about 3 or 4 months old and Diana joined Charles on a trip to Spain. I remember Charles going out one day on a sailing trip with the King and Diana spoke to me about his relationship with Camilla and hers with James Hewitt."
In the past, Prince Charles would reportedly call and check in on his staff members when they weren’t on duty.
According to Ken, the lack of personal contact may be the reason for the current staff resignations.
"Now, being a royal bodyguard is not a career for life," he says.
"The new style of protection has removed the personal touch, and thus created a ‘them and us’ protection without a professional closeness which in my view was an important ingredient of protection – chemistry. The recent changes and how it is managed has removed decades of experience."
Earlier this year, Meghan's female bodyguard, resigned just weeks after joining the Duke and Duchess of Sussex on their tour of Australia, New Zealand and Fiji.
Though her departure was said to be due to "personal reasons," Ken believes otherwise.
"This may well be the case, however, the pressure placed on her with no personal protection experience, generated negative criticism, notably in Fiji. From my own observation, this lack of personal experience and daily press coverage was perhaps the catalyst that led to her retirement from the post."
WATCH: Lady Colin Campbell says Prince Harry is beguiled by Meghan Markle
However, Ken also denied reports that Meghan was difficult to work for.
"Having worked throughout my career with all members of the royal family, all at times could be painfully difficult to work with. Meghan and Harry are global celebrities and in a way, no different to Diana and Charles throughout the 80s. Meghan, like Diana, does not conform to royalty in a way that Kate does. She appears to be independent, curious with initiative and new ideas."
Just months after Princess Diana’s death in 1997, Inspector Wharfe was honoured by the Queen at Buckingham Palace and made a Member of the Victorian Order, a personal gift of the Sovereign for his loyal service to her family.