Prince Harry’s legal team could turn to the example Taylor Swift set to help them overturn a legal decision over the royal’s security.

When Swift was performing her record-breaking sell-out shows at Wembley Stadium in August, she was given police escorts to the Eras Tour concerts. That came in the wake of a serious security issue: a terror plot that prompted the cancellation of three of her Eras Tour dates in Austria.

Harry, 40, is locked in a battle over a ruling, made in February, to remove the automatic right to security for him and his family. The initial decision was made in February 2020 when he and wife, Meghan Markle, stepped away from their roles as working royals.

Harry has taken the issue to the courts for more than four years. He lost an initial decision in April, but is appealing it, and the case is set to be heard in the spring of 2025.

A source with knowledge of the situation tells PEOPLE that the prince wants to “ensure the safety of himself and his family while in the U.K. so his children can know his home country” and work with his charities without any fear.

“The Government’s decision to provide Taylor Swift with armed police escorts during her August 2024 Wembley Stadium performances highlight significant inconsistencies in how protection decisions are made by U.K. authorities, raising questions about the transparency and consistency of the process,” the source says.

“Clearly Prince Harry is being treated completely differently to everyone else,” the insider continues.

The source adds that the case “demonstrates the need for a more systematic and transparent approach to protection decisions, ensuring both public safety and equitable application of security resources.”

The development was first reported by The Daily Telegraph on Friday, Nov. 29.

The Metropolitan Police previously said in a statement to PEOPLE: “The Met is operationally independent. Our decision making is based on a thorough assessment of threat, risk and harm and the circumstances of each case.”

“It is our longstanding position that we don’t comment on the specific details of protective security arrangements,” the Metropolitan Police continued.

The situation has exacerbated his estrangement from his father King Charles. When Harry was over in the U.K. in May, he wanted to discuss the issue with Charles, whom he believes has the power to reinstate the security to the former high levels. (Buckingham Palace will not comment on security provisions, but a palace source previously told PEOPLE the notion that Harry’s security is in Charles’ hands is “wholly incorrect.”)

Those close to Harry point out that he served two tours of duty in Afghanistan, becoming a target of terrorist threats as a result, and his family has been subjected to domestic extremist threats. His team argues that his threat level has not changed irrespective of his non-working role within the royal family.

Neil Basu, a former head of counterterrorism for the Metropolitan Police, has spoken out about the fact that Meghan faced serious threats while in the U.K., noting that both Harry and Meghan had among the highest threat assessments within the royal family.

“Harry’s military service, his global recognition and his marriage to a mixed-race woman all contribute to his high threat level,” Basu told PEOPLE in July.

The security downgrade for Harry and Meghan was ultimately introduced by a committee called RAVEC. Its membership includes the Home Office (the U.K. version of Homeland Security), the Metropolitan Police and the Royal Household. Although the King has no direct involvement with RAVEC, his staff sits on the committee. A government spokesperson earlier this year said that they provide “rigorous and proportionate” security and do not disclose details “as doing so could compromise their integrity and affect individuals’ security.”