The Sussexes were picked up by a Land Rover sent by the Queen after they arrived by private jet from California for the Jubilee celebrations.
It is the first time that Harry and Meghan have been back together in the UK since they left the country over two years ago.
They have been outspoken in their criticism of the Royal Family since making home in the US and talking to local media.
It has led to a strained relationship but the Queen is reported to be keen to see them and she sent a vehicle to pick them up along with her own protection officers, reported The Sun.
It will also be the first time that the Queen gets the chance to meet the Sussexes’ youngest child Lilibet, aged one.
A convoy of cars took the the family from Farnborough Airport to Frogmore Cottage, that used to be their home before they moved abroad.
On the decision by the Queen to send a car for the Sussexes, a Sun insider said: “The Queen believed it was the right thing to do.”
Harry and Meghan won’t be on the Buckingham Palace balcony today after the Trooping of the Colour but they will be involved in other festivities.
Buckingham Palace has confirmed that the Queen will not take part in the Trooping the Colour and the Sussexes are expected to have a prime vantage point as they watch Charles inspect the guardsmen and officers and take their salute.
After the ceremony, the Queen will acknowledge the salute of the returning Household Cavalry Mounted Regiment from Buckingham Palace's balcony, with her cousin the Duke of Kent, Colonel of the Scots Guards, before she later joins working royals on the balcony to watch the traditional fly-past of aircraft.
The Sussexes are expected to join the congregation for Friday's service of thanksgiving for the Queen's 70-year reign at St Paul's Cathedral.
Meanwhile, reports have also suggested that on the Saturday, the couple will finally formally introduce Lilibet to the Queen on her first birthday at a possible party.
On Saturday, it is the Epsom Derby - an event that the Queen often attends, but it has been reported she may no longer be planning on going.
The 96-year-old monarch, who has been suffering mobility problems, is said to want to pace herself during her milestone celebrations, with her daughter Princess Anne representing her at the racecourse instead, The Sunday Times reported.
The Sussexes may also attend the Jubilee Pageant, which is taking place on Sunday - although again this has not been confirmed.
The monarch, 96, has sent a message to the nation and said "many happy memories" will be shared in the next few days.
Seated on a cushioned window seat with the historic residence's famous Round Tower visible in the distance, she looked contented and happy in the photograph.
Events and initiatives have been taking place throughout the year, with Elizabeth having technically started her reign on February 6, 1952.
But they culminate in a long weekend beginning on Thursday which is the anniversary of her being officially crowned.
The Queen's birthday Trooping the Colour by the 1st Battalion Irish Guards will kickstart celebrations at 11am on Thursday, along with the RAF flypast.
