The couple, on holiday with their young son Wilfred, were also accused by landowner Kenny Cameron of taking a risk by lighting a bonfire on their getaway on the remote Scottish peninsula of Applecross.
Mr Cameron said Mr Johnson and Ms Symonds set up their bell tent in the highlands field, reportedly believing it belonged to the cottage they were renting for the trip.
He added that he thought they had used wooden chairs from the cottage to scale the fence into the field.
Mr Cameron told the Mail Online: “Mr Johnson is meant to be leading the country and yet he is not setting a great example.
“Usually if people want to go inside a fenced area, they ask for permission first, but I was not asked at all. It is only polite to ask.
“He could have put up his tent in the garden of the cottage and there would have been no problem – but he didn’t do that.
“He could easily have damaged the fence by climbing over it as a short cut. There is a gate a little way up and they could have just used that.
“Having a bonfire is always a risk when you have dry weather. I know we had rain last night but it has been very dry.
“It was the first time I had seen a tent in the field. I last came up here a few days ago and it was not here then.”
Mr Cameron reportedly visited the field on Thursday, having been tipped off by photos in newspapers of Mr Johnson’s tent.
The PM’s staff are said to have cleared away the tent when hearing of the misunderstanding and apologised to Mr Cameron. There was no damage done to the field, the sheep farmer added.
Scottish independence supporter Mr Cameron said he was however pleased that the couple picked the Highlands cottage for their trip.
He added: “It is nice that the Prime Minister came here with his family. This area is very remote. There is plenty of peace and quiet.”
But the couple have cut their holiday short after the location of their cottage was leaked, according to reports.
The Sun reported that the PM’s security guards thought it was unsafe for them to stay in the area.
A source close to the couple reportedly said: “They had been camping in the garden of the cottage but the security threat was too much.
“The tent could be spotted by a sniper from too far away for them to stay after the location was published.”
Mr Johnson has previously urged Brits to stay in the UK for their holidays this year.
He said: “This country is uniquely blessed with fantastic places to holiday, whether coastal or otherwise – and I am certainly going to be doing that.
“Obviously, if people feel the need for a foreign holiday, then that’s completely a matter for them, I totally understand it.
“But there are fantastic, fantastic places, peerless, wonderful, superlative places, in the UK to go on holiday.”
Mr Johnson has been on holiday during a week of torrid headlines for the Government, with pressure on Education Secretary to step down after the A-level results debacle. The PM has not waded into the debate.
Downing Street declined to comment.