International Trade Secretary Liam Fox said it would be “easier” for Boris Johnson to “just give an explanation” about the row recorded at his home.
The MP said reports of the row between Mr Johnson and his partner should not be a “distraction” from policy discussion in the leadership race.
On Saturday, leadership frontrunner Johnson dodged questions on the issue.
Fellow Tory Liz Truss, who backs Mr Johnson for the leadership, said it was a “private matter”.
It comes after the neighbour who called police and recorded a loud row at the home Mr Johnson shares with Carrie Symonds in south London defended his actions.
Tom Penn told the Guardian he was worried about his neighbours’ safety, adding: “I hope that anybody would have done the same thing.”
The Guardian, which obtained the recording, said Ms Symonds, Mr Johnson’s partner, could be heard telling the Tory MP to “get out of my flat”.
Speaking on the BBC’s Andrew Marr Show about the row, Mr Fox – who backs Jeremy Hunt in the leadership race – said: “It’s always easier to just give an explanation.
“The key thing is then how you get on to the issues. What we can’t have is it being a distraction from explanations about wider policy.”
He said it was “fair” for candidates to be asked questions about their character, but added: “I’m not sure what we’ve seen over the last few days is a fair reflection of that.”
Mr Fox also dismissed suggestions that Mr Johnson was a potential security risk.
Recalling Mr Johnson’s previous role in government, he said: “Do you think Theresa May would make him foreign secretary if there were genuine worries about him being a security risk?”
“I think we have to get away from these distractions and talk about policy issues.”
Meanwhile, speaking to John Pienaar on BBC Radio 5 Live, Ms Truss said Mr Johnson had a proven record, so “people know what he’s like in office”.
The Chief Secretary to the Treasury said: “There’s no point asking me. I believe it’s a private matter – I don’t think the public are concerned about that.
“Boris served for eight years as Mayor of London, did a brilliant job; he’s served as foreign secretary – people know what he’s like in office, and that’s what’s important.”