Mr Johnson, clearly flustered, said: "All my experience so far is that people want to concentrate on the stuff that really is going to make a difference to their lives.
"They want to hear more about cutting council tax, or bringing down crime, or making London’s air quality better."
He said he was "not convinced" that the public were driven by questions of politicians’ personal lives when going to the polls.
He claimed that his "appetite for power" had been "absolutely glutted" by being Mayor of London.
He added: "Like lots of people, I am intellectually curious and restless, and want to get on with things".
He admitted he has to "dare to be dull" when making appearances in front of audiences expecting his usual gags.
"You turn up at an event and I can see people hungering for the birth of the joke, they are waiting like midwives or staff at an operating theatre for me to produce this thing and nothing comes out and they look absolutely baffled.
"I simply say something completely reasonable. They want the gag and the gag doesn't come because there isn't a gag sometimes."
He added: "The answer to that is sometimes to have the wit to be dull... to dare to be dull."
Asked if he was worried he was seen as "not serious enough for the top job", Mr Johnson said: "I don't care about that. Being Mayor is unbelievably full of difficult executive decisions.
"I really don't have enough time to worry about that kind of thing. I've got almost three more years as Mayor... it's a long time, you can get a lot done in that period and I want to do that."
At last year's Tory Party conference, veteran Conservative minister Ken Clarke said Mr Johnson needed to "settle down" if he had ambitions beyond the mayoralty.
Mr Clarke said: "If he really wants to be a prime minister for serious reasons and not just getting his picture in the paper more often, he really does have to settle down and demonstrate he can seriously deliver on some complicated subjects."