There’s a time and place for everything, but Frank Sinatra didn’t seem to know that when he kissed Harry Connick Jr.‘s wife, Jill Goodacre, on the mouth. The “Whisper Your Name” singer recalled the experience during a Tuesday, November 19, episode of This Morning, and said the whole thing was very “inappropriate.”

“That was an unpleasant experience because as much as I wanted to sing for him and everyone else … I don’t get nervous, but I was really nervous for that and I blanked out,” Harry said. “I completely forget the words. I was with my wife, then my girlfriend, and then we saw Frank at the lift and I said ‘we should go, I need to explain myself.'”

Harry Connick Jr. and wife Jill

“I said ‘I’m sorry I’m much better than what [you] saw’, and he took my wife and held her face and said ‘you’re beautiful,’ and kissed her on the mouth and that’s that,” Harry continued. “I’ve never been able to sing for him again. it was probably unacceptable then, it was a situation where it was an interesting thing, I think my affection for him in the context made it hilarious for her, but obviously it’s completely inappropriate.”

After they got married in 1994, Harry and his wife, who’s also a former Victoria’s Secret model, welcomed three kids —  Georgia, 23, Sarah, 22, and Charlotte, 17. They all get along so well that Harry said they never feel embarrassed by each other.

“You constantly hear stories about kids who get embarrassed by their parents or daughters who get embarrassed by their dads … and I’m not saying this to be cool, but we never really got into that,” he previously explained to People. “It was never really that, ‘Can you drop me off a block away from school because I don’t want to be seen with you?’”

Harry Connick Jr. and family

“It was never like that with us. We never had that, and I don’t know why,” he added. “But we’ve always had a unique relationship and it’s been pretty great.”