Say what you want about Prince Charles, there’s no denying that seeing the future King of England in the role of grandfather has shown a whole new side to the 68-year-old.

Even Charles admits he melts around his grandchildren, Prince George, 3, and Princess Charlotte, who turns two on May 2.

“[They’ll] look after me when I’m tottering about,” the royal has affectionately mused of Prince William and Kate Middleton’s two little ones.

MORE: Prince Charles Disses Prince William and Kate Middleton In a Major Way!

prince charles prince george getty images

Charles with George and William.

However, according to the Mail On Sunday, Prince Charles doesn’t get to spend as much time with George and Charlotte as a result of his royal duties — which can total around 600 public engagements a year — and the fact Kate’s parents live close by and can help out with babysitting duties.

“William chose the Middletons as his surrogate family long ago, even before the children were born. He has nailed his colors to the mast in that respect,” an insider revealed to the publication of the dynamic.

MORE: Prince Charles “Wept” Before His Wedding to Princess Diana — Get the Details!

“Let’s just say [The Duke and Duchess of Cambridge] haven’t really done anything to make it easy for him. Charles does the best he can, but there are constraints.”

“Does every grandparent want to see more of their grandchildren? Of course they do. But the Prince understands that mothers naturally gravitate towards their own families. And he acknowledges that he is in a unique position which means his time is not often his own,” the source added.

prince george prince charles getty images

Charles with George and Kate.

The report also claims Prince Charles doesn’t want to rock the boat but often gets told about family events at the last minute and can’t attend them due to his royal duties, which are locked in months in advance. That’s not to say Charles hasn’t found his own ways to bond with George and Charlotte.

MORE: Fans Are Speculating Prince Harry Isn’t Prince Charles; Real Son —
Get the Story

In fact, the loving grandfather is passing on his green thumb to the mini-Monarch. Charles revealed he plants trees at his Gloucestershire home with the little Prince, telling Gardeners’ Question Time he hopes George will inherit his passion for gardening.

“I hope, you just never know what people are going to be interested in; the most important thing is that I got him planting a tree or two here.”

This story originally appeared on NowToLove.com

More From Closer Weekly

Prince Harry and Prince William Remember Mother Princess Diana 20 Years After Her Death

Kate Middleton Makes a Surprising Parenting Confession — Find out Here

See the Funniest Quotes From the Royal Family!