It’s no secret that Alec Baldwin and his wife, Hilaria Baldwin, have quite the lovely family filled with four kids, but while having a fifth child has been discussed between the couple, the actor isn’t exactly sure he’s up for it.

“Oh yeah, she’s up for the challenge. I’m the one who’s not up for the challenge,” the 60-year-old recently admitted to People during the New York City premiere of Drunk Parents at the Roxy Hotel’s Roxy Cinema. “When my kids graduate school, I’ll be 85.”

“As my friends say, when your kids are climbing out the window and they’re going to do really, really bad things when they get older, when they’re in their late teen years, you’ll be deaf anyway so you won’t have to hear it,” he later joked. “Let’s see what happens.” So is he saying there’s still a chance for baby No. 5 with Hilaria, 35, in the future?

View this post on Instagram

Happy Love Day! ❤️, Us

A post shared by Hilaria Thomas Baldwin (@hilariabaldwin) on

The It’s Complicated star also praised how great of a mother Hilaria is to their young kids. “My wife is the star,” he said. “My kids, they love me, they think I’m great, and as soon as my wife walks in, they’re like, ‘Okay, bye’ and they’re gone.” And that doesn’t make Alec feel too great. “I feel terrible, I feel sad. It’s lonely as soon as my wife walks in,” he joked.

The pair are parents to daughter Carmen, 5, and sons Rafael, 3, Leonardo, 2, and Romeo, 10 months. Alec also shares a 23-year-old daughter named Ireland Baldwin with his ex-wife, Kim Basinger.

The Beetlejuice star also explained just how much work being a mom-of-four is for Hilaria, especially since she seems to be the kids’ favorite. “My wife is the casualty there because no matter how much help we have — and we both have to go to work, so we have some help — they only want their mother,” he told the outlet. “What I always say is I’m doing really well with the kids until my wife comes home and then I’m invisible.”

It seems like the couple already has their hands full, but perhaps another baby is not completely out of the question as long as Hilaria is for it. “I think she wants one more,” Alec said during an appearance on The Ellen DeGeneres Show last month. “She wants to have a girl. She wants Carmen to have a little sister.”

We’ll just have to wait and see!