The couple sat down together for the second part of Smith’s appearance on Facebook Watch’s Red Table Talk on Monday.
The 50-year-old actor admitted he had been blind to his family’s unhappiness — something he only realised when his 17-year-old daughter Willow shaved her hair off in protest to doing any more music after her hit “Whip My Hair” came out.
“I saw for the first time what Jada had been saying the whole time about hiding behind my ego, my dreams and my desires and pretending like it’s love,” Smith said. “And I realised [Jada] wanted none of the stuff I had built.”
“The house was too big, there was always too many people here, she didn’t want to travel the way we traveled,” he continued. “She didn’t want all of her kids to be getting trolled on Twitter. She didn’t want any of that.”
Pinkett Smith, 47, added, “And there was a decline from there.”
“Yup,” her husband replied. “To have to let go of the picture, to have to let go of the dream was devastating. We essentially had to destroy our marriage.”
“I was reading in the tabloids that we were getting divorced and all of that stuff,” Smith continued.
Pinkett Smith recalled her reaction to the tabloids, saying, “I had never even seen a divorce lawyer! It never even crossed my mind.”
When her mother, Adrienne Banfield-Jones, asked Smith if divorce was something he had thought about, he said, “Because I had been divorced before, I wasn’t getting divorced again. Divorce wasn’t an option.”
Smith was previously married to Sheree Zampino, but the two divorced in 1995. The share one son, Trey Smith, 25. Smith and Pinkett Smith share 20-year-old son Jaden and Willow.
“I was devastated even worse than a divorce. We broke up within our marriage and got back together again,” he continued. “We had to rebuild with new rules and something way, completely different.”
Even though the conversations on Red Table Talk are ultra-personal, Pinkett Smith told PEOPLE earlier in October that she does not find them uncomfortable.
“That’s really how we talk on a daily basis,” Pinkett Smith said. “That’s why it’s not difficult for us. It’s how we communicate.”
“I always leave feeling like I’ve learned something from them,” Banfield-Jones said. “I love coming to their house. I love coming to L.A. and spending time with them because I just leave so filled up.”
This article originally appeared on PEOPLE