Jen Affleck Felt Like Zac’s Parents ‘Blamed’ Her for Marriage Backlash

Jen and  Zac Affleck were overwhelmed with the opinions people had about their marriage after the first season of The Secret Lives of Mormon Wives.

The reality star couple appeared on The Viall Files to catch fans up on where they’re at since taking a break from the show for Jen’s mental health. In their discussion, the two explained that they were already in a place of struggling when they started filming season 1, which put more pressure on the marriage.

“We went through hell, to be honest,” Zac shared. “Season 1 had a lot of ups and downs, and unfortunately, just kinda how it worked out was it only showed a lot of the negative. And that’s not to say that there wasn’t a lot of things that needed to change in our marriage, but unfortunately, you didn’t get to see some of the other awesome sides of our marriage.”

“I was in the middle of medical school and season 1 comes out and there were so many emotions,” he continued. “I had family and friends reaching out to me, like, ‘Why are you married to her? You should leave her.’ She has the whole world saying, ‘Why are you married to this crazy narcissist? You should leave him.’ We’re just trying to tune out the noise and fight for our family and be like, ‘We know why we got married in the first place,’ trying to get back to those things.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

He added, “We have these two beautiful kids that we want to fight for. Obviously, two healthy houses are better than one toxic one, but we knew that we could work through this. And so I just made the decision that, and I was telling Nick this the other day, that just I decided to step away from medical school. There was no way I was gonna do medical school and support Jen in her career, and so I decided I’m gonna step away from this, and I’m gonna fight for my family, I’m gonna fight for my wife, and try to make the changes I need to, to be the person that she deserves.”

Zac said that the two have tried tuning out the noise in working on their relationship, and through “a lot of therapy, a lot of ups and downs,” they’ve found “some skills and the ability to communicate in a different way.”

“We’re more of a team now,” he said. “But yeah, we were going through a ton season 1, and there was a lot that led up to Vegas that was never shown. There was a lot of resentment on my end and a lot of miscommunication on Jen’s end, and it just kind of led to this massive explosion, which was unfortunately the biggest fight in our marriage.”

“And I’m embarrassed for how I acted, but it is what it is, and we just have to keep moving forward and fighting for each other each day.”

Of course, while Jen and Zac were able to forge ahead, that didn’t change the opinions of those commenting on their relationship, both in the public and closer to home. Jen admitted she “definitely” felt like Zac’s parents “blamed” her for how he came across in the show’s first season.

“I do feel like I could have navigated season 1 better for sure,” she said. “And I could have been better at communicating. And, obviously, you’d see in season 2 that there is lack of communication.”

“I do think I was living in fear a lot of the time, but it was such a hard place to be in after season 1 because part of me felt this guilt for putting my relationship on the show and, like, exposing that and being the one to push for that,” Jen acknowledged. “But at the same time, I was like, ‘Wait a second. I’m not responsible for how Zac acted and his actions.’ So there is a lot of guilt in me feeling like, ‘Oh, I did play a huge part in this,’ and then part of me was like, ‘No, actually, I’m not responsible for this. Your son is responsible for this.'”

“I do feel like I was blamed based off of some comments that were made from what’s shown on social media,” she said. “Everybody has seen it. And, yeah, I know I was definitely blamed, but I definitely think that everyone has learned from this situation, and I’m so grateful because I also feel like his family has been able to take a step back and also see everything.”

Zac acknowledged it was hard to see his family disparage Jen in trying to be “defensive of me.”

“It was more like siblings and cousins and people who actually know me…” he said. “I think they saw, ‘Yeah, Zac made a huge mistake,’ but they also have seen me my whole life, and they’re like, ‘He’s a good kid. He acted immature and he did something really dumb, but that’s not the full context of who he is.'”

He continued, “So they were defensive of me, which made Jen feel unsupported. And then on the flip side, Jen’s parents are the most loving, supportive people ever. So I think where most parents would be like, ‘Yeah, leave that loser,’ they were like, ‘Jen, Zac’s an amazing person.’ They were calling me, they were crying with me. Jen’s parents have been so amazing when it comes to this. So I think that was really hard for Jen, to be like ‘You got that much support from my parents when you were the one who messed up. Why can’t I get the same?'”

In his parents’ defense, Zac said, “I think it was just a really tricky situation to navigate because what do my parents say? ‘Hey! We wanna congratulate you on this show where our son looks like an a——.’ Because that’s that’s what — I mean, that’s how I acted.”

The soon-to-be father of three also said that people forget, “We have our reality TV life, but then we have our real-life relationships with our family and friends, and that is really tough to navigate, to be honest.”

The Secret Lives of Mormon Wives season 2 is now streaming on Hulu and Hulu on Disney+.