Speaking with Pete Timbs, the 42-year-old opened up about the loss of his brother Patty who wandered off from a bush track and never returned.
The unimaginable happened when Pete was just 10, and his brother only nine.
"It's one of those moments when the ground beneath your feet shifts a little bit," Joe said.
"Suddenly all these things that you assumed were certain, all these things that you assumed would last forever, that life would be a certain way, just disappeared."
Joe explains. "The whole family love bush walking, my brother in particular, and we were all on a very short bush walk which is only about 4km and he basically just wandered around a bend and disappeared and we never saw him again. It was really, really surreal and scary.
"In a lot of ways I'm glad I was just a child at the time and couldn't fully comprehend what happened. My mother, I can only imagine the unimaginable grief."
Joe also spoke to Pete about his career and estrangement from his father in the latest episode of Who Are You.