Filming for judges' houses took place in late August and early/mid September. On 21 September 2016, it was revealed that there would be a new wildcard twist. The Girls Wildcard was nominated by Louis, the Overs Wildcard was nominated by Nicole, the Groups Wildcard was nominated by Sharon and Simon nominated the Boys Wildcard, making it 7 acts in each category and 28 in total.

Ivy Grace Parades was originally selected for Judges' Houses, but due to visa issues, she was unable to travel to Beverly Hills where Sharon Osbourne would be her mentor. Her replacement was revealed on The Xtra Factor as Honey G, who had originally been eliminated at the Six Chair Challenge.

Summary of judges' houses
Judge Category Location Guest mentor(s) Contestants eliminated Wildcard
Cowell Girls Malibu Mel B and Emma Bunton Soheila Clifford, Olivia Garcia, Kayleigh Marie Morgan, Caitlyn Vanbeck Samantha Lavery
Scherzinger Boys French Riviera Calvin Harris Christian Burrows, Nate Simpson, James Hughes, Niall Sexton Ryan Lawrie
Osbourne Over 25s Beverly Hills Robbie Williams James Wilson, Christopher Peyton, Samantha Atkinson, Janet Grogan Saara Aalto
Walsh Groups Ibiza Alesha Dixon & Fleur East 4 of Diamonds, Skarl3t, Tom and Laura, Yes Lad Yes Lad