Yes, the backlash is my biggest worry about this idea but I don't think it will be too severe on the children's part since many children segregate themselves anyways* even when going to integrated schools because they can relate to each other and they live in the same neighborhoods.
*
http://www.learntoquestion.com/resou...es/000781.html
I am much more worried about the backlash coming from parents then how the kids grow up.