"Black-balling." You have stated the reason why most whites are afraid to show allyship with blacks, even if they recognize their own prejudices and want to address them. Just as the women you describe are considered "undateable" because they have been identified as being with white men, so many white people are "blackballed" for expressing even friendliness or lack of prejudice toward blacks.

Being shunned by your "tribe" is a frightening thing, and more people will bow to harsh social pressure rather than attend to their own happiness if at comes at the price of isolation. We so often discuss how terrible it is that teenagers adjust their behavior to conform to the norms of their friends when we adults do exactly the same thing. We fail to reflect on how the level of our fears limits our social choices. That's stupid, but that's also life.