I also live in Boston (uh-oh, I've made myself discoverable.) Yes, wealth disparities keep the city more segregated, but I'm surprised that you don't find either the diversity, or the tolerance, you expected. Boston's South End has a large Spanish-speaking population, and though my experiences include racism, I've found the South End to be much more tolerant than other neighborhoods.