I've been to ski resorts. Its not a traditional country club elitism, its a you don't have the best skis/equipment snoot.
Certainly, for most folks outside a few geographic areas, its limited to the wealthy.
I just don't see it. After graduating, I got into skiing and did at least a one week vacation every year to go living in WPB with friends. None of us lived in on Palm Beach island or hung at Mar-o-lago. It's how we chose to spend our vacation time and money, but wasn't a crazy expensive elitist thing to do at all.
Hell, the North East isn't a limited geographical area. A huge percentage of the US population lives there, and is within a few hours of skiing. From DC, you can easily drive up to somewhere like (~4 hours) SnowShoe for a long weekend, and it's not a crazy expensive nor elitist thing to do.