It becomes financially/physically difficult impossible after twice. Say you want to be a warehouse supervisor. You graduate high school, get hired, and you pay your dues by working a low paying job while learning the ropes and getting forklift certified. But after five years, you're not getting where you hoped and you're really not satisfied. So you decide you want to be a counselor. You spend $30K getting your bachelor's and then another $37K getting your master's and you get into the field. It pays well but after a couple of years you find yourself unfilled. But now you've got a $700 monthly student loan payment due each month so you can't intern to get a foot in the door anymore and you have to pick something more reliable so your options are more limited. But you decide to be a manager of some retail store. You use your previous experience, get hired as an assistant manager and within six months work yourself into manager.
At this point, if you wanted to completely retool, you'd be pretty screwed. You're thirty four and $67K in debt. There are ways to work around it, but it gets increasingly difficult. Now, to be fair, this is a complete retooling. If you were a nurse, and then you wanted to become a doctor, that would be way easier and something you could do after two screw ups.