A new study (Ingersoll & May, 2012) offers some surprising data on this issue.
Furthermore, when these teachers do leave a school, they are no more likely to take a non-education job than other teachers: about 8% of "leavers" took another job outside of education. Much more common reasons to leave the classroom were retirement (about 15%) or an education job other than teaching (about 17%).
The authors argue that teacher turnover, not teachers leaving the field, is the engine behind staffing problems for math and science classes.
The authors argue that on this dimension math and science teachers differ. Both groups are, unsurprisingly, motivated by better working conditions and higher salaries, but the former matter more to math teachers, and science teachers care more about the latter.
But in both cases, the result is that math and science teachers tend to leave schools with large percentages of low-incomes kids in order to move to schools with wealthier kids.
Ingersoll, R. M., & May, H. (2012). The magnitude, destinations, and determinants of mathematics and science teacher turnover. Educational Evaluation and Policy Analysis, 34, 435-464.