Simulated tempering is a sampling method used by practitioners to overcome obstacles that make various simple Markov chains converge slowly. Like simulated annealing, tempering algorithms add a temperature parameter which changes during runtime. The hope is that using this more complicated chain we can sample at low temperature, the distribution of interest, by capitalizing on the rapid convergence at high temperature. First, we show that tempering can require exponential time to converge to equilibrium on a natural example. Even more surprisingly, we further show that it can actually slow down the convergence rate by an exponential factor, contradicting the common wisdom that it can't hurt. We demonstrate this on the mean-field 3-state Potts model, a fundamental model of ferromagnetism.