I hate the cold, (if it's less than 70 F, I start to complain) so I don't go to the zoo much in the winter unless we get a warm winter day.
I live in south Texas, where summers get extremely hot. I do go to the zoo a lot in the summer, but I prefer the spring the most. (and to a lesser extent, the fall) Since I'm in college, I'm able to go in the middle of a weekday so I don't have to deal with crowds. YEEEEAH COLLEGE LIFE. My mom hates going to the zoo in the summer, she's organized a trip to New Orleans with the Audubon Zoo & Aquarium as a major piece of it, and she's specifically not doing it in the summer. (granted, she has a much lower heat tolerance than I do)
One time I went to the zoo when it was lightly raining, in the fall. There was practically no one at the zoo, and a lot of the animals were enjoying the rain.