Caching failed with panels in legacy mode

Did an deploy on one of the sites that we are working with, we did not get any errors on our staging server, but when we put it out on the production server we really had a big problem with caching. Suddenly cache would choke, display white screens of sadness, just caching part of the page, etc.