Sounds like dropping the main and going under jib only would be a real good idea. Once the rudders kick up, you get a ton of weather helm. Dropping the main and going under jib only will counteract that with a lot of lee helm, so it should help balance things out. This will only really work if you're heading downwind. Going upwind under jib alone would be tough.
Also keep in mind that if the water is shallow, you can always have someone ready to jump off and help stop/turn the boat (assuming you aren't going too fast). If someone jumps off holding the bow (bridle wire), the boat will immediately spin up into the wind and depower.