if the voltage on one of the batteries is dropping down that much then the low battery is probably going bad or is bad.
run a load test on both after charging, if you still get that big of a voltage drop then replace the battery that is falling off, it most likely has a bad cell causing the voltage drop.
either battery should not drop more than 1.0v during a 15 to 20 sec load test running the hydraulic pump.