I'm pretty sure if you take the relief valve out you wont have any brakes at all. That is a 85 psi relief valve to keep the shoes out close to the drums so that you have a good pedal.
1. The relief valve could be clogged up.
2. Pedal linkage adjustment. Set the pedal so that you have to slightly push down on the valve spool to get the clevis pin in the clevis. This keeps pressure in the system from pushing up on the valve spool when you are not applying the brakes.
3. As was mentioned above, since the relief valve applies 85 psi to the wheel cylinders the condition of the brake shoe return springs is very important. The springs keep the 85 psi from moving the shoes out to contact the drum.

This is a tough brake system to troubleshoot... Good Luck!