Could be some water in the line closest to that side. As the fluid temp. gets hot the water expands and cause the brake to drag slightly. Just an idea. I've heard of it happening before.
2003 Ranger FX4 level II. 3" BL, 33x12.50 GT Radial Mud Claws, Custom front and rear bumpers with winch hookups, custom flatbed, with hi-lift, and full size spare mount, 12,000 lb winch mounted under bed, Electric fan, and of course a flowmaster.