You are rightit's the heater core and you can bypass the heater core with the two hoses. My buddy's car's heater core has been bypassed for a couple of years at least. No problems wit hit whatsoever. Good Luck with it!
2004 Ford Ranger, 2WD, 5 Speed, 2.3 4 cylinder, stock.