Leon Rogers, Gerwyn Coleman and Jarred Evans all scored for the Blues with Owen Watkin and replacement James Taylor going over for the Ospreys.
Rogers opened the scoring in just the second minute with fly-half Callum Hall adding the conversion and a penalty to put the Blues 10-0 up.
Scrumhalf Matthew Aubrey kicked a penalty for the Ospreys before hooker Coleman extended the Blues lead with a 41st minute try.
Aubrey kicked another penalty after half-time before Evans added a try of his own. With a 20-6 lead, it looked like being an easy win for the Blues but tries from Watkin and Taylor made sure it was a nervy ending for the home side.