Ulster, unbeaten in 2014 and Heineken Cup quarter-finalists next weekend, went ahead with a try from Darren Cave after four minutes.
But Blues then produced one of the most impressive 20-minute spells of the season as the forwards went to work in destructive fashion.
A brilliant try from back row Robin Copeland, on his 50th appearance for the region, capped it as the home side roared into a 22-5 lead at the half-time whistle.
Back came Ulster and the boot of Paddy Jackson ate into the lead but a late penalty from Gareth Davies and brilliant defensive effort in the dying seconds saw Blues hold out to spark wild scenes of celebrations on the final whistle.
Blues went on the attack early on with Isaia Tuifua and Alex Cuthbert busy in the opening exchanges before Copeland broke the line and fed a marauding Gethin Jenkins on his shoulder.
Spilled ball though saw Ulster attack from deep at pace and when they ran some strong lines in midfield it was strong running centre Cave who handed off his man and confidently burst over to score in the corner.
More slick handling from Ulster saw them flick a pass to Ireland star Tommy Bowe who kicked in behind into space. Scrum half Lewis Jones made the footrace to the loose ball, but kicked it off before it ran dead to hand the Irish province a five-metre scrum.
The scrum was solid and Ulster picked and went around the base, but after coming up close they lost control just inches from the line and knocked on to relief of the home faithful.
A penalty for not rolling away from the tackle gave Blues a way back into the game on 13 minutes and Gareth Davies made no mistake as he drilled over the three points from the right.
A sliced clearing kick from full back Jared Payne gave Blues more field position. However, the full back more than made up for it a minute later when he picked up a grubber from Davies and tore a direct line down the middle before forcing the home side to run it out.
Cory Allen and Jones looked to spark Blues as they found holes of their own with direct running lines while Czekaj was confident under the high ball as Payne looked to drill ball into the corners.
A powerful driving lineout made big gains before Wales international Cuthbert cut a superb line and was only a tap tackle away from racing away under the posts as he cut inside.
Blues were looking much stronger in defence than last week and some thundering hits from Josh Navidi and Lou Reed rocked Ulster back before a penalty was awarded to the hosts to huge cheers.
Home No.9 Lewis Jones was impressing with a typically lively show as he disrupted an Ulster scrum and left Michael Heaney on his back as he tore ball away to Copeland. Blues immediately took advantage with a penalty from Davies from the right touchline.
Navidi produced a barnstorming blast down the middle as he rolled out of three tackles before Filo Paulo unloaded on Heaney who continued to be in the wars.
Ulster’s penalty count was mounting and Italian referee Marius Mitrea lost his patience with visiting lock Dan Tuohy who was sin binned and Davies sweetly struck the resulting penalty home.
A monster driving lineout had the Arms Park on its feet as the pack drove Ulster back 30-metres with an astonishing show of power. The deserved resulting kick from Davies put Blues 12-5 ahead.
Blues were now on fire and a quite brilliant score from halfway saw Cuthbert break away down the right and time a sublime pass back inside to Copeland who showed his pace and power to beat the retreating Bowe and touch down to a huge roar.
More brilliant work on the floor secured another penalty and a stunning ten-minute spell for Blues was finished by another Davies kick as the home side went 22-5 ahead at the interval.
Ulster needed a second half revival and they looked to kick-start that after 44 minutes when a penalty at scrum time saw Paddy Jackson slot his first kick of the night to eat into the lead.
Three minutes later the home side looked to play from deep inside their own half and paid the ultimate penalty as the ball was spilled and another kick from Jackson reduced the lead once more.
A superb tackle and counter ruck from Kristian Dacey from the restart secured vital ball for Blues as Paulo and Reed looked to bulldoze down the right. But Ulster held firm and a hammer like kick from Payne forced the home side back.
Some smart running and quick hands found space for Trimble who kicked in behind and collected but could not find Payne on his right shoulder for what would have been a great try.
Nick Williams crunched past men to put Ulster on the back foot, but a superb tackle from Copeland wrapped up a potentially pass for a try and then visiting Number Eight Williams was forced back by a no-nonsense hit from Jenkins.
Blues made a mess of clearing from a scrum to hand ball back to Ulster and the visitors were not as forgiving and quick ball released Luke Marshall who thought he was over the line only to be held up by Navidi. But the penalty was already secured and Jackson added three more.
The Arms Park was alive as Matthew Rees entered the fray on 58 minutes to a massive ovation from all four corners as fans from both sides took to their feet to acclaim the returning hooker.
A charge down by Jones sparked Blues into life and the forwards and backs combined to rumble forward, but when composure was needed most the penalty could not be secured and instead it was Ulster who came away with the ball.
A lost lineout saw Navidi gobble up ball and tear into space before Tau’Fao Filise arrived to take it on. This time a penalty was secured and Blues scored their first points of the half as Davies slotted the kick.
The home side, though, were guilty of presenting three points back within a minute as a high up and under was taken by Bowe and Blues were pinged for offside.
Jackson slotted his fourth penalty of the second half and four minutes later added a fifth as the lead was down to just five points with only nine minutes left on the clock.
Nerves were now mounting inside the Arms Park as the penalty count was killing Blues as yet another saw Jackson make no mistake.
Blues thought they had grabbed decisive possession when a poor pass was almost taken by Czekaj but he knocked on under pressure as he looked to collect the bouncing ball.
A massive tackle from Allen saw Payne knock on and then the full back got caught the wrong side to hand Blues a penalty. Davies lined up the kick and smashed it through the posts to wild cheers.
A final thrilling few minutes saw Ulster go on the attack to try and snatch the win, but Blues would not budge and when they secured a penalty with time up the whistle was met with huge cheers as the Arms Park waved goodbye to PRO12 rugby this season in some style.
Timeline:
04m Cave t 0-5
13m Davies p 3-5
26m Davies p 6-5
30m Davies p 9-5
34m Davies p 12-5
36m Copeland t Davies c 19-5
39m Davies p 22-5
44m Jackson p 22-8
48m Jackson p 22-11
58m Jackson p 22-14
65m Davies p 25-14
67m Jackson p 25-17
72m Jackson p 25-20
75m Jackson p 25-23
78m Davies p 28-23
Cardiff Blues: 15 Dan Fish, 14 Alex Cuthbert, 13 Cory Allen, 12 Isaia Tuifua, 11 Chris Czekaj, 10 Gareth Davies, 9 Lewis Jones (Williams 72); 1 Gethin Jenkins, 2 Kristian Dacey (Rees 58), 3 Taufa’fo Filise, 4 Lou Reed (Down 70), 5 Filo Paulo, 6 Macauley Cook, 7 Josh Navidi, 8 Robin Copeland
Replacements: 16 Matthew Rees, 17 Thomas Davies, 18 Scott Andrews, 19 James Down, 20 Ellis Jenkins, 21 Lloyd Williams, 22 Simon Humberstone, 23 Gavin Evans