Dipo Akinyemi fired his 14th goal in 16 games as his incredible season continued to keep Ayr top of the Championship.
Logan Chalmers inspired a second half comeback in the Highlands with a brilliant solo display as United claimed a point.
But they were denied all three as Nathan Shaw bagged a double for Caley with his second strike coming just three minutes from the end.
Mark McKenzie's strike had cancelled out Shaw's early opener before Akineymi stepped up to fire Ayr ahead late on.
But Shaw had the last say with the fourth goal of a dramatic contest.
Lee Bullen had made two changes from the side which beat Hamilton last week with Daire O'Connor replacing the Jayden Mitchell-Lawson and Sam Ashford in for Paul Smith as Ayr went 4-4-2.
And United should have been ahead early on when Akinyemi fed Ashford but his shot from 12 yards was turned wide by Cammy Mackay.
The hosts quickly made United pay when hitting the front as a long ball caught out skipper Sean McGinty and Shaw stole in to round Charlie Albinson and burst the empty net.
Ayr struggled for much of the first half to gain a foothold in the contest as Caley dominated but neither side could create clear chances in front of goal.
The nearest Ayr came was an O'Connor cross which flew off the head of Sean Welsh and inches wide, before the winger hobbled off just seconds later to be replaced by Chalmers.
And the former Caley man quickly threatened to make an impact with a driving run and shot which stung the palms of Mackay early in the second half.
Akinyemi then turned to fire just wide as Ayr showed far more zip about their play in the early exchanges after the interval.
But it soon proved to be all about Chalmers and he was at the heart of everything Ayr were doing as he drove them forward at every attempt.
He produced a brilliant bit of skill to tee up McKenzie for the goal that put Ayr on level terms as the game was thrown back in the balance.
Ayr were transformed for Chalmers' arrival and he combined with Akinyemi to torment the home defence at every opportunity.
And they turned the game on its head when Akinyemi strode clear of the Caley defence to hammer home his 14th of the season and seemingly put them on course for another three points.
But Shaw had the last say when he wrestled clear inside the box to fire home a low effort and make a point for Caley.
Don't miss the latest Ayrshire headlines – sign up to our free daily newsletter here
