Philippe Clement's side were rocked by an early own goal from captain James Tavernier, whose penalty was then saved by Will Dennis after Kilmarnock's Joe Wright was sent off for handball.
But Rangers finally drew level through Portuguese forward Fabio Silva deep into first half stoppage-time at Ibrox.
Ben Davies came off the bench to put the hosts ahead. Then Tom Lawrence curled in a superb third and John Souttar headed the fourth in the final seconds.
With just three games remaining in the title race, Rangers are three points behind leaders Celtic and have an inferior goal difference by five.
The Glasgow rivals meet in the last league Old Firm game of the season at Parkhead on Saturday.
Champions Celtic had cemented their lead with a 3-0 win over Hearts at Parkhead on Saturday.
Rangers were in danger of surrendering any genuine hope of catching Brendan Rodgers' men when Kilmarnock took a shock 12th minute lead.
Matty Kennedy did not connect cleanly with a deep cross from Liam Polworth at the far post but the ball somehow beat Rangers keeper Jack Butland, came off Tavernier and rolled over the line.
In the 21st minute, Kilmarnock defender Wright used his hand to deny Dujon Sterling close to the goal line following a Silva cross.
Wright was dismissed, but Dennis brilliantly saved Tavernier's penalty.
Making the most of their numerical superiority, Rangers drew level in the eighth minute of stoppage-time when John Lundstram's lofted cross was directed in by Silva from 12 yards.
Davies took over from Leon Balogun for his first action since December and reacted quickest to net after Dennis spilled a powerful drive by Lundstram.
Lundstram came close with a curling shot from 20 yards before Lawrence took a pass from fellow substitute Ross McCausland, turned inside Lewis Mayo and curled the ball past Dennis.
There was still time for Souttar to head in from close-range after Dennis had parried a Lawrence drive.
