Tommy Johnson fired a reminder to manager Martin O'Neill to bag Celtic's winner at Parkhead to keep their 100% start to the new season intact.
The former Aston Villa striker had been forced to sit in the shadows following the £6million signing of Chris Sutton, who has formed an immediate understanding with Henrik Larsson in their first three games.
However, Johnson got the chance to impress with Sutton suspended ahead of transfer-listed youngster Mark Burchill and he showed O'Neill that he too could play in his Parkhead revolution.
However, Swedish international striker Larsson can make most strikers stand out and he rescued his side with the second-half equaliser after Kilmarnock had threatened to end their unbeaten run when they took the lead through a glorious Andy McLaren strike.
The home side were not at their best but Larsson was again involved to create Johnson's winner and give O'Neill his fourth win in as many games.
His side started with the confidence of a team that had won their two previous league games and blew away Luxembourg minnows Jeunesse Esch in midweek.
Their patient build-up play almost paid dividends in the eighth minute when Alan Stubbs, Larsson and Johnson combined to send a rejuvenated Bobby Petta down the left flank and Kilmarnock goalkeeper Gordon Marshall had to punch clear with Larsson looming with intent at the far post.
Moments later they should have again opened the scoring when Stubbs met Lubo Moravcik's corner, but James Fowler cleared off the line and Petta followed up to blaze erratically over.
Kilmarnock were struggling to impose themselves in the opposition's half early on but they gave Celtic a warning in the 16th minute when Alan Mahood flashed a speculative right-foot shot just past the post.
A minute later and Marshall saved low to deny Johnson just inside the box, but in the 18th minute Kilmarnock stunned the home crowd to take a shock lead when McLaren curled a brilliant right-foot shot from 25 yards into the top corner.
But Celtic could have cancelled that out just a minute later when Stilian Petrov was played clear through by Johnson inside the box but he shot high over the bar.
However, O'Neill's men could have fallen further behind on the half-hour when Jonathan Gould pulled off a wonderful acrobatic save to deny Peter Canero after he exchanged passes with Ally McCoist on the edge of the area.
Marshall was again called up to save his side in the 42nd minute when Larsson headed the ball through to Johnson but the goalkeeper raced off his line to clear into touch.
Celtic were riding their luck and just a minute into the second half they had Gould again to thank after defence hesitated to let Mahood clean through but the Scottish international saved bravely with his feet.
A minute later and he was again called upon to deny Gus MacPherson, who produced a powerful 25-yard effort on target.
However with Larsson in the side Celtic always have a chance of scoring goals when not at their best and he got the Hoops back on level terms in the 50th minute.
Moravcik found Paul Lambert on the right and the midfielder played the ball in to the Swede, who drew Marshall before sliding a right-footed passing shot past the keeper.
Celtic then produced their best move of the match in the 63rd minute when Petta sent Moravcik galloping down the left and he pulled the ball across the face of goal into the path of Larsson, but Marshall blocked to deny the dreadlocked hitman.
Larsson was causing the visiting defence all kinds of problems throughout and it was the Swede who created the winner in the 73rd minute when he played the ball through to Johnson, who beat the offside trap, and drew Marshall before casually slotting a right-foot shot into the corner.
Five minutes later and Celtic came within inches of extending their lead when the impressive Petta crossed for Moravcik to head just past the post.
Kilmarnock, who were unlucky to lose against Rangers last week, were again letting their frustrations affect them, but substitute Christophe Cocard missed a great chance with three minutes remaining when he fired McLaren's centre past the post.
However, Kilmarnock's misery increased in the very final minute when Martin Baker was dismissed for a foul on Moravcik and referee Kenny Clark had no hesitation in brandishing a second yellow card.