Andy McLaren fired his third goal in as many games to give Kilmarnock a shock half-time lead at Parkhead.
After his impressive double against their old firm rivals Rangers last week, he proved that his problems were behind him as Celtic's 100% start to the new season looked in danger of coming to an end.
Celtic manager Martin O'Neill made four changes from the side which crushed Luxembourg minnows Geunesse Esch in the UEFA Cup in midweek.
Tommy Johnson replaced the suspended Chris Sutton while Alan Stubbs, Tom Boyd and Stilian Petrov replaced the suspended Jackie McNamara, Vidar Riseth and Swede Johan Mjallby.
Kilmarnock manager Bobby Williamson sprung a surprise with the inclusion of former Rangers man Ally McCoist, so often the thorn in Celtic's side over the years, and he came in to make his first start of the new campaign.
Defender Kevin McGowne was ruled out after his dismissal against Rangers last week.
Celtic started with confidence with a patient build up play and had two good chances to take the lead in the eight minute.
First Stubbs, Henrik Larsson and Johnson combined to set Bobby Petta free down the left flank and his dangerous in-swinging cross was punched clear by goalkeeper Gordon Marshall with Larsson looming at the back post.
Then moments later Stubbs met Lubo Moravcik's corner at the far post and James Fowler was on hand to make a goal-line clearance and Petta blazed the follow-up high over the bar.
Kilmarnock were looking to ease some of the pressure on their defence and in the 16th minute Alan Mahood tried a speculative right-foot shot that flashed just past the post to give Celtic a warning.
A minute later and Marshall again had to make a save low to deny Johnson with a fierce effort just inside the box.
But it was Kilmarnock who drew first blood to take a shock lead in the 18th minute when McLaren, who scored twice against Rangers last week, curled a brilliant right-foot effort into the top corner from all of 25 yards.
However less than a minute later the home side should have been back on level terms when Johnson released Petrov one-on-one with the goalkeeper but he drove high over the bar.
Kilmarnock needed to be tight at the back, but goalkeeper Marshall was doing nothing to steady their nerves at the home side pressed for an equaliser.
On the half hour his blunder almost cost his side when he spilled a dangerous cross from Petta after good work down the flank from the lively winger but Frenchman Frederic Dindeleux cleared the danger.
Jonathan Gould prevented his side from going further behind when he pulled off a wonderful acrobatic save to deny Peter Canero, after he exchanged passes with McCoist outside the box.
Larsson tried to beat Marshall in the 40th minute with a free-kick, but Marshall saved comfortably.
The Kilmarnock goalkeeper was again in the thick of action in the 42nd minute to maintain his side's lead when Larsson played the ball through to Johnson inside the area, but Marshall raced off his line to clear the ball into touch.
With two minutes of the half remaining, Martin Baker was booked for a late challenge on Paul Lambert and Petrov followed the Kilmarnock man into the referee's notebook for his protest.