City came into the game at the Etihad Stadium on the back of 12 successive Premier League wins, but they were up against a Foxes side which have improved greatly since Brendan Rodgers took charge and who also defeated the Citizens on home soil on Boxing Day.
And, while the hosts enjoyed the majority of possession, Leicester more than held their own for the first half before defending deeply and resolutely after the break.
But, just as it looked like it might not be City’s night, veteran defender Kompany popped with a stunning pile-driver 20 minutes from time to snatch the win.
The three points leave City one ahead of Liverpool going into Sunday’s final round of fixtures and knowing a win at Brighton will see them defend their title.
Phil Foden replaced Leroy Sane in just one change for Manchester City from the 1-0 win at Burnley last time out, while Rodgers named the same Leicester team that started last weekend's 3-0 win against Arsenal at the King Power Stadium.
City started on the front-foot but found it difficult to create anything, while it was Leicester who carved out the first opening in the eighth minute.
James Maddison’s clever flick picked out the run of Ricardo Pereira down the right of the area and he cut inside before seeing his low strike blocked by Kompany who threw his body in the way of the ball.
Kyle Walker then shot straight at Kasper Schmeichel for City, before Youri Tielemans picked up possession from Jamie Vardy and left fly from 25 yards but Ederson was equal to the task in the City goal.
City had a hopeful penalty appeal ignored just after the 20-minute mark when Raheem Sterling’s attempt to dink the ball into the middle of the area appeared to catch the hand of Jonny Evans.
The hosts’ pressure almost paid off on 32 minutes when Sergio Aguero got his head to a corner at the near post and the ball came down off the underside of the crossbar where Schmeichel pushed it clear before it could fully cross the line.
City continued to lay siege to the visitors’ area with Kompany heading the ball back across the area and it rebounded out to Ilkay Gundogan who hit a 20-yard effort inches wide of Schmeichel’s far corner.
On the stroke of half-time Bernardo Silva was given a brief sight of goal by the retreating Harry Maguire but his strike from 25 yards was comfortably held by Schmeichel.
It was more of the same after the break as David Silva did well to divert a cross towards the back post back into the middle where Sterling was stood unmarked, but the ball would not drop quickly enough and he could only send his header over the top from six yards.
On 58 minutes, Gundogan looked to have made the breakthrough but he watched as his dipping 25-yard strike from the right channel bounced agonisingly wide of Schmeichel’s far post.
Two minutes later, another penalty appeal was turned down by referee Mike Dean when Hamza Choudhury got a slight touch on the ball as Sterling went to ground from the challenge inside the area.
Despite City’s dominance, it was Leicester who had the next opportunity on the counter-attack when Maguire surged forward before picking out Maddison and he took a touch before curling a strike just wide from distance.
City went so close on 69 minutes when Aguero looked certain to score when the ball dropped to him inside the area, but Schmeichel produced a fantastic save to deny him.
However, the hosts did finally get their noses in front a minute later when Kompany picked up possession of the ball 25 yards from goal and the big centre-half sent a rocket of a shot flying into the top corner of the net to send the Etihad into raptures.
The goal appeared to lift a weight off the shoulders of the City players and eased the growing tension in the stands with Bernardo Silva looking to emulate his skipper on 84 minutes only for his strike to fly high and wide.
However, Leicester should have levelled just three minutes from time when Choudhury won possession and played in substitute Kelechi Iheancho, but the former City forward got his angles all wrong as he side-footed wide of the advancing Ederson's goal from just outside the box.