The South Americans should really have scored five minutes later when Giancarlo Gonzalez met a corner-kick at the back post, but under no pressure he headed over the target from six-yards out.
SerbiacoachMladen Krstajic handed Branislav Ivanovic his 104th worldwide appearance, making the former Chelsea centre-back the most-capped player in his nation's history, moving clear of Dejan Stankovic in the all-time list.
SerbiacoachMladen Krstajic, taking charge of his first competitive game, picked in-form striker Aleksandar Mitrovic to spearhead the attack.
The Costa Rican bench were not impressed with the Serbian's vigorous efforts on the sideline.
Serbia were not made to rue those misses, however, and - although a touchline skirmish between Nemanja Matic and a Costa Ricacoach threatened to derail proceedings - stood firm to claim just their second victory from their last seven World Cup group stage matches.
"It was an wonderful goal", Costa Rica defender Bryan Oviedo said of Kolarov's effort.
He has become the third player to score a free-kick in the World Cup thus far, following on from Aleksandr Golovin and Cristiano Ronaldo's excellent efforts. "We know the eyes of the world are on us".
The Lazio playmaker was Serbia's most creative influence, although his blushes were spared by the linesman's flag when he fluffed a first-half chance after racing through on goal.
The Eagles were presented with another opening soon after as Mitrovic's deflected strike from inside the box was comfortably gathered by Keylor Navas, before a quiet spell ensued.
Now with Italian club Lazio, rumors have swirled that Manchester United might be interested in the dynamic 23-year-old Milinkovic-Savic.. The goal wouldn't have counted, as Milinkovic-Savic was offside, but it was a handsome effort nonetheless.
In a bad position after losing, Costa Rica will quickly need to pick themselves up ahead of their second game - against tournament favourites Brazil in five days' time.
"From what I've seen in the locker room, we still believe", the Costa Ricacoach said.
Who and what are behind Germany's government crisis?
She has meetings planned this week with the leaders of Italy and France, and an European Union summit is scheduled June 28-29. Migration "is a European challenge that needs a European answer", she said in her weekly video message Saturday.
Trump slams media coverage of North Korea summit
The summer exercise, Ulchi Freedom Guardian, consisted mainly of computer simulations to hone joint decision-making and planning. Trump said via Twitter on Sunday that it was his initiative to suspend military drills - a step North Korea has long sought.
Eurydice Dixon: Comedian’s killing prompts anger in Australia
A vigil has been organised for Monday evening at Princes Park, aimed at reclaiming the public space where Ms Dixon was found. At a press conference on Thursday, Superintendent David Clayton urged women to "be aware of their own personal security".