The Serbian is 40 places above Montanes in the world rankings, but there was little to separate the two in a tight contest, with Lajovic eventually coming through 6-3 1-6 7-6 (7-4) at the ATP 250 clay-court event.
World number 73 Lajovic showed his quality on clay by reaching the last 16 of this year's French Open before being beaten by eventual champion Rafael Nadal, and started well in Bastad.
Despite a second-set stumble, the 24-year-old recovered to seal his progress in a third-set tie-break.
Lajovic will now face big-hitting Pole Janowicz, who was awarded a bye in round one.
The fifth-seeded Joao Sousa is also into round two. The Portuguese was on court for just 10 minutes, as opponent Pere Riba retired with a neck strain.
Elsewhere, Paolo Lorenzi won on the ATP World Tour for the first time since March with a clinical 6-2 6-1 win over wildcard Markus Eriksson.