In many ways, though, they still came out as the vote's biggest winners. These results were predicted, as were the subsequent talks among the parties.
Instead, Sunday's results effectively provided no clear victor and no convenient or simple narrative to explain the shifts underway in Swedish politics. That third-place finish is their best electoral result. The center-left party emerged with the greatest share of the vote - 28.4 percent as the count neared completion - yet looking at holding fewer parliament seats than four years ago.
Sweden's ruling party was headed for its worst showing in decades as voters flocked to an anti-immigrant party with white supremacist roots that was poised to become the third-biggest force in parliament.
He added: "It is clear that a cross-bloc coalition is needed to govern".
While the result is a boost for the Sweden Democrats, the party fell short of pre-election predictions.
"This government we have had now. they have prioritised, during these four years, asylum-seekers", Akesson said, giving an exhaustive list of things he says the government has failed to do for Swedish society because of migrants.
Like its far-right counterparts across Europe, the Sweden Democrats party has taken pains to move into the mainstream and, to a certain extent, tone down its rhetoric in order to appeal to a broader swath of the electorate.
But the far-right has presented it as a vote on immigration and integration, after Sweden took in more than 160,000 asylum seekers in 2015 alone, a per capita record in Europe.
"In Sweden we live in a false dictatorship because none of the other parties will ever let the Sweden Democrats have any power", he complained, as his colleague Adin shook his head in amused disagreement. However, in 2006 the party lost to the center-right Alliance for Sweden and wouldn't regain control of the government until 2014. Lacking the tough mountain geography of fellow-neutral Switzerland, they've had to make a lot of compromises to maintain Sweden's independence. "Historically it has been always been this way in Sweden", he said. And this wasn't just campaign speak.
George Papadopoulos, former Trump campaign adviser, sentenced to 2 weeks in prison
The judge had planned a harsher sentence but said he was moved by Papadopolous' words of contrition. That account conflicts with what Sessions, now attorney general, testified before Congress.
The polls showed far-right, anti-immigration Sweden Democrats would get 19.1 percent of the votes. However, the mainstream parties are extremely reluctant to work with the group, because of its nationalist, exclusivist values.
Mohamed Nuur, a 26-year-old Social Democratic candidate of Somali descent, said he sees Akesson taking Sweden back to the past. The Social Democrats had dominated Swedish politics through much of the 20th century.
Analysts said that the most likely outcome was a centre-right alliance forming a minority government, with backing from the radical right.
Meanwhile, Liberal Party leader Jan Bjorklund has said he wants a centre-right alliance government, and has also ruled out any cooperation with the Sweden Democrats. Both the Social Democratic Party and the Moderate Party have always had to rely on the support of smaller allied parties to form working government coalitions; some would certainly argue that Sweden's diverse political landscape is a feature, rather than a bug.
But party members say there isn't really another option for a stable coalition government: Each side-Social Democrats, Greens, and the Left Party on the left and the Moderates, Christian Democrats, Center Party, and the Liberals on the right-is running at about the same level of support.
When voting for a prime minister in parliament, a proposal is passed unless a majority of all 349 lawmakers vote against. They'll "have to face the facts", Andersson said.
So - why won't the Sweden Democrats form a coalition Government?
The opposition is intent on ousting Lofven, with some Moderates willing to go so far as to put an end to Sweden Demorats' pariah status and open negotiations with them.
He sparked an outcry during a televised debate on Friday when he said foreigners had more difficulty finding jobs "because they're not Swedes".