Only about half the countries on Earth have any athletes at all in the 2014 Olympics, so it's a hundred-way tie for fewest.
The Caribbean island of Dominica has only two athletes competing in Beijing at the 2008 Olympics. It is the smallest team.
11 nations have 2 athletes
There are 11 countries with only 2 athletes
Many countries have 5 or less athletes and it is more than a 10 way tie for those with only 2
More than 10 countries had only 2 athletes
In checking the teams from each country, I found that Nauru has only one athlete participating in the Beijing Olympics. Brunei and Bhutan each have two athletes. There may be other countries with only 1-2 athletes total at the summer Olympic games.
Iran have not provided any athletes this year
Antarctica Well Sealand has one of the fewest people living, but it's not noticed as a country.