In the light of this, we take a look at other instances where sports made the Olympic cut or were unceremoniously dropped.
We start with the Rio Olympics this year, where rugby and golf are poised to make long-awaited returns.
1. 2016 RIO DE JANEIRO
Sports introduced: Golf, rugby sevens
Sports removed: None
After an absence of 112 years, golf is making its belated Olympic comeback in Rio. However, more than half of the men's world's top 10 have pulled out due to concerns over the Zika virus, leaving world No. 5 Henrik Stenson of Sweden as the leading contender.
Rugby last featured in the Games from 1900 to 1924, with the Americans winning the final two tournaments in 1920 and 1924. Back-to-back men's world series champions Fiji are favourites for the gold in Rio and will be looking to win their country's first Olympic medal.
2. 2012 LONDON
Sports introduced: Women's boxing, tennis mixed doubles
Sports removed: Baseball, softball
Women's boxing was announced as an Olympic sport in 2009 to much opposition. One of the detractors, British boxer Amir Khan, felt "deep down" that women should not fight.
In fact, the sport was completely banned in the UK until 1996 because the British Board of Control had ruled that pre-menstrual tension made women too unstable to box.
As Brit Nicola Adams fought her way to Olympic gold with skill and dexterity, her triumph was as much a victory for women's boxing.
Baseball was voted out by IOC members in 2005, making it the first sport to be removed since polo in 1936. The constant absence of Major League Baseball (MLB) players from the Olympics was a big negative for the sport's Olympic bid due to scheduling clashes with the MLB . However, the sport will be reinstated in the 2020 edition of the Games after the latest IOC vote.
3. 2008 BEIJING
Sports introduced: BMX cycling, women's steeplechase, open water marathon swimming
Sports removed: None
Marathon swimming saw competitors race over a 10km open-water circuit, which was an initial concern in heavily polluted Beijing.
Bicycle motocross racing also made its debut at the Games, even though it was not regarded to be as popular as fellow action sports like skateboarding or freestyle. A crucial reason for this was its strategic decision to be acquired by the Union Cycliste Internationale, which gave it the necessary organisational clout to be considered for the Olympics.
4. 2004 ATHENS
Sports introduced: Women's wrestling
Sports removed: None
While wrestling has been an Olympic sport since the inception of the Games, women's wrestling made it to the party only in 2004. It had faced the same discriminatory treatment as its boxing counterpart where people could not imagine women belonging on the wrestling mat.
5. 2000 SYDNEY
Sports introduced: Triathlon, taekwondo, trampoline
Sports removed: None
Taekwondo became one of two Asian martial arts to be included in the Olympics alongside judo. The sport's inclusion has often been under a cloud, as it was marred by judging scandals from the beginning.
The president of the World Taekwondo Federation, Chong Woo Lee, admitted to intimidating judges to favour South Korean athletes at the 2000 Games.
6. 1996 ATLANTA
Sports introduced: Softball, beach volleyball, women's football and mountain biking
Sports removed: None
An increased focus on gender equality in sports led to the inclusion of women's football in addition to the men's tournament which had featured in every Summer Olympics.
The inclusion of beach volleyball raised eyebrows as its legitimacy as a sport was questioned. It was also shrouded in controversy due to its usual attire of bikinis being seen as the main draw for fans and also an obstacle to the participation of more conservative countries.
Sources: ABC News, BBC, Fox Sports, The Guardian, Wall Street Journal, ESPN, The Independent, Reuters