One big advantage to hosting a games on the scale of the Olympics (specially summer) is the multilevel government spending on things like transit and roads (because they are now forced to). If done well that is the big positive legacy (assuming they go to the right places). For an area like the GTA this could be a big win, the money has to come from somewhere though so expect cuts to other areas and more of the GTA's tax money being spent in the GTA (as opposed to be dispersed across the province/country).
One big downside are these event locations that become white elephants after the games like the pictures show. There needs to be a plan on how to use these locations after the games and to follow the new trend of building disposable or portable stadiums. Smart architecture like the foundation for the temporary stadium is designed as the foundation to a condo tower to be built there right afterwards...
For new stadiums that will remain in place. The tax payers should be paid back via a seat tax. For every event in the stadium as long as it stands say $2 (or maybe say 10%) of every ticket sold goes back to the government coffers. Not sure if that would work for the one in Hamilton since it will be way underused but for things like a hockey arena, etc. (every game, every concert, forever).
There is also the big uptick in employment leading up to the games (construction, etc.) but this leads to a big hangover afterwards as these people hit the unemployment lines.
Many of these issues can be dealt with, by thinking a little outside the box...