IANAL, but I do have a lawyer that I consult with in my own photography business, and his legal opinion to me for my photography business was that the public has no right to an expectation of privacy when in a public place, or at an event 'where photography is a common and expected thing' (e.g. a birthday party, wedding or other similar event), so as long as the following conditions are met:

1. The people are 'background players' in a photograph (e.g. Guests at a wedding, not the bride)
2. The use-case is portfolio and advertising of my own business (e.g. Selling to a stock site, or using it as a commercial work is a different story with different release requirements)
3. I'm not altering the people in the picture in a way that could be construed as libelous or scandalous (e.g. I can't use Photoshop to alter a picture to falsely portray a wedding guest snorting coke off the wedding cake... Unless it actually happened!)

As long as those three requirements are met, according to my lawyer, I'm all good to use the picture for portfolio or advertising purposes. In practice this means that I get model releases from any main subject(s) in a picture I take (for weddings I generally have the entire bridal party, minister, etc. sign releases), and even though they have no legal right to ask me to, if someone pictured asks me not to use a picture with them in it I generally comply because I'd rather take down a picture than have to field a lawsuit (even one I know I would win). I've never actually had to take down a picture, however.