How does a DSLR choose aperture in Program mode?

This is called the "program line". Imagine a grid labeled aperture in one direction and shutter speed the other way, as in the "exposure rectangle" in the middle of this answer. There are many different combinations which could produce the same exposure, and the program line is literally the line drawn on the graph to represent what a given program actually chooses for a certain lens. If you add in auto-ISO, it becomes three-dimensional but is still basically the same.

In older cameras, the line was simple, and the desired exposure value was all that was considered. If a camera decided to give you f/4 and ¹⁄₅₀₀th to hit EV 13 (at ISO 100) in a given situation, it would do the same thing in any situation with the same target exposure. Modern cameras now have algorithms that result in multiple lines (or, a complex polyhedron within the cube, if you want to get all mathy), depending on factors like lens focal length, flash, user-selected parameters, and maybe even image stabilization.

These days, the exact program is secret sauce and there's no general answer; each brand (and possibly even each model from the same brand) has a different algorithm. As an example, my Pentax camera offers the following choices, which influence which program line is used:

• Normal — Pentax calls this "Basic Program Automatic Exposure", and it tends towards middling values for everything.
• Hi-speed Priority — The documentation says "prioritizes high shutter speeds" (helpful, guys). Based on experience, the basic program for shutter speed takes the focal length into account (faster speeds for longer lenses), and I'm not sure if this acts as a weighting of that factor or if it has a pre-set idea of "high" in this mode.
• DOF Priority (shallow) — Pretty self-explanatory: the program attempts to hold aperture as wide as possible, unless it can't to make the exposure work. Shutter speed is informed by the focal length, and how that and ISO are adjusted depend on the auto-ISO setting (see below).
• DOF Priority (deep) — The opposite, of course.
• MTF Priority — When used with a Pentax lens with electronic coupling, prioritizes the lens's technically-best "sweet spot" (see this question on MTF charts)

And to add complication, there's also an Auto, which has no technical documentation, but presumably chooses one of the above programs based on... who knows, actually. Maybe focus mode, probably the metering. Maybe the current zodiac sign.

If you have auto-ISO enabled (a separate choice from that "Auto"), there's a complex interaction with the program mode choices and the settings for auto-ISO, which can be "slow", "standard", or "fast". If you have it set to "slow", something else — shutter or ISO, depending on how that fits the program mode choice — will be increased first. If you have it set to "fast", raising ISO is the preference, and "standard" is somewhere in between. The upper and lower bounds of automatic ISO are chosen in by yet another set of customizable parameters.

Lower-model cameras will just keep everything secret — although "scene modes" like sports or landscape will serve a similar function. Other high-end cameras may have similar complex parameters to tweak, although in general they have fewer options than my example camera.

There's a detailed discussion of Canon's program mode in this Photography On the Net thread, and Canon officially provides a nice explanation of how the basic algorithm works for their older cameras, including a handy chart:

It's almost easier to describe than to read the chart, though: as brightness increases from EV -5, the aperture stays wide open and shutter speed increases until stopping down is an option, at which point shutter speed and aperture are both reduced evenly until the minimum aperture of the lens is reached, at which point it's back to the shutter.

Canon doesn't publish charts like this for their current cameras. The referenced thread suggests that the basic program is the same with auto-ISO enabled, and Canon does not provide official explanation for what logic that mode uses, although anecdotal evidence suggests it attempts to keep the shutter speed fast enough to match the focal length.

Nikon also does not offer many choices on their current DSLRs, although some Nikon film cameras offered choices between normal and high speed programs. Now, from the entry-level models to the high-end, the P mode is a black box with regard to shutter speed and aperture, and ISO is controlled via a minimum shutter speed — if the regular program would require a shutter speed below that, and auto-ISO is on, ISO is increased instead. This has some advantages (particularly when you want to control subject motion), but doesn't appear to take lens focal length into account (although the program line is probably already adjusted for that), and the Min Shutter Spd option is buried several levels deep in the "Shooting Menu".

With some experimentation with P mode, one could reverse-engineer the program mode logic, but I think mostly, Canon and Nikon expect you to move on to the more-controlled semi-auto modes once you care about stuff like that.

Pentax, though, goes out of their way to make the automation accessible and flexible. On my camera, a Pentax K-7, and on other recent Pentax DSLRs, the "program mode" is easily selectable via the "info" button menu, where it's both discoverable and quick to use if you really want to learn and use the ins and outs of the different modes. A screen capture:

And ISO is similar, although actually changing the auto-ISO parameters is one level deeper:

(An aside: if you just want to change the current ISO, the camera has a dedicated button for that. It can also be changed through the Info menu if you want or desire, without going into this submenu.)

In general, though, if you really care about getting a certain aperture, that's what Aperture Priority mode is there for. My informal impression (see for example many of the answers to this related question) is that aperture priority is the most commonly used automatic mode for more technically-inclined photographers. It's interesting that Pentax has chosen to work on providing advanced options here, but I think the bigger companies might be right in considering it not particularly important.

One could try to remember all of the complicated interactions above and choose the intersection that best matches your intentions for a given situation — or you can just, y'know, choose directly. That actually seems easier than all the automation.

I leave my camera's P mode set to MTF Priority — there, the automation is actually doing something useful for me, since the camera knows the MTF for my various lenses at different apertures, and I don't carry that in my head.