How do I choose an f-number when designing a camera system?

Assuming you're designing the internals of the camera, it isn't completely unreasonable to simply specify a "reasonable" figure for focal length (to give the desired field of view) and aperture (to provide sufficient light) and call out a commercially available lens with those specs. For instance, if you had an old 1/3 inch (diagonal) 4:3 sensor and wanted a "normal" field of view (30-35 degrees corner to corner) you'd specify a lens in the range around 8-10 mm focal length and f/4 or so (maybe faster if you wish to design in low light capability), then look for commercially available lenses in with those parameters. Given that's a fairly common range for video cameras, you shouldn't have any problem finding a make/model to call out "or equivalent".

Now, if you want/need to design the actual lens, that's a much more complex problem and would require you to spend considerable time with optical design ray tracing software, modifying existing designs to meet your specific needs -- but in that case, your professor should have at least introduced you to the school's copy of such software (it typically costs thousands of dollars for a license, so you probably aren't expected to buy your own) and given a starter on how to use it.