This is a frequently asked question on the WordPress site. The answer is there if you have nice URLs enabled. If you do not, the answer is:

  1. Mouse-over your category list and note the integer after the cat=? parameter of the URL
  2. Append ‘&cat=?’ to your normal RSS feed URL

For example, the RSS feed for rowing posts only on this blog is:
http://powertwenty.com/kpd/blog/?feed=rss2&cat=2

Update 1/17/2007:
Using Weichung Yang’s Urlencoded Permalinks plugin this is even easier as the link becomes:
http://powertwenty.com/kpd/blog/index.php/category/rowing/feed/