A category page is like an index or a subsection of your site. For an ecommerce site that sells shoes it would be the page that lists all the men's hiking boots. For a content site such as this one it would be the page that lists all the search engine optimization articles.
As a general rule category pages carry more specific keywords than homepages. If the competition for your homepage keywords is too great there is a chance the competition for your category page keywords will be palatable. Sites with just one homepage can include dozens of category pages (or more) and so there can be significantly more traffic potential.
There are also sometimes different levels of category pages. For instance, using the shoe store example. The first level category page would be "Men's" then perhaps "Men's Boots" then finally "Men's Hiking Boots." If your site contains multiple category levels you should decide which ones you want to focus on the most. Some things can be done for all category levels, but others will favor one or the other.
Category page optimization is simpler since you will be responsible for most of the incoming links for each category page as they will be coming from other parts of your site. You must link to the category page using keywords as the anchor text, you should put the keywords prominently in the title tag and in headings on the page itself. It also can help if you include a paragraph or two or keyword rich content. This is especially true for ecommerce sites which so often do not include any text at all on category pages.
The important thing here though is your internal linking structure. You should link to every category page at the level which you are targeting on your homepage. This can be a lot of links, and that may affect your decision of which level to target. For instance on this site's homepage the left menu includes just the first level category, plus Search Engine Optimization, which is a second level category but one I want to give extra attention too. However on the homepage itself I also list all the second level categories.
Linking to the categories on every page of your site also helps, which is why I put the search engine optimization link on the left menu. However the other secondary level categories are not there because I want to keep the left menu small. Also the more links you add, the less weight each will receive.
The important thing to remember when doing category optimization is that the more links you can add to that category on your site the better off it is. Putting a link to the category on your main menu, which is included on every page, is the most you can do. But if you can't do that you can still add as many cross links between categories as you can. Also, breadcrumb navigation is a good way to add a link back to your category page from your content or product pages.
You should also focus on gaining incoming links for these pages as well. Many people focus only on incoming links for the homepage when you can submit category pages to directories and or use them for link exchanges.
You're still linking back to your homepage on every page of your site, for usability reasons, but by adding more internal links to and among the category pages you are spreading your available weight around more and thus your homepage will have less than if you minimized internal links and used a highly hierarchical, as opposed to mesh, linking structure. So while you can do both, you can't do both equally well.