The is built into the product as Friendly URLs, which you can enable on the Friendly URL tab in the System Configuration settings under Administration menu in the manager. You will also need to rename the ht.access file to .htaccess and modify it as necessary for your installation (see notes in .htaccess). You may also need to disable the php directives for setting compression if your host does not support php directives in .htaccess. This does require the Apache mod_rewrite extension to be available.
There are lots of posts about this topic here in the forums; if you have any trouble, a search should reveal a number of problems/solutions related to this topic. Remember to search for terms with more than 3 characters though...