That’s definitely doable. See if this helps:
http://bobsguides.com/permissions.html
This might help:
To stop a manager user from accessing a document,
1. the document must be in a document group, and
2. the document group must be linked to a user group that the user doesn’t belong to