You also need to create a manager user group and connect it to the document group. Then only manager users who also belong to that manager user group can see the TV. The groups are centered around users.
Template Variables are also tied to the documents’ Template. Have one Template for the documents that you want your TV to apply to (and assign your TV only to that Template), and assign all other documents to another Template.
It is often the case that you may want the contents of the Template itself to be the same, but just have different TVs apply to them. This is a problem that has a few different solutions, depending on your philosophy. I usually just end up duplicating templates, because I find in the end that they change very little (most of the interesting stuff gets abstracted into Chunks or various Snippet calls.)
Yes, I do it all the time. I create a user group and a document group and connect them (in the manager permissions section). I assign my admin user to the user group, and assign the TV to the document group. Other users cannot see the TV. Or is that not what you want?
If you only want the TV to be associated with certain documents, as netProphet says, TVs are associated with templates, not documents. All documents using a template get all of that template’s TVs. You’ll need different templates in that case.