First, try logging out, deleting all files in the core/cache directory, clearing your browser cache and cookies, and logging back in -- then try removing the package again.
That said, I had a similar problem with Articles and I ended up manually removing the components, wading through the DB to delete the final pieces of it, and then deleting the DB record for the package and the package file. I have since re-installed Articles and it is working fine, so I think this was a problem with an early version.
Just deleting the package file by itself will confuse Package Manager because the package will still be in the modx_transport_packages table in the DB and the namespace, lexicon entries, system settings, category, and elements may still be around.
------------------------------------------------------------------------------------------
PLEASE, PLEASE specify the version of MODX you are using.
MODX info for everyone:
http://bobsguides.com/modx.html