The reason that you may not be able to delete the calculator is that if there is a copy open when you try to delete it, Windows will see it as bing "in use" and not let you until it thinks the calc is not being used.
The keyboard could be brand new ad still have some weird short or something that causes it to send out the code to start the calculator. I've seen bad keyboards do some very strange things.
