The feathers are most likely part of the normal molting and/or preening process. Are they the very small, fluffy, down feathers? Small down feathers all over the place is typical of a normal preening Grey.
Banana Peel? I'd just give her the banana and not the peel. The general rule is that if you wouldn't eat it then you probably shouldn't give it to them.
"Growling" is a sign that they are afraid. Keep strangers a safe distance from her cage to keep her from getting stressed. Everyone wants to get as close as possible to get a better look but you'll be able to tell at what distance your Grey is comfortable at. With time, it will get more accustomed to the "strangers".
No you're not being neurotic All of your questions are very good and show that you are a thoughtful caregiver. I'm sure everything will be fine.