This female he used to live with must not have treated him very well and that is why he seems to not like women and this is something he may never get over. Greys do tend to choose their favorite people and some greys will choose one gender over the other but that does not mean you will never have any kind of relationship with him.
You will have to continue to interact with him, sit by his cage and talk softly to him, offer him treats, maybe take him into another room to work with him and preferably when no other males are around, I believe you will have more luck that way. Be prepared to spend a lot of time and be very patient with him as you don't know about his past and he has some serious trust issues to deal with, this may take months and even years depending on how old he is.
Just don't take it personally, he is being affected by his past and whatever happened and it will take a long time for him to come to trust you but he may never warm up to you like he does for your dad so don't be too disappointed if he always favors him over you.
It is wonderful that you could take in this grey and give him a good home, he needs the love and attention of someone who will give him what he needs the most, a forever home.