Out of all of my stable, I think I only have three that I really love and could never give up. I do have a handful of pets that have special memories for me (like when I tried out PVPing for real for the first time, or when Ravenlord dropped after randomly taking one along as a good luck charm, or one that gave me the run around for a long time before I was able to tame it) and they will probably stay in my stable for a good long time. Then I have a bunch of pets in my stable that I like, but don't have any special connection to them, and they're mostly there just so I have their particular buff on hand.
Lately, while doing dailies, I have been bringing along pets that haven't been out for a while. Because of this, I have discovered a new fondness for my green Molten Front spider (Kririx). I like how he matches the green glow of my transmog armor, and the way he swims looks really silly. I tamed him in Cata mostly because he was a challenge tame, but now I'm growing to really like him beside me.
Sometimes, when bored, I open up Petopia and look at the gallery of pet models and randomly choose one I haven't ever tamed, haven't tamed in a very long time, or one that might look good with my transmog. When I do this, it usually becomes an evening of taming a million pets and "trying them on" (taking them along to farm instances or quest). Sometimes I find a pet that I'll really like and will keep around for a while. Once in a while, I'll find one I absolutely adore; like my Bronze Quilen. He was a random tame and now I couldn't possibly get rid of him.
So really, don't worry about it if you don't fall in love with every pet you have, it's ok to just have a few favorites. And it's certainly fine to only like some for a little while and trade them in for something new. Some hunters absolutely adore every pet they have and use them all, but some of us don't.
I agree that you might enjoy the
Random Pet Challenge. I found a pet on my alt that I adore which I never would have liked on my main that way.