Being silly, but where is the fairness?

I can understand the frustration. Right now he's just used to Daddy so Daddy's the one he wants. As he gets older and spends more time with you he'll start to share time equally. Just spend as much time with him as you can for now.