Why does my son call my exe's boyfriend dad?

I recently came home from a great tour in the Army. I love my son and see him all the time. I even call him just about every day when I dont have him. I show up at all his football games and school functions. I dont mean to bragg, but im there for him all the time. I think we have a great relationship. Recently i was at a football game with him and his mothers boyfriend. I noticed him calling him dad on several occasions when he didnt think i heard him. My son is 6 and they have only been together 2 years. I later asked my son if he ever calls his mothers boyfriend dad and he said no. So obviously he knows i am a little uncomfortable with it. Am I being selfish or hotheaded? I didnt say anything negative to my son because I know he is young and it is not his fault. There was a little dispute a while back because he tried calling my wife mommy. His mother blew her lid, but she says that its ok for him to call her boyfriend dad because he lives with them. I know there is nothing I can do about it because the court systems are so heavily geared toward the mother (no matter who would give the child the best life). Any input would be great. Am I being unreasonable?