Should I exterminate a colony of field ants in my yard?

I recently bought a house in an established neighborhood (built in mid 50's with lots of trees). I was showering the other day and noticed an ant had gotten in. Since then i've only seen about 1 every 2 or 3 days but I know where their colony is. Found the anthill while mowing. My question is if I kill the ants will I have a bigger problem with some other creature or should I just leave it be and spray around my house to discourage their exploration of my home? Pretty sure they are field ants by the description and nest. Thanks