Tiny black ants typically found in and around the garden are not harmful as they do serve important functions out doors. Ants aerate the soil, break down organic matter and control the population of other insects. So your goal should not be to destroy them altogether, just to control them and keep them out of your home. Damage from ant can also vary by type. Most are just a nuisance, carpenter ants however nest inside wood and can lead to weakened wood structures. Any sign of ants may raise concern in another area, aphids, which are detrimental to your plants. Ants have a special relationship with aphids. Aphids secrete honeydew, which is excess of sugars that ants feed on. Ants also protect aphids from other predators in the garden. So watch for any decline in your plants and look for aphids.  Your local lawn and garden centers, including most hardware stores, carry chemicals labeled to control ants without harming nearby plant material. There are a variety of control agents for different ants species, garden ants and/or carpenter ants, which are a little harder to get rid of. If the chemical is not labeled for use on or around a specific plant, try to avoid hitting the plant directly with the material.