Gems are the bread and butter of this game. Luck will land you some or all of these gems, but over time you will learn each and every one of them. Knowing where to focus all of your gems is something you have to experience, but a good rule of thumb is to try and get the best gems in the center of your maze for maximum coverage.
A tower's base damage isn’t final, towers level up every 10(25 in swarm) kills they make. There are 12 possible gem levels and each level they gain gives them 10% bonus damage. This bonus caps out at 120(300 in swarm) kills.