Work the fast twitch muscle fibers and stretch a lot so you don't lose your flexibility.
Of course, you can have large muscles and still be fast. Look at Brock Lesnar.
For explosive power, use correct form with a weight that you can move quickly. Bodyweight exercises are best. Do push ups where you push yourself into the air, clap your hands before landing and then repeat. Exercises similar to that will help with explosive power.