Avoid large amounts of saturated fats. Dietary cholesterol has very little effect on blood plasma levels. High levels of LDL cholesterol are mostly a result of large intakes of saturated fat. Always avoid trans fats. Saturated fats should make up no more than 20% of your daily fat intake. This doesn't mean you should remove them entirely because testosterone is created from saturated fat.
Increase your fiber content to at least 25g a day.
Exercise is always a plus too!