graphite deployments

Hi Is anybody on the list using a large scale graphite deployment? The company I work for is thinking of deploying graphite across our production infrastructure (3000 machines in 20 DCs) and I was hoping to pick someone's brains for layout hints, tips and gotchas. TIA Ashley