After successfully deploying Varnish2 on our solr cluster I decided to give it a try on the web nodes and use it to strip the analytics cookies off. The results have been impressive.
The above is the CPU usage on www3.drupal.org. As you can plainly see, the difference between Squid/No Caching and Varnish/Anonymous Caching was significant. While we are still resolving some issues after the switch, it has been a large success on the whole and has lowered the load on our web nodes across the board.