Elias Lyman Magoon, 1853: Vassar College trustee, 1861-1886