The project had been criticized for proposing too many changes to the area directly surrounding the original Spanish El Presidio settlement, a National Historic Landmark that was built in 1776.
"In calling off an effort that began with acclaim but turned into the city's fiercest development battle in a decade, the family holds open the possibility it might still try to build a home in the Presidio for its collection of work by such artists as Andy Warhol and Alexander Calder.
But the Fishers also say they are open to looking outside the city - and the Bay Area - before deciding what to try and do next."