Dr. Fisher, a wealthy whaling ship owner, built his home to be the hub of Edgartown society in the mid-nineteenth century.