In the late 1600s, the land, Middletown Village was the heart of Monmouth County's first European settlement. By 1668, the village had been laid out into thirty-six "home lots" along an Indian path that later became Kings Highway. These lots were used as family farms and passed down through the families until 1890. In 1890.

In 1928, the Danglers constructed the two-story Colonial Revival Mansion on the hilltop. Then, in 1935, the reputed crime boss Vito Genovese purchased the property and re-fashioned the grounds in a pseudo-Italian style reminiscent of Naples, Italy, where he was born. Caruso Construction Company of Atlantic Highlands carried out the construction and Lovett's Nursery of Little Silver planted the gardens according to the plans of Theodore Stout.

Stout, who wanted to create "something big enough to make an impression from the top of the hill," was given free reign in designing the garden. Mr. Genovese's only requirement was the Mt. Vesuvius rockery. The design was a mixture of English and Italian styles.

In 1937, an unexplained fire destroyed the mansion. In 1948, the property was sold to Dominic Caruso who conveyed it later year to Mary Gladys Cubbage. On December 31, 1953, Karl and Marjorie Sperry Wihtol purchased "Deep Cut Farm" and erected the existing house to replace the one that had burned down. The Wihtols also renovated the greenhouse and worked on the gardens.