In September 1871, as John Wesley Powell was leading his second expedition down the Green River, Powell and Stephen Vandiver Jones hiked from the river to investigate the San Rafael Reef. Another expedition member, Walter Clement Powell, wrote in his journal that the Indians called the strange formations "Sau-auger-towip" or Stone House Lands.

The town of Green River sprang up in 1878 at the Old Spanish Trail crossing of the Green River. Soon afterwards, in 1883, the Denver and Rio Grand Western Railroad track was built to Green River and Price. Conductors began pointing out to passengers the same ragged skyline of the San Rafael Reef where Gunnison had imagined temples, buildings, and ruined cities, and they called it "the silent city."

On 21 April 1897 Butch Cassidy and Elza Lay robbed the coal mine payroll at Castle Gate, Carbon County. They galloped their horses into the Swell and escaped. They were only two of probably dozens of outlaws who found a haven there over the years.