Milford is located in a broad valley a few miles east of the geographical center of Beaver County. Originally, Milford was nothing more than a few shacks built on the hills near mines under excavation. Many of the miners who first came to Milford left within a few years after trying their fortune, but others came to stay. Arvin Stoddard was the first settler of the area, claiming 160 acres of land in 1880, building the first house in the area, and planting the first trees. During this same time, prospectors were searching the hills to the west and southwest for lead, silver, and gold. A Welsh smelterman, John D. Williams, came to Milford in 1880. He eventually built a smelter on land adjoining the Stoddard claim. Some contemporaries described Milford as a "perfect mudhole," or the "perfection of desolation."

Cattle-raising was also important in Milford's development. In the early 1870s three brothers settled at Pine Grove in Pine Valley west of Milford and established a cattle ranch. Within a few years several cattle companies had stock grazing in the land surrounding Milford. B.F. Saunders of Salt Lake City owned Utah's largest cattle herd--the Pike Springs Ranch--and he made Milford his shipping point. Cattle grazing was possible on nearby public domain land year round. Meadow grass covered the Beaver and Milford valleys from Hay Springs to Black Rock and supported as many as 20,000 head of cattle and 5,000 head of horses.