In Madison, Wright's father was a pastor of the Unitarian Church. Wright lived in Madison from about age 11 until about age 20. At age 11 Wright also began spending summers with his uncle James Lloyd Jones on his farm located near the Taliesin hill.

Those early years in Wisconsin had a profound effect on Wright: "As a boy," he wrote in his autobiography, "I learned to know the ground plan of the region in every line and feature... I still feel myself as much a part of it as the trees and birds and bees are, and the red barns."

Wright's father and mother divorced in 1885 and Wright never saw his father again.