Brady was born in Manhattan, New York City, the son of James Cox Brady, Jr., and his wife, Eliot Chace. He was named for his paternal great-uncle, businessman and philanthropist Nicholas Frederic Brady. His great-grandfather was industrialist Anthony N. Brady. He grew up on an estate in Far Hills, New Jersey. After graduating from St. Mark's School, Southborough, Massachusetts, Brady attended Yale University (B.A., 1952), where he was a member of Chi Psi fraternity He received his M.B.A. from Harvard University in 1954.
He was appointed on April 12, 1982, as a Republican to the United States Senate to fill the vacancy caused by the resignation of Harrison A. Williams, Jr. He took oath of office on April 20, 1982, and served from April 12, 1982, until his resignation on December 27, 1982 and did not seek election in 1982. He resumed banking and business interests in New York City; chairman of the Presidential Task Force on Market Mechanisms (Brady Commission), 1987; Secretary of the Treasury in the Cabinet of President George Herbert Walker Bush 1989-1993.