His parents were William and Camille Fleming. His mother later married William L. Gordon, and Charles adopted his stepfather's surname. He added the "e" to "Gordon" when he found that another Charles Gordon was already registered with Actors Equity (the stage actors union) in New York.