Kelly was born to Charlene Cudzylo and Kevin Kelly, a supermarket manager and younger brother of Pro Football Hall of Fame quarterback Jim Kelly. He spent his early years in Western New York, his mother's home area, not far from where his uncle Jim was playing professional football. Chad Kelly was a consistent strong performer in Punt, Pass, and Kick competitions as a youth, becoming national champion four times. His family moved to Red Lion, Pennsylvania after Kevin was assigned to a Wegmans in the area, but unexplained disciplinary issues removed Chad from the Red Lion High School football team, and he returned to Western New York. Others thought that Jim Kelly helped Chad's career. Jim had only one son of his own, who died in childhood, so tried to guide his nephew, but felt that Chad did not listen to his advice and more than once gave up on him. A football coach said, "Being Jim Kelly's nephew has been harder for him than it has helped him. A lot is expected of him because he’s Jim's nephew. He was never just 'Chad' or even 'Kevin Kelly’s son. ' He was always 'Jim Kelly’s nephew. '"