Kelly was born to Charlene Cudzylo and Kevin Kelly, a supermarket manager and younger brother of quarterback Jim Kelly. He spent his early years in Western New York, his mother's home area, not far from where his uncle Jim was playing professional football. Chad Kelly was a consistent strong performer in Punt, Pass, and Kick competitions as a youth. His family moved to Red Lion, Pennsylvania after Kevin was reassigned to a grocery store in the area, but unexplained disciplinary issues had him removed from the Red Lion High School Football team and therefore prompted Chad to return to Western New York. As Jim Kelly had only one son of his own (Hunter Kelly), who died from congenital illness in childhood, Chad Kelly earned a reputation as Jim's heir apparent. Jim attempted to guide his nephew, but felt that Chad was not listening to his advice.