Caroline's maternal grandfather was J. C. Caroline, who was a running back for Illinois and led the nation in rushing as a sophomore in 1953 before playing 10 seasons for the Chicago Bears, primarily as a defensive back. His father, Simeon Rice, was an All-America defensive end at Illinois in 1994 and 1995 and was picked third in the NFL draft and played 12 NFL seasons. His mother Jayna Caroline, was a track athlete in high school and graduated from Illinois. Growing up, Jordan never felt any pressure to play football though did dabble in the sport as well as swimming, though he focused on basketball. In middle school he played on the same AAU program, Team Trouble, as future Loyola-Chicago forward Donte Ingram. Caroline went to Edison Middle School in Champaign, IL. Caroline played two years of high school basketball for Champaign Central High School before transferring to Montverde Academy. Playing alongside Ben Simmons and D'Angelo Russell, Caroline helped the team to back-to-back national titles in 2013 and 2014.