Simon grew up in New York City during the Great Depression, with his parents' financial hardships affecting their marriage, giving him a mostly unhappy and unstable childhood. He often took refuge in movie theaters where he enjoyed watching the early comedians like Charlie Chaplin. After graduating from high school and a few years in the Army Air Force Reserve, he began writing comedy scripts for radio and some popular early television shows. Among them were Sid Caesar's Your Show of Shows from 1950 (where he worked alongside other young writers including Carl Reiner, Mel Brooks and Selma Diamond), and The Phil Silvers Show, which ran from 1955 to 1959.