Rust is best known as a stand-up and sketch comedian at the Upright Citizens Brigade Theater (UCB) in Los Angeles. At UCB, he is currently a member of the sketch comedy groups A Kiss From Daddy and Last Day of School. He has appeared in films such as Inglourious Basterds, Ass Backwards, Freak Dance, Paper Heart and Semi-Pro. He made his most high-profile acting role in the summer of 2009, playing the male lead in the film I Love You, Beth Cooper. Rust has written for television programs such as Human Giant and Moral Orel. Rust and comedian Charlyne Yi formed the band The Glass Beef in 2006. The two appeared together in a video for "Song Away" by Hockey. He is also the lead singer and bass player in the comedy rock duo Don't Stop or We'll Die with fellow comedian Michael Cassady. Until his death, the band also featured the late Harris Wittels on drums.