In November 1950, Moore joined the service and remained enlisted for more than 34 months. Stationed in Fort Campbell, Kentucky but later traveling to Korea and Berlin, Rudy would often perform for his fellow service men who nicknamed him "The Harlem Hillbilly" for his R&B style of singing country songs. One night while putting on shows at the service club, the following act was slow to come out on stage. The crowd began to yell for something to happen, so Moore performed a comedy routine he had learned from Caldonia Young, a Cleveland comedian whom Moore would watch a lot. The crowd loved it and at this point, he decided to pursue his comedy career.

After the service, he continued to perform and moved to Seattle, Washington to visit his family and wound up recording some songs for Federal Records. The most famous of these recording sessions was "Step it up and go." Although Moore is singing career did not hit the big time, he did deliver some truly great soulful rock n' roll songs in a similar vein of Richard Berry. Remaining the "Turban headed prince of the blues" up through the late 1950's, Moore decided to perform under his real name and moved to Los Angeles, California in 1959.