You're thinking a lens that protrudes into the camera behind the mounting flange might interfere with something in there? No concern there with the CLE... people hear about such interference with the Leica/Minolta CL, and that is caused by a moving arm holding the meter cell. This does not apply to the Minolta CLE, a totally different camera. A marvelous little camera! You'll be fine with the 35 Summicron ASPH.
Edit: But, with the 35mm lens the CLE will show only the 28mm framelines... as mounting your 50mm lens adds the 40mm framelines in addition.