The nice thing about wide angle lenses is that "slow" is relative. I'll bet you can hold the Elmar 35mm as steady at the speed needed for f/3.5 as you can a 50mm lens at the speed necessary for f/2.8 or maybe even f/2.
The 35mm f/3.5 Elmar is a good lens that should serve you well if it's in decent condition to begin with. It also makes a very flat package. My father used to carry his in a watch pocket when he had the 90mm Elmar mounted on the camera.
Do not turn down the 35mm 3.5 Summaron, its a great lens. I was just looking at some B & W photos I took in the late 1970s And I really like the" Look' this lens gave me . The only japanese lens that comes close was the rare Topcor 35mm 2.8 in LTM. that I had borrowed, to try out in the early 1980s.