I had an M3 which had the similar effect with a 90mm Summicron, but which was fine on my M2. All other lenses were fine.
I think with wear and age the original tolerances have weakened. If the lens is loose I'd get it checked though.
Leica M2, Nikon D700, Bronica RF645, Leica CL, Summicron 40mm, Rolleicord Va, Hasselblad 500 CM Zeiss Planar, Leica 50mm Summicron V3, Hasselblad PME51 metered prism - all gone.