Yes, to do the job you need to remove the rangefinder from the camera body and dismantle it, some of the glasses in the finder are sandwiched with metal masks and other components, and these bits get removed before cleaning.
Three components have silvered surfaces, and that silver just rubs off VERY easily. Other glass components are much more robust and can be cleaned just like you probably already cleaned the outside surfaces on your viewfinder.
Once the job is done, with the rangefinder reassembled, it has to be adjusted for image alignment, vertical and horizontal. Not a job for the faint-hearted. It takes me about an hour to strip, clean, reassemble and adjust one.