Thank you, Martha, and how do you like that new ankle bracelet?
Seriously, I had a look, and it turns out that this battery behavior is not only normal, but a designed-in feature.
If you look down the battery compartment, you'll see a dark, roughly triangular wire catch about 2/3 of the way down the side. This catch is positioned so that it latches into one of the slots that runs part of the length of the battery. When you open the battery door, the battery slides down until the catch reaches the bottom of the slot; then it holds the battery there until you pull hard enough to overcome the pressure of the spring.
A thoughtful touch, really, since otherwise if you opened the battery door with the bottom of the camera facing downward, the battery would just fall right out! Think of it as a modern version of the Contax II/III takeup spool retaining spring (the one they left off the IIa/IIIa, to the consternation of spool-droppers everywhere!)
Sorry for the slightly blurry pic, but I had a hard time getting any light down the battery compartment AND holding a tiny digicam steady (had to use a tiny one because anything larger blocked the light!)