Natural, not processed in any way, is by far the best food for the mouse IMO.
Also, make sure the mouse is not eating any plastic. Houses, wheels, food bowls etc in the cage - mice will often attempt to eat these. From my experience it eventually kills the mouse, generally from cancer.
Another one is the water. Let the tap water sit 24 hours in a jug before giving it to the mice - the chlorine will evaporate during that time.
I've had a lot less trouble with fish since I started only using water from the hot tap (let if cool first). Not sure if this offers any benefit to mice however, but presumably it kills any nasties in the water.