So great a god, so small a mount. But this mouse was once a gandharva.In Indra's court Krauncha trod on sage Vamadeva's foot. Cursed, he became a giant mouse and began wrecking Parashara's ashram.Ganesha threw his noose and bound the mouse. It begged — do not kill me. Ganesha said: then carry my weight.The mouse shrank and became his mount. Devotees read it so: the mind's restless greed is the mouse — let wisdom ride it.
Source
Ganesha Purana; Mudgala Purana (versions of the story differ)