The magnet is activating the reed switch for the time/seconds display. Try experimenting with the position of the magnet, twist it around, spin it upside down, move it around on the case back, try a different magnet, there is normally a sweet spot which when found will advance the minutes but it can be tricky to find.
Good luck and welcome to the forum.