Counterweights are mounted by nestling down into shaped seating platforms that are like a vee. The vee shape pulls the CW toward the frame of the truck so it fits relatively snug. Gravity helps with that.
But, there is at least 1 counterweight "mounting bolt" and sometimes more than 1 that prevent the CW from jumping upward and coming off on rough surfaces or collisions.
If you are detecting movement of the counterweight on its seating platforms, the mounting bolt(s) are not secure and need to be tightened right now, if....they haven't already fallen out.