Draw a grid on paper, and write one number in each box. Get a coin or other object to use as a die. (A real die can be confusing with this game.) The player who tosses the "die" gets to choose something for the other players to do with X, where X is the number in the grid where the "die" landed.
Things to do: X jumping jacks, frog jumps,or steps; build a tower with X blocks; say a silly word X times