Combine dry milk powder, sugar, and cocoa powder. Add water and vanilla; stir well to dissolve. Cover; cook on low setting for 3 to 4 hours or or high setting for 1 to 1 1/2 hours. Before serving, carefully beat cocoa with rotary beater to make frothy. Ladle into mugs; top with marshmallows and sprinkle with cinnamon, if desired. Makes 9 (6 oz. each) servings.