Snow is nothing more than a large collection of ice crystals. To get ice crystals to stick (pack) together, the crystals need to melt. On a day when the temperature outside is warmer and the sun is out, the snow on the ground will likely be melting somewhat. This makes it easy to grab a handful of snow and pack it together in a snowball. However, when it is colder out, the ice crystals are not melting and the snow doesn't stick together.
There are ways to pack white fluffy snow that won't stick. Grab a handful and try to squeeze it as tight as possible. This creates greater pressure on the ice crystals, making it more likely they will stick. You can also blow hot air on the snow to increase its temperature and cause some melting.
Now head outside and make some snowballs!!!