Collect toilet paper tubes. If you would like you can spray paint them.
Take a knife and make four slits in the toilet paper tubes as shown.
Take craft sticks and have the children insert them into the slits to create a fence. These can be put together and taken apart to make fences.
This would work great with the Clothes Pin Horses.