Anything they can climb on and jump off of safely, but not near a fence that they can launch themselves over with it.
My buck loves to bonk a basketball around his pen. It was in there for almost a year until he figured out how to play with it. He also likes to wear his feed pan on his horns like a hat and will keep busy with that for an hour.
My herd queen loves large branches on the ground in the paddock. She picks them up between her horns and rubs and rubs and rubs.
I put a plastic drum in the kid pasture so they could crawl in and sleep if they wanted to get out of the wind and as soon as they laid eyes on it they forgot I existed. They spend hours jumping on and off of it and crawling in and out of it. A couple of doelings really enjoyed "log rolling" on it and it would be all over the pasture when they were here.
All of my goats love scrub brushes and a push broom head screwed to the barn wall (some in a doorway, too) at the perfect height to rub their bodies and heads against. They also had a rubber welcome mat attached to the wall that they loved....took it down to reconfigure a stall, gotta get it back up. They really groomed themselves quite nicely on those things.
mine have a toddler playscape they LOVE. Just like this one: http://www.mylittletikes.net/Images/waveclimber.jpg
You can find them pretty cheap at garage sales and such.
I also recently put old large tractor tires in their yard (one laying flat on the ground and the other leaning against the first one) and they like to climb and walk around on those.
We also have a large rock for them to jump up on and an old wooden table...things like that.
You could easily stack cinder blocks and use 2X4 boards to easily build ramps and platforms for them to walk on and climb around.
climbing and jumping is a goal of a goat, the higher and more levels t he better.
Here is a little something we through in with some of our show whethers when they started getting bored. I have seen some people build some very fancy climbing areas, just make sure they aren't near your fencing and the goats can use them to jump out.
If you are going for a more natural look, a couple fallen trees/logs always makes a goats day.
I have large sandstone boulders that they love to climb on , jump from and even do somersaults off of.. yes thats right.. I had a kid that would lay down and purposely roll off the rock flipping over and then get up and do it again and again and again.. I caught her doing it the other day a couple times and she is now a yearling.. sometimes the simplest things make goats happy!