To avoid mess, and putting out big big bucks, get a camping candle lantern. They use the 1" diameter plumbers candles, are glassed in, the wick and candle have a receding cap to promote efficient burning, and come with a handle and sometimes a hook.
Works great! We've also caved on (really cheap!) beer can candles:
Take a beer and drink it. (yeah, a 12 oz soda can will work, too).
Remove pop top tab.
Take a tin snips and cut the can off 1/2 of the way of its height.
Punch a hole in the bottom for a 1/2 inch white candle. Poke candle through, with the can forming a cup around the candle.
If you have a second candle (they come in paks of 4), a friend and some tinfoil, line the top of the can with tinfoil, punch hole in foil through the former drinking hole, and insert candle.
You now have two, cheap, drip controlled cave candles.
Light and enjoy. But carry spare candles.