I did an experiment last year where I deliberately tried to 'spoil' chaga by innoculating it with rotten eggs and raw pork. I left the samples out for three months. Although I admit my microscope isn't the best, I couldn't see any signs of bacterial colonization. This is consistant with other reports that chaga is antibacterial. If you leave it out long enough, however, it can ferment or get moldy, but that happens really slow, if at all. And fermentation doesn't actually make it poisonous, it just changes the flavour.
How long can it stay out of the fridge? That varies. I've had chaga sitting on my counter for months and it's still fine. Other times I'll notice it starting to ferment and developing off flavours as quickly as 3 days. You're usually good for at least a week outside the fridge, though.