Strange mushrooms

My plant was looking pretty thirsty when I got back from vacation. Poor thing. I gave it some water and the next day these weird mushrooms started growing in the soil:

Some of the leaves seem to be dying now and I think the mushrooms might be to blame. Has anyone ever seen this before?