Use the mic at the conference.  And if you’re the emcee or organizer, it’s your job to make sure that the mic works and that folks use it.
I think that’s all you need to know, but here are some details if you’re interested.
Don’t just assert that everyone can hear. This is false. I say this as a person with a degree in hearing people and making people hear my voice in big rooms, and it barely works in ideal situations. At most conference spaces, there is noise in the hallways and from other rooms, noise from the air conditioning system, and noise from the participants. And the mic generates more even sound across many rooms than a speaker shouting from one corner.
Don’t just assume that people can hear. It essentially demands that people who need the mic have to make their accessibility needs explicit in front of other participants, and it usually involves them shouting about something barely heard in a situation when folks are expected to be quiet. The speaker and the organizer sitting right near them are specifically the least well positioned folks to notice issues with the audio, so they shouldn’t try to generalize from their experiences. And louder people using mics normalizes it for everyone, keeping mic usage from becoming a big deal, somewhat analogously to cis folks explicitly sharing their pronouns.
And even when everyone can “hear” the speaker without a mic, this will often involve more cognitive demand put into audio perception to filter out the background noise, which makes it harder to think about the ideas in the session!
Audio systems are never perfect, but using them appropriately is a bare minimum level of accessibility support that people should be able to trust when they show up to large events. There’s enough emotional labor involved in fighting white supremacist ideas at math conferences; we shouldn’t also have to fight to hear what folks are saying.
 Use the fucking mic.
Thanks to Grant Lakeland for his feedback and contributions on a draft.