Swapping LEDs wouldn't cause that, unless you were wild with the replacing of them and have some solder splash somewhere it doesn't belong, or some part was knocked off the board in the process. Either the headset connector is causing the issue, or the MCU (big IC) is most likely the problem. I've yet to see a cold solder joint on the headset connector, and that is a valid thing to check out, but I have seen them messed up internally where a contact is 'sprung' and doesn't make a good enough connection, same problem as a cold solder joint sometimes, but different cause.
There's only a few parts between the MIC and the MCU, R10, R55, C10, C17, so make sure those are all in place or don't have any solder splash on them. It could even be a bad solder joint on the MCU causing it, really hard to tell without poking around in there a bit.