I'm not sure if I had a PSU failure, but the power brick's red button does pop up when plugged in.

The thing is, even if I unplug the brick to the Xbox it still pops up.

Does the pop up feature depend on some kind of feedback mechanism from the console to ensure the system is working (in which case I would expect it to pop when not connected to the Xbox and my unplugged test is meaningless) or not (in which case I think my brick, and not my PSU, has the problem)?

If you're talking about the replacement power cable with the red and yellow buttons, one of mine does that too, you just have to hold it in (but not all the way down) until the light stops blinking and you're good.

Wow that actually worked!

Its stuck on solid red, though, now. Even though it works is that safe?