The framerate seems smooth enough, but the game reacts as if there's a slight delay from pressing the button (moving the stick) to the action.

Also, it takes a hard push of the trigger to even get a shot out.

Is this anything I should be worried about?

Its just how the game is. Takes a milisec or two to start those actions.