If you're doing what I think you're doing, don't worry. Are you referring to your settings in the "Options->Controls" menu? If so, the fact that it says Keyboard 1 whenever you go back to it doesn't matter. Try using your gamepad in single player and see what happens.