Yep I remember that pain :-( And I promise it does go away if you stick with it! As long as you continue to put baby to the breast you will still produce milk. The same thing happened with my first except eventually (around 6 weeks) my daughter would only feed from the nipple of a bottle and not my breast - so a slightly different scenario.
Perhaps the soreness is from incorrect latching? It may be that baby is getting confused between the two nipples and latching incorrectly to your breast?
You will come out the other side I promise! If you can bear with the pain (it has always been short-term for me) things will only get better. I think we all have doubts as to whether we produce enough milk at some point.
As to your question of upping your milk supply - it can and will happen if you keep putting baby to the breast :-) All babies go through growth spurts where they seem to feed continuously for a couple of days - this is to up your supply to meet their new demands as they grow. And having a baby attached to your breast for that long can absolutely cause sore nipples. Keep applying a lanolin based barrier cream - these always worked best for me.
Ultimately you know what is best for you and for your baby. Good luck!