As someone who started with similar issues. Face pulls, Deadlifts and pushups.
Odds are your pecs, upper traps and anterior deltoids are tight. Face pulls will help lots with the shoulders and mid traps. Deads for lumbar and lower back will pull you upright. pushups work the serratus anterior, which stabilizes the lats.
These are the big ones that are helping to fix my back and shoulder issues.