So besides the fact that i can obviously tell when lifting that one side of my body is stronger than the other, I looked in the mirror the other day and could actually notice a stark physical difference as well, which kinda bugged me out. Any suggestions on ways I could alter my workout to try and even up my left and right sides (mostly talking about arms) ?

i have the same problem. i have been doing one armed bench presses on the smith machin to even my chest out. it's working pretty well, almost even

Unilateral work will help, so will multi-group free-weight exercises like barbell squats.