Do MP3 videos significantly increase the size of SCORM? (compared to embedded YouTube videos)

The .MP4 videos will increase the package size since you're inserting them into the course. The video size is dependent on a number of factors. I'd insert some and then test to see the impact.

You can always make the video size smaller before inserting. For example, you don't need 4k and can probably get away with something smaller than 1920x1080 and still have good quality.