Thanks guys, so I enabled nvidia nvenc in the encoding settings. That way it used the gpu, instead of the cpu. I only did 4, cause I didn't feel like opening more browsers, but at least this time things were running fast, while encoding. Maybe devs have a setting where they don't get d/c for idling? Otherwise you would have to move the screen a bit every 5 mins or so.
Note: You have to disable hardware acceleration in order for the image to show up on obs.