Constant Crashes with Memory Leak

Since 4.0 The game constantly disconnects and crashes every 20 minutes. No error message. The game gets slower and slower, the FPS gets less and less, untill I'm getting only a few frames. Then black screen, then get a disconnected message.

Then the game auto loads back into the map with the grinding gears image, the map is reloaded and lag is gone, then over 20 minutes the frame rate gets worse and worse and the lag is unbareable, then it goes black and repeats.

Sitting in my hideout is fine, but when Im in a fight with a large party and theres lots of fire and explosions (caused by other players, mine is fine) the game crashes, goes to black, and repeats.

While the game is running the computer gets worse and worse, untill I restart the game application entirely, then computer runs fine while playing untill the crashes start happening again.

Computer:
15th Gen Ultra 7 CPU
32GB G.Skill RAM
ASUS ROG RTX 4070
Samsung 990 4TB SSD

I Did some more testing. I looked for a solution, found some advice which was to clear cache. And enable features in windows for gaming (Already turned on)

I played single player for an hour, I did a vaal temple run. I didnt crash but I was getting frequent frame drops and lag spikes (ON SINGLE PLAYER VAAL TEMPLE)

I Checked my RAM before play, I was using 8 GB with just windows, 9GB with browser runing. I played the game, RAM usage was 15GB. I closed the game after running the vaal temple. RAM usage was still 15gb.

I ran another game (Rimworld), RAM usage went up to 16GB+. Then I closed Rimworld because it was running like shit and my web browser. RAM usage was still 14GB with just windows.

POE2 is not clearing its self from RAM.