Network Connection Issues only in POE 2

My network connection is spiking like crazy in POE 2 only. I can play competitive shooters with no frames lost and be perfectly fine. Then I boot up poe2 and my network connection spikes like crazy. Constant stuttering and it is making the game nearly unplayable

Having the same issue. Seems like it started Monday and continued when I played last night. Didn't have any issues over the weekend. My character is constantly stopping moving and coming back before the game pauses itself. Sometimes it seems to improve if I portal to town and then come back. Seems like the FPS is better this league, but this is making Act 1 difficult to play.

I have a PS5 and been playing the early access since start. Stuttering was this bad only in 0.2 . In 0.4, with every patch released, it got worse. Today, it’s unplayable. I’ve finished the game and will not be playing tiered maps until it is fixed.

I’ve tried the following:

Upgraded the modem (it was really old)
Changed DNS
Tried a lot of different servers
Played with graphics settings just to try

none of them worked. btw, I’m not alone. We are playing in a group and all PS5 players are suffering from the same issues.

UPDATE
I live in UTC+3 timezone. In the mornings, up to 12pm - 1pm , game is quite playable and no issues. But comes afternoon and evenings, it's unplayable. I think all the gamers are from Americas and they are eating the bandwith of PS5 servers.