Collider gets sometimes stuck at wall

So I tried to find a solution for days but I didn't find anything that covered that specific issue:

My player circle collide SOMETIMES gets extremely close to a Wall (it gets detected on OnCollisionStay2D). When that happens, I can still move verticaly (jumping) but I can't leave the wall. If I slighty change the x position, it can leave the wall.

My wall is a simple large Box collider 2D and the player moves like that:

 horizontalInput = Input.GetAxis("Horizontal");
 body.velocity = new Vector2(horizontalInput * speed, body.velocity.y);

I don't know if it has something to do with friction of if it's something else :/ Any help is appreciated :)