"Swooping" flight

I know that there are a lot of tutorials for flight physics floating around, but I've been struggling to replicate a specific type. I'm trying to recreate the Mario 64 style, where you have to "swoop" to gain altitude, and otherwise start to stall when facing upwards. My current code looks like this

         var vertVelocity = rb.velocity.y;
         var forwardVelocity = rb.velocity.z;
 
         if (vertVelocity <= 0)
         {
             rb.AddRelativeForce(Vector3.forward * airSpeed * 2f);
         }
         else if (forwardVelocity > 50f)
         {
             rb.AddRelativeForce(-Vector3.forward * airSpeed * 0.5f);
         }
 
         float rotateHorizontal = Input.GetAxis("Horizontal");
         float rotateVertical = Input.GetAxis("Vertical");
         transform.Rotate(Vector3.right * rotateVertical);
         transform.Rotate(Vector3.up * rotateHorizontal, Space.World);

This has a lot of problems. It behaves erratically, sometimes shooting me off in random directions. It doesn't preserve local velocity, so when I turn around it takes time decelerating before it starts moving in the new direction. It also just doesn't work; I can't swoop with it. Your gained speed from swooping is lost the moment that you turn upwards slightly. Currently, I'm thinking that I might need to have a "base" speed that it tries to return to, and then otherwise just adds and subtracts within a range. Any help or advice to set me on the right track?