Fog Of War with line of sight - Tons of raycast or better solution?

So I have done some implementation of Fog of war in my game. 1. Placed a plane between the character and the camera. 2. Raycast from the camera to the player and find the hit point of the FOW plane. 3. Loop trough all vertices on the plane and turn the alpha to 0 for those within radius r of hit point.

Now I want to take this to the next step and only reveal areas that are within line of sight. My first thought was to draw a raycast from the character to the x and z position of the FOW and see if it collides with anything, but this becomes quite a few raycasts if done for all the vertices within the radius. So does anyone have a suggestion on how to improve this? Suggestion for making more efficient FOW is welcome too if you think there are more efficient ways of accomplishing what I am looking for.