I don't think this statement really holds true.
If there are a fix number of sellers say 10 then for those number of sells there can be relatively infinite number of buyers.
For Example
10 Sellers total 1,000,000 Shares
1 Buyer total 1,000,000 Shares
10 Buyers each 100,000 Shares
100 Buyers each 10,000 Shares
1000 Buyers each 1,000 Shares
but you are correct they are both only potential buyers and potential sellers.
What does it matter how many buyers there are?
All that matters is how many shares are traded and at what price.