All products will be upgraded to 4.00 after the official nopCommerce 4.00 release.
Which version are you using? We have updated the plugin and now the performance is improved.
If you are using the latest version of the plugin for nopCommerce 3.80 or 3.90 you can limit the shops in the shops' list by a small customization in the JS files. On the map, all shops will be visible.
Please note the search by tag and by distance will work only with the visible shops (in your case only 50 shops).