You can flag as navigation categories the different categories you want to exclude from your Top Categories panel. 

Example

I want to exclude the categories /All Products, /More Sneakers, /More Sneakers/Other Brands from the Top Categories panel as they don't provide any relevant Insights to me as a retailer.

Under Account Settings, I will flag these 3 categories as navigation categories:

Learn more about how to flag navigation categories here.

As a result, the navigation categories are not automatically hidden from the Top Categories panel:

If you want to learn more about how Top Categories are generated, you can read our article here.