OrderlyEmails does need to be installed for a few of the more "advanced" features to continue working correctly.
The Recommended products section: They would stop updating if you uninstall the app.
The Email click tracking: The emails push conversion and tracking data to your Shopify reports. This would stop working if you uninstall the app.
Other than that, everything else would continue working. OrderlyEmails does not actually send any emails itself, nor trigger the emails to be sent.
The app is a design tool that upgrades the emails that Shopify is sending for your store. Therefore your store continues to work exactly the same as before, just now with much better email designs.