To overcome this problem, you'll either have to use an "Extended Validation" or an "Organization Validation" code signing certificate (see below). However, note that if you release an updated version of your app, then you'll also have to request a new review again. You need to have a Microsoft account to submit your app for review.
If the review was successful, the Microsoft SmartScreen warnings will go away faster, or sometimes even instantly (it worked instantly for one of my own apps). According to Microsoft, this will help developers to "validate detection of their products". Microsoft allows software developers to submit a file for malware analysis. Option 1: Submit your app for malware analysis to Microsoft
Read on for the details about these different options.