Is there a way to delete a file uploaded to the blacklist/whitelist?

There are numerous ways to achieve this.
Through the List Widget, upload the same list to the normal state to cancel the previous upload.
The other way to do this is to delete the uploaded entries manually or through bulk selection through the same List Widget. 
Alternatively, you may also use the Lists API for bulk delete, more information on this read our API reference.