You need to check the status of the RecoverableItems subtree. When you permanently delete items, they actually end up in a hidden folder within the RecoverableItems subtree, where they are kept for the duration of the SingleItemRecovery period, or for the duration of any hold(s) applied to the mailbox. Depending on your situation, you have to follow the instructions in one of the two sections in this article: https://learn.microsoft.com/en-us/exchange/security-and-compliance/recoverable-items-folder/clean-up-deleted-items
Just to save you some headache, you might try doing the deletion via the webmail/OWA first. If you observe the same behavior therein, you (or an admin within your organization) will have to follow the steps in the article above.