It's possible your settings didn't save correctly. Try toggling the setting off and on again, then click "Save." You may also need to refresh your browser or clear your browser's cache and cookies. If the issue persists, you may be using an older version of OWA where the setting is not available.