Overview
You'll need to enable IMAP to use your gov.uk mailbox in 3rd party apps such as Apple Mail, Outlook, Thunderbird or K9-Mail.

If you do not follow this step, connecting to Outlook or other 3rd party apps will not work
Method
To enable IMAP for your account;
- Log into webmail
- Click the Settings cog icon in the top right
- Scroll down the settings to "Mail Accounts"
- Scroll down to the "IMAP" section
- Tick "IMAP Access"
- Note down the Server Hosts, Ports and Modes. You'll need those to add or modify your account settings in your apps.