[neomutt-users] Figuring out IMAP w/O365, xoauth2, and pizauth
ajs at anvilwalrusden.com
Tue Apr 25 19:43:34 CEST 2023
I've made a few desultory postings about this before, but I don't seem to make any progress debugging it, so I thought I'd try to summarize in one place. Note that I am treating all these issues as potentially related, but they may not be.
I have neomutt configured to connect to more than one account. To achieve this, I use various hooks to configure the variables appropriate to that account when changing mailboxes. I use it on MacOS.
I access both mailboxes via IMAP and (SMTP) submission. The first is a bog-standard dovecot+postfix configuration that currently still uses password-based login. That mostly works flawlessly. The other is an Office365 Exchange service using the O365 implementation of IMAP. Authentication is via xoauth2.
I tried to use the mutt_oauth2.py mechanism recommended in the manual, but while I was able to get it to work on its own it seemed to present a lot of trouble once I tried to integrate it with all the hooks for account switching. Fortunately, I found pizauth (https://github.com/ltratt/pizauth), which seems to be working for me. But there are a few issues.
First, it seems that neomutt interacts badly with the system going to sleep while in a mailbox index. It appears that the sleep mode disconnects the network connection, and when reawakened neomutt doesn't know what to do with the now-defunct connection it had. If there were no mails marked deleted but where the mailbox is disconnected, then _usually_ one can get reconnected to the dovecot-hosted mailbox by cycling through accounts. Neomutt tries to use the connection, fails, and then authenticates again. This is about 50% reliable, also, on the O365 connection: the rest of the time, it claims to reauthenticate but there are no mails in the relevant inbox, and you have to restart neomutt to get back to a functioning mailbox.
If the mailbox had marked-deleted but unexpunged messages, however, I get stuck: trying to leave the affected mailbox results in an expunge confirmation, and then since the connection is dead the expunge fails and I get into a confirm-expunge, fail loop (it doesn't matter whether I say yes or no to the expunge question).
The IMAP connection to O365 also seems to be flaky. It disconnects from time to time without any real explanation. I have tried fiddling with the timeouts on both the IMAP and pizauth systems, and none of this seems to matter.
In both of the latter cases (the expunge-loop problem and IMAP confused on O365), I often find I have to eXit in a forced way. Perhaps 30% of the time when I do that, neomutt reports a ssegfault. I won't dump the entire system log report in here, but messages tend to look like this:
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000008
Exception Codes: 0x0000000000000001, 0x0000000000000008
Exception Note: EXC_CORPSE_NOTIFY
VM Region Info: 0x8 is not in any region. Bytes before following region: 105553518919672
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
MALLOC_NANO (reserved) 600018000000-600020000000 [128.0M] rw-/rwx SM=NUL ...(unallocated)
If there is more that I can do to try to help diagnose this issue (these issues?), I'm happy to try. Previous efforts have not led me to anything conclusive, but I guess I'm vaguely hopeful that a segfault might provide enough info to suggest what's happening.
Thanks again to the developers for the work on this.
% neomutt -v
Copyright (C) 1996-2022 Michael R. Elkins and others.
NeoMutt comes with ABSOLUTELY NO WARRANTY; for details type 'neomutt -vv'.
NeoMutt is free software, and you are welcome to redistribute it
under certain conditions; type 'neomutt -vv' for details.
System: Darwin 21.6.0 (arm64)
ncurses: ncurses 6.4.20221231 (compiled with 6.4.20221231)
libidn2: 2.3.4 (compiled with 2.3.4)
OpenSSL: OpenSSL 1.1.1t 7 Feb 2023
PCRE2: 10.42 2022-12-11
storage: tokyocabinet, lmdb
Configure options: --prefix=/opt/homebrew/Cellar/neomutt/20230407 --sysconfdir=/opt/homebrew/etc --autocrypt --gss --disable-idn --idn2 --lmdb --nls --notmuch --pcre2 --sasl --sqlite --tokyocabinet --zlib --with-lua=/opt/homebrew/opt/lua --with-ncurses=/opt/homebrew/opt/ncurses --with-ssl=/opt/homebrew/opt/openssl at 1.1 --with-sqlite=/opt/homebrew/opt/sqlite
Compilation CFLAGS: -D_ALL_SOURCE=1 -D_GNU_SOURCE=1 -D__EXTENSIONS__ -D_XOPEN_SOURCE_EXTENDED -I/opt/homebrew/opt/libgpg-error/include -I/opt/homebrew/Cellar/libassuan/2.5.5/include -I/opt/homebrew/Cellar/gpgme/1.19.0/include -I/opt/homebrew/Cellar/neomutt/20230407/include -I/opt/homebrew/include/lua -I/opt/homebrew/Cellar/neomutt/20230407/include -I/opt/homebrew/Cellar/neomutt/20230407/include -I/opt/homebrew/opt/sqlite/include -I/opt/homebrew/Cellar/neomutt/20230407/include -DNCURSES_WIDECHAR -I/opt/homebrew/Cellar/neomutt/20230407/include -I/opt/homebrew/Cellar/openssl at 1.1/1.1.1t/include -I/opt/homebrew/Cellar/libidn2/2.3.4_1/include -I/opt/homebrew/Cellar/pcre2/10.42/include -I/opt/homebrew/Cellar/neomutt/20230407/include -I/opt/homebrew/Cellar/tokyo-cabinet/1.4.48/include -I/opt/homebrew/Cellar/neomutt/20230407/include
+attach_headers_color +compose_to_sender +compress +cond_date +debug
+encrypt_to_self +forgotten_attachments +forwref +ifdef +imap +index_color
+initials +limit_current_thread +multiple_fcc +nested_if +new_mail +nntp +pop
+progress +quasi_delete +regcomp +reply_with_xorig +sensible_browser +sidebar
+skip_quoted +smtp +status_color +timeout +tls_sni +trash
+autocrypt +fcntl -flock -fmemopen +futimens +getaddrinfo -gnutls +gpgme
-gsasl +gss +hcache -homespool +idn -inotify -locales_hack +lua -mixmaster
+nls +notmuch +openssl +pcre2 +pgp +sasl +smime +sqlite +sun_attachment
ajs at anvilwalrusden.com
More information about the neomutt-users