[neomutt-devel] [PATCH] add -Werror and -pedantic build flag; fix warnings

[toogley]

On Sat, Mar 06, 2021 at 02:41:55PM +0000, Richard Russon wrote:
> ASAN only reports after running NeoMutt.
> Running NeoMutt requires a complex set up of config, data and servers.
> Running all that in a CI requires containers, e.g. Docker.
> 
> That would be brilliant if we had that,
> but it's a HUGE amount of work to set up.

Yes i understand - but i also thought about running tests with ASAN?
would that make sense?

> > i have been running locally with ASAN enabled for some weeks and
> > continuously receive many memory leaks.
> 
> Again, these are hard to replicate by automatic testing.
> If you get leaks, you need to tell us about them.
> ASAN will tell you the call stack that lead to the memory allocation.

Ah yes, my memory leaks are in many cases (but not all) in IMAP/SMTP.

> > I thought making CI stricter (for instance also enabling
> > -fsanitize=undefined, -fcf-protection=full -fstack-protector-strong
> > -fstack-clash-protection and such things
> 
> Yes, modern compilers have lots more options.
> We could add some more to the automated builds.
> (Note that CIs often run on older stable releases)


So at least i think the we have now the consensus for the CI. Maybe i
can come up fix a patch for that or something.

> > making neomutt more deterministic by hard-failing if something's wrong:
> 
> No.
> 
> Through our history, we've been making some big changes to improve the code.
> Occasionally, this has led to NeoMutt being unstable.
> Once a project gets a reputation for crashing, it's game over.
> 
> Crashing on bugs, can be very helpful for developers (if reported correctly)
> but it will drive away users.

Yes i understand, but this is also was i was saying about OpenBSD. (see
also Stuart's comments about OpenBSDs mmap). In OpenBSD you cannot
disable these things easy. Programs just crash if the slightest thing is
wrong. And i think this *is* the reason why OpenBSD is so stable:
no/very little hard to reproduce bugs, no hidden errors. This is the
reason why i was saying OpenBSD uses these hard failures as a utility
for bug-fixing.

So i'd suggest to at least before every release create Beta releases
with as much as of these sanitizers/stack protections and stuff turned
on, including ASAN. With CI on OpenBSD as well as FreeBSD and others.

Won't be pretty for now as neomutt's code is bad/hard to test, but i
think it is better.

Ah and this is also the reason why i was thinking about increasing the
release cycle. Automated tests are bad in neomutt, we need to refactor.
Refactoring is difficult because of fear of introducing bugs. Bugs
cannot be prevented by harsh testing because of the lack of good
automated tests.

=> radical refactoring, many alpha/beta-releases and good stable
releases that we have trust in because we tested things properly.

So this is only a suggestion as you are both more experienced and more
responsible for neomutt, but i think this could also improve your live
as developer of neomutt. Being radical is good, sometimes?