[neomutt-users] Neomutt: now featuring Lua scripting!

[Floyd Anderson]

On So, 30 Apr 17:50:28 +0200
Guyzmo <z+mutt+neomutt at m0g.net> wrote:
>On Sun, Apr 30, 2017 at 01:47:52PM +0200, Floyd Anderson wrote:
>> # Structure of mutt namespace
>>
>> Before the mutt namespace grows up, I think it probably needs to be a little
>> bit more structured, for instance:
>>
>>    _G.mutt = {
>>        const = { VERSION = '', QUAD_NO = 0, QUAD_YES = 1, … }, -- ①
>>        props = { … }, command = { … }, sidebar = { … },        -- ②
>>        index = { … }, pager = { … },                           -- ③
>>        event = { OnFolderHook = function(mailbox),             -- ④
>>                  OnSendHook = function(header), … },
>>        …,
>>    }
>
>I've added above some numbers as Lua comments for reference below.
>
>## ① Constants
>
>> This way constants like QUAD_* can be easier kept as read-only by users by
>> manipulating the metatable of ‘const’. Currently the expressions:
>>
>>    :lua mutt.QUAD_YES = 'The answer is 42!'
>>    :lua mutt:set('copy', mutt.QUAD_YES)
>>
>> results as one does ‘set copy = no’. But on the other hand, if you would
>> argue: “When a user put nonsense in, he should not wonder why nonsense comes
>> out!“, I would say you are right — it’s just only an idea.
>
>Well, I did not actually know you could protect a table that way. But
>then, why not expose all the constants at the `mutt` table level, and
>make that table constant?

Unsure on wich side you would apply this. Within ’mutt_lua.c’, no please 
don’t do it because users should have the ability to redefine e.g.:

    mutt.print = function(s, ...) mutt.message(s:format(...)) end

when they like. Also, I as a user like the freedom to shoot myself in 
the foot (linked with a lesson learned effect of course).
>
>Using the [protect function][1] we could do:
>
>    _G.mutt = {
>       VERSION = '', QUAD_NO = 0, QUAD_YES = 1, -- ①
>       command = { … },
>       …,
>    }
>
>    protect(_G.mutt)
>
>then all the members of the mutt table would be made const (but not
>their members).

This works on user side even it brakes unpack(mutt) for example. My main 
intention was/is to point out not bloating up the mutt namespace itself 
with all kind of constants (e.g. from ‘mutt.h’) on way to the final API.
>
>BTW I saw you're calling `mutt:set()` and not `mutt.set()`. As it is
>currently implemented the `mutt` table is merely a namespace and
>encapsulates no value. That means the `set()` and `get()` methods do not
>expect self as first argument.

That is some kind of bad habit, I admit. I use that often (without too 
much thinking) to visually isolate callable Lua functions from some kind 
of variables, properties, fields or what else.

>> Another one is to let NeoMutt call user declared event handler until the
>> queue of handler functions is empty or one of them returns true (means event
>> is consumed).
>
>I'd love to see a much tightly coupled integration of the event handlers
>in Neomutt and the Lua API, where from Lua you could list the registered
>events and modify that list.
>
>And also make it possible to register lua functions directly as hooks,
>without going through the parsed string representation.

Have a look into my attachment and the notes below about it. Generally 
it’s a little bit unclear to me how to register an event handler. 

Should the user define a Lua function like OnFolderHook(mailbox) and if 
NeoMutt founds it, it will be internally registered and called when the 
appropriate event occurs?

The terminal emulator rxvt-unicode (urxvt) does it this way with its 
Perl extensions [1]. In my attached ‘Class.lua’ script I push the 
fictional handler to _G (_G.mutt.events is probably a better place) 
where NeoMutt could find them without parsing any configuration or 
script files.

Or does the user register the handler actively by calling functions 
which will be provided by NeoMutt? For instance:
    local hookID  = mutt.AddEventHandler(ref_folderhook)
    local success = mutt.DelEventHandler(hookID)

[1] <http://pod.tst.eu/http://cvs.schmorp.de/rxvt-unicode/src/urxvt.pm#Hooks>

>> # Extending of mutt.set()
>>
>> Since mutt.set() is a function, it can probably do more by default for users
>> convenience and may also results in shorter, clearer Lua scripts.
>> For instance:
>>
>> 1. reset an option to its default by passing a nil (not in list) value
>>        :lua mutt.set('option', nil)
>> 2. set a boolean option to true and return its current value afterwards
>>        local current = mutt.set('option', true)

[…]

>From an OOP generalist perspective, the `set()` method is a procedure
>that shall return nil ; so that was my rationale when I built that
>method.

Yes, your are right and it’s totally okay for me. Sometimes it’s useful 
to get a response if a call was successful, like:

    optionB = set('optionA', false) ? false : true

or consider the case a user wants to reset some options without knowing 
all the exact default values. Those user must read the fine manual 
instead of simple doing mutt.set(option, nil) in a for-loop.

But as I said, all is fine. After some thinking I realised that I can do 
all that by redefine mutt.set() with Lua — as long as _G.mutt isn’t 
protected by NeoMutt internally.

>Though, what I would really have loved would be to expose the options
>within a table so that you interact with them like they are just
>variable, but actually the value they contain are in sync with the
>Neomutt internals. i.e.:
>
>    > mutt.options.visual = "vim"
>    > mutt.options.connect_timeout = 42
>    > mutt.options.mbox_type = "fubar"
>    stdin:1: error: invalid value for mbox_type
>
>That being said, is it something that *can* be done in Lua?

Why not? (theoretically) — IMHO the existing mutt.get()/mutt.set() could 
do their jobs in the metatable of ‘mutt.options’:

    mutt.options = {
        -- define some defaults
        visual='vim', connect_timeout=30, mbox_type='mbox',
    }
    setmetatable(mutt.options, {
        __metatable = true, --> all but nil protects the metatable
        __index     = function(t, k) return mutt.get(tostring(k)) end,
        __newindex  = function(t, k, v) mutt.set(tostring(k), v) end,
    })

or from the beginning:

    mutt.options = setmetatable({}, […same as above…])
    mutt.options.visual = 'vim'
    mutt.options.mbox_type = 'fubar' --> :1: Invalid mailbox type: ?

This is compact because your getter/setter validates all its parameter. 
I said theoretically above due to the fact that I’m a little bit off the 
track with the incompatible changes between the different Lua versions. 
As far as I know Lua 5.1 calls __index/__newindex only for absent keys. 
But with C/C++ within ‘mutt_lua.c’, all should be possible, or not?

>And would it be something expected by seasoned Lua users?

I would say yes. SciTE does it this way. It exposes ‘props’, a 
pseudo-table (whatever that is — I’m not a C/C++ programmer) 
representing all SciTE properties. Access/Assign like:

    -- emulate the ternary operator (x = a ? b : c)
    props['ext.lua.reset'] = props['ext.lua.reset'] and 1 or 0


>> These are just my first impressions with Lua enabled in NeoMutt. Keep up the
>> great work and let me know if I can help within the limits of my resources.
>
>Well, there are three aspects where help would be welcomed:
>
>1. Write scripts with the current API, to make cool stuff that can be
>   used for showing off 😀

I attached an archive with some small Lua scripts. They can be used as 
templates and a starting point for own tests or ideas. Currently it is 
only a skeleton and doesn’t configure or run any NeoMutt Lua API things.

But it provides some object inheritance and shows one way to register 
several event handler from the user side with Lua which can be called by 
NeoMutt when their definitions are valid. Also a working pseudo event 
handler (OnStartup) is implemented.

The main scripts are ‘neomutt-startup.lua’ (prepare and load all other 
modules) and ‘Class.lua’ (base class for inheritance). Extract the 
archive in an appropriated directory, tell NeoMutt where the start up 
script lives by appending:

    ifndef USE_LUA finish
    lua-source /path/to/neomutt-startup.lua

to the neomuttrc file and lastly adapt package.path inside the start up 
file to the extracted ‘scripts’ directory. After ensuring that there is 
nothing dangerous in the scripts, maybe read the hopefully informative 
comments and run NeoMutt in batch mode: `mutt -B`.

To get a little bit more printed informations about certain objects, 
namespaces, event handler, comment out the ‘do return end’ statement at 
line 90 in ‘neomutt-startup.lua’.

Hope that allow users to start using Lua with NeoMutt.

>2. Help build the mutt API in Lua, so it's consistent with how any 
>other
>   Lua library is built… Rule of least surprise ☺
>3. Help in the Lua code implementation itself, help with making the
>   mutt_lua.c code neater and nicer, and help extending the API to
>   internal APIs that are not (yet) accessible.

I am not a developer and also have a deep respect for braking things 
where I miss the general survey. But it’s interesting enough for me to 
have a look at that periodically. When I get an idea or found some weird 
things, I’ll report it in a reasonable way.


-- 
Regards,
floyd

-------------- next part --------------
A non-text attachment was scrubbed...
Name: neomutt_lua-skeleton.tar.gz
Type: application/octet-stream
Size: 7145 bytes
Desc: not available
URL: <http://mailman.neomutt.org/pipermail/neomutt-users-neomutt.org/attachments/20170503/ff6dcad2/attachment.obj>
-------------- next part --------------
949887258aa4d39a0e86836695b67a6ae97e2773a44f1e54bd849e0b9842ddb6  neomutt_lua-skeleton.tar.gz