Pure Nginx external HTTP upload

As i leraned yesterday, the Movim requires HTTP upload servico on XMPP server, which i have not enabled yet. After fast look i found two Prosody's modules, which provides it. Because i don't believe Prosody's HTTP server (eg. not SNI support yet), i focus to its external HTTP upload module, which require external service.

Background

The mod_http_upload_external page describes some external solutions:

1. the PHP and Go solution is not for me...
2. the most close to my experiences goes the Python's solution, but i want to avoid external daemon, especially the Flask one, despite that i have all needed for it installed and used already (uWSGI, Flask, Python3, ...), because i know its performance limits and memory requirements.
3. which interested me, is Nginx's Perl module. I consider this as good solution, ecause Debian's Nginx comes with Perl support via dynamic module. I am not very familiar with Perl and this implementation sounds as very simple task...

I decided to try to utilize the Nginx's built-in DAV module and some external modules, namely HMAC Secure link and (as i learn later) Set misc.

The DAV module itself provides support for PUT requests, nothing special. The "HMAC secure link" module is able to verify HMAC signature, but it cannot handle prosody's HMAC-SHA256 digest directly, because it expects Base64URL encoded HMAC, there is need to encode Prosody's hex string, which allows the "Set Misc" module in two steps, first decode hex string into binary value, and then then encode it as Base64 string.

Nginx location

After some playing i got this location config. Some point about:

• here are two nested locations, not really needed, but Nginx search all regex global locations on any request, what can a litle reduce performance, this limits searching this location only for chosen prefix
• nested location uses regular expression capture group, to eliminate prefix form URI, without needing map directive
• nested location doesn't properly handles the missing token, while there is an if directive for it at start of location, it returns 500 code
• nested location properly handles 409 response on existing file and 403 response on bad or missing token by some Nginx's set directive magic

location /xmpp {
    root                    /srv;

    location ~ /xmpp/(?<fpath>.+)$ {
        dav_methods             PUT;
        create_full_put_path    on;                     # create directory, if needed
        dav_access              user:rw group:rw all:r; # set permissions
        client_max_body_size    100m;                   # default prosody's body size is 100 MB

        # encode $arg_v into Base64URL digest
        set                             $digest $arg_v;
        set_if_empty                    $digest "00";
        set_decode_hex                  $digest;
        set_encode_base64               $digest;

        # verify $digest
        secure_link_hmac                $digest;
        secure_link_hmac_secret         "123456";
        secure_link_hmac_algorithm      sha256;
        secure_link_hmac_message        "$fpath $content_length";

        # handle missing token
        set                         $missing $request_method$arg_v;
        if ($missing = "PUT")            {return 403;}

        # do not overwrite existing file
        if (-e $request_filename)   {set $exists $request_method;}
        if ($exists = "PUT")        {return 409;}

        # handle bad HMAC token
        if ($request_method = "PUT") {set $verified $request_method$secure_link_hmac;}
        if ($verified = "PUT")      {return 403;}
    }
}

The missing token (and related 500 response) can be solved by set_if_empty directive, but i didn't play with it. solved.

Testing

I prefer shell testing of HTTP services, because i can do some scripts for it. Here is simple script to generate HMAC token and fire PUT request with it. It is not very intelligent, all things are hardcoded inside variables, but as an inspiration:

kluc="123456"
subor="/tmp/aaa.txt"

velk=$(stat --printf="%s" "$subor")

text="${subor#/tmp/} $velk"

hmac=$(echo -n "$text" | openssl dgst -sha256 -hmac "$kluc" | cut -d" " -f2 )
echo "v=$hmac"

wget -qO- --server-response --body-file="$subor" --method=PUT \
    https://bonifac.skk/xmpp/aaa.txt?v=${hmac}

Conclusion

I am sure, that this solution can provide better performance than ĺinked Flask module, but I am not happy with this solution, because i see some problems:

• while it is basically working, the "HMAC Secure link" module expects Base64URL encoded token, but "Set misc" module can provide only pure Base64 encoding, thus some combinations of the filename and size can be refused
• it requires two external modules, which are not included in standard Debian package, and thus recompilation of Nginx can be required

I spent some hours with this (including repackaging Nginx, reading docs, coffee and launch pause, etc), but it was wasted time. Next i will try the linked Perl Nginx's module. If someone know how to solve the Base64URL encoding problem, i will be happy if (s)he will share it.