close
    • chevron_right

      Newsletter: Summer in Review

      Stephen Paul Weber · Wednesday, 13 September - 20:30 edit · 2 minutes · 6 visibility

    Hi everyone!

    Welcome to the latest edition of your pseudo-monthly JMP update!

    In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

    Since our launch at the beginning of the summer, we’ve kept busy.  We saw some of you at the first FOSSY, which took place in July.  For those of you who missed it, the videos are out now.

    Automatic refill for users of the data plan is in testing now.  That should be fully automated a bit later this month and will pave the way for the end of the waiting list, at least for existing JMP customers.

    This summer also saw the addition of two new team members: welcome to Gnafu the Great who will be helping out with support, and Amolith, who will be helping out on the technical side.

    There have also been several releases of the Cheogram Android app (latest is 2.12.8-2) with new features including:

    • Support for animated avatars
    • Show “hats” in the list of channel participants
    • An option to show related channels from the channel details area
    • Emoji and sticker autocomplete by typing ‘:’ (allows sending custom emoji)
    • Tweaks to thread UI, including no more auto-follow by default in channels
    • Optionally allow notifications for replies to your messages in channels
    • Allow selecting text and quoting the selection
    • Allow requesting voice when you are muted in a channel
    • Send link previews
    • Support for SVG images, avatars, etc.
    • Long press send button for media options
    • WebXDC importFiles and sendToChat support, allowing, for example, import and export of calendars from the calendar app
    • Fix Command UI in tablet mode
    • Manage permissions for channel participants with a dialog instead of a submenu
    • Ask if you want to moderate all recent messages by a user when banning them from a channel
    • Show a long streak of moderated messages as just one indicator

    To learn what’s happening with JMP between newsletters, here are some ways you can find out:

    Thanks for reading and have a wonderful rest of your week!

    • chevron_right

      JMP is Launched and Out of Beta

      Stephen Paul Weber · Monday, 12 June - 11:00 edit · 2 minutes · 9 visibility

    JMP has been in beta for over six years, and today we are finally launching! With feedback and testing from thousands of users, our team has made improvements to billing, phone network compatibility, and also helped develop the Cheogram Android app which provides a smooth onboarding process, good Android integration, and phone-like UX for users of that platform. There is still a long road ahead of us, but with so much behind us we’re comfortable saying JMP is ready for launch, and that we know we can continue to work with our customers and our community for even better things in the future.  Check out our launch on Product Hunt today as well!

    JMP’s pricing has always been “while in beta” so the first question this raises for many is: what will the price be now? The new monthly price for a customer’s first JMP phone number is now $4.99 USD / month ($6.59 CAD), but we are running a sale so that all customers will continue to pay beta pricing of $2.99 USD / month ($3.59 CAD) until the end of August. We are extending until that time the option for anyone who wishes to prepay for up to 3 yearsand lock-in beta pricing. Contact support if you are interested in the prepay option. After August, all accounts who have not pre-paid will be put on the new plan with the new pricing. Those who do pre-pay won’t see their price increase until the end of the period they pre-paid for.  The new plan will also include a multi-line discount, so second, third, etc JMP phone numbers will be $2.45 USD / month ($3.45 CAD) when they are set to all bill from the same balance.  The new plan will also finally have zero-rated toll free calling.  All other costs (per-minute costs, etc) remain the same, see the pricing page for details.

    The account settings bot now has an option “Create a new phone number linked to this balance” so that you can get new numbers using your existing account credit and linked for billing to the same balance without support intervention.

    Thanks so much to all of you who have helped us get this far.  There is lots more exciting stuff coming this year, and we are so thankful to have such a supportive community along the way with us.  Don’t forget we’ll be at FOSSY in July, and be sure to check out our launch on Product Hunt today as well.

    • chevron_right

      Newsletter: Jabber ID Discovery, New Referral Codes

      Stephen Paul Weber · Monday, 1 May - 05:00 edit · 4 minutes

    Hi everyone!

    Welcome to the latest edition of your pseudo-monthly JMP update!

    In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

    It has been a while since we got a newsletter out, and lots has been happening as we race towards our launch.

    For those who have experienced the issue with Google Voice participants not showing up properly in our MMS group texting stack, we have a new stack in testing right now.  Let support know if you want to try it out, it has been working well so far for those already using it.

    If you check your account settings for the “refer a friend” option you will now see two kinds of referral code.  The list of one-time use codes remains the same as always: a free month for your friend, and a free month’s worth of credit for you if they start paying.  The new code up in the top is multi-use and you can post and share it as much as you like.  It provides credit equivalent to an additional month to anyone who uses it on sign up after their initial $15 deposit as normal, and then a free month’s worth of credit for you after that payment fully clears.

    We mentioned before that much of the team will be present at FOSSY, and we can now reveal why: there will be a conference track dedicated to XMPP, which we are helping to facilitate!  Call for proposals ends May 14th. Sign up and come out this summer!

    Quicksy Logo For quite some time now, customers have been asked while registering if they would like to enable others who know their phone number to discover their Jabber ID, to enable upgrading to end-to-end encryption, video calls, etc.  The first version of this feature is now live, and users of at least Cheogram Android and Movim can check the contact details of anyone they exchange SMS with to see if a Jabber ID is listed.  We are happy to announce that we have also partnered with Quicksy to allow discovery of anyone registered for their app or directory as well.

    Tapbacks Jabber-side reactions are now translated where possible into the tapback pseudo-syntax recognized by many Android and iMessage users so that your reactions will appear in a native way to those users.  In Cheogram Android you can swipe to reply to a message and enter a single emoji as the reply to send a reaction/tapback.

    Cheogram Android There have been two Cheogram Android releases since our last newsletter, with a third coming out today.  You no longer need to add a contact to send a message or initiate a call.  The app has seen the addition of moderation features for channel administrators, as well as respecting these moderation actions on display.  For offensive media arriving from other sources, in avatars, or just not moderated quickly enough, users also have the ability to permanently block any media they see from their device.

    Cheogram Android has seen some new sticker-related features including default sticker packs and the ability to import any sticker pack made for signal (browse signalstickers.com to find more sticker packs, just tap “add to signal” to add them to Cheogram Android).

    There are also brand-new features today in 2.12.1-5, including a new onboarding flow that allows new users to register and pay for JMP before getting a Jabber ID, and then set up their very own Snikket instance all from within the app.  This flow also features some new introductory material about the Jabber network which we will continue to refine over time:

    Welcome to Cheogram Android ScreenshotHow the Jabber network works ScreenshotWelcome Screen Screenshot

    Notifications about new messages now use the conversation style in Android.  This means that you can set seperate priority and sounds per-conversation at the OS level on new enough version of Android.  There is also an option in each conversation’s menu to add that conversation to your homescreen, something that has always been possible with the app but hopefully this makes it more discoverable for some.

    For communities organizing in Jabber channels, sometimes it can be useful to notify everyone present about a message.  Cheogram Android now respects the attention element from members and higher in any channel or group chat.  To send a message with this priority attached, start the message body with @here (this will not be included in the actual message people see).

    WebXDC Logo

    This release also brings an experimental prototype supporting WebXDC.  This is an experimental specification to allow developers to ship mini-apps that work inside your chats.  Take any *.xdc file and send it to a contact or group chat where everyone uses Cheogram Android and you can play games, share notes, shopping lists, calendars, and more.  Please come by the channel to discuss the future of this technology on the Jabber network with us.

    To learn what’s happening with JMP between newsletters, here are some ways you can find out:

    Thanks for reading and have a wonderful rest of your week!

    • chevron_right

      Verify Google Play App Purchase on Your Server

      Stephen Paul Weber · Tuesday, 11 April - 03:00 edit · 5 minutes

    We are preparing for the first-ever Google Play Store launch of Cheogram Android as part of JMP coming out of beta later this year.  One of the things we wanted to “just work” for Google Play users is to be able to pay for the app and get their first month of JMP “bundled” into that purchase price, to smooth the common onboarding experience.  So how do the JMP servers know that the app communicating with them is running a version of the app bought from Google Play as opposed to our builds, F-Droid’s builds, or someone’s own builds?  And also ensure that this person hasn’t already got a bundled month before?  The documentation available on how to do this is surprisingly sparse, so let’s do this together.

    Client Side

    Google publishes an official Licensing Verification Library for communicating with Google Play from inside an Android app to determine if this install of the app can be associated with a Google Play purchase.  Most existing documentation focuses on using this library, however it does not expose anything in the callbacks other than “yes license verified” or “no, not verified”.  This can allow an app to check if it is a purchased copy itself, but is not so useful for communicating that proof onward to a server.  The library also contains some exciting snippets like:

    // Base64 encoded -
    // com.android.vending.licensing.ILicensingService
    // Consider encoding this in another way in your
    // code to imp rove security
    Base64.decode(
        "Y29tLmFuZHJvaWQudmVuZGluZy5saWNlbnNpbmcuSUxpY2Vuc2luZ1NlcnZpY2U=")))

    Which implies that they expect developers to fork this code to use it.  Digging in to the code we find in LicenseValidator.java:

    public void verify(PublicKey publicKey, int responseCode, String signedData, String signature)

    Which looks like exactly what we need: the actual signed assertion from Google Play and the signature!  So we just need a small patch to pass those along to the callback as well as the response code currently being passed.  Then we can use the excellent jitpack to include the forked library in our app:

    implementation 'com.github.singpolyma:play-licensing:1c637ea03c'

    Then we write a small class in our app code to actually use it:

    import android.content.Context;
    import com.google.android.vending.licensing.*;
    import java.util.function.BiConsumer;
    
    public class CheogramLicenseChecker implements LicenseCheckerCallback {
        private final LicenseChecker mChecker;
        private final BiConsumer mCallback;
    
        public CheogramLicenseChecker(Context context, BiConsumer<String, String> callback) {
            mChecker = new LicenseChecker(  
                context,  
                new StrictPolicy(), // Want to get a signed item every time  
                context.getResources().getString(R.string.licensePublicKey)  
            );
            mCallback = callback;
        }
    
        public void checkLicense() {
            mChecker.checkAccess(this);
        }
    
        @Override
        public void dontAllow(int reason) {
            mCallback.accept(null, null);
        }
    
        @Override
        public void applicationError(int errorCode) {
            mCallback.accept(null, null);
        }
    
        @Override
        public void allow(int reason, ResponseData data, String signedData, String signature) {
            mCallback.accept(signedData, signature);
        }
    }

    Here we use the StrictPolicy from the License Verification Library because we want to get a fresh signed data every time, and if the device is offline the whole question is moot because we won’t be able to contact the server anyway.

    This code assumes you put the Base64 encoded licensing public key from “Monetisation Setup” in Play Console into a resource R.string.licensePublicKey.

    Then we need to communicate this to the server, which you can do whatever way makes sense for your protocol; with XMPP we can easily add custom elements to our existing requests so:

    new com.cheogram.android.CheogramLicenseChecker(context, (signedData, signature) -> {
        if (signedData != null && signature != null) {
            c.addChild("license", "https://ns.cheogram.com/google-play").setContent(signedData);
            c.addChild("licenseSignature", "https://ns.cheogram.com/google-play").setContent(signature);
        }
    
        xmppConnectionService.sendIqPacket(getAccount(), packet, (a, iq) -> {
            session.updateWithResponse(iq);
        });
    }).checkLicense();

    Server Side

    When trying to verify this on the server side we quickly run into some new issues.  What format is this public key in?  It just says “public key” and is Base64 but that’s about it.  What signature algorithm is used for the signed data?  What is the format of the data itself?  Back to the library code!

    private static final String KEY_FACTORY_ALGORITHM = "RSA";
    …
    byte[] decodedKey = Base64.decode(encodedPublicKey);
    …
    new X509EncodedKeySpec(decodedKey)

    So we can see it is an X509 related encoded, and indeed turns out to be Base64 encoded DER.  So we can run this:

    echo "BASE64_STRING" | base64 -d | openssl rsa -pubin -inform der -in - -text

    to get the raw properties we might need for any library (key size, modulus, and exponent).  Of course, if your library supports parsing DER directly you can also use that.

    import java.security.Signature;
    …
    private static final String SIGNATURE_ALGORITHM = "SHA1withRSA";
    …
    Signature sig = Signature.getInstance(SIGNATURE_ALGORITHM);
    sig.initVerify(publicKey);
    sig.update(signedData.getBytes());

    Combined with the java documentation we can thus say that the signature algoritm is PKCS#1 padded RSA with SHA1.

    And finally:

    String[] fields = TextUtils.split(mainData, Pattern.quote("|"));
    data.responseCode = Integer.parseInt(fields[0]);
    data.nonce = Integer.parseInt(fields[1]);
    data.packageName = fields[2];
    data.versionCode = fields[3];
    // Application-specific user identifier.
    data.userId = fields[4];
    data.timestamp = Long.parseLong(fields[5]);

    The format of the data, pipe-seperated text. The main field of interest for us is userId which is (as it says in a comment) “a user identifier unique to the <application, user> pair”. So in our server code:

    import Control.Error (atZ)
    import qualified Data.ByteString.Base64 as Base64
    import qualified Data.Text as T
    import Crypto.Hash.Algorithms (SHA1(SHA1))
    import qualified Crypto.PubKey.RSA as RSA
    import qualified Crypto.PubKey.RSA.PKCS15 as RSA
    import qualified Data.XML.Types as XML
    
    googlePlayUserId
        | googlePlayVerified = (T.split (=='|') googlePlayLicense) `atZ` 4
        | otherwise = Nothing
    googlePlayVerified = fromMaybe False $ fmap (\pubKey ->
        RSA.verify (Just SHA1) pubKey (encodeUtf8 googlePlayLicense)
            (Base64.decodeLenient $ encodeUtf8 googlePlaySig)
        ) googlePlayPublicKey
    googlePlayLicense = mconcat $ XML.elementText
        =<< XML.isNamed (s"{https://ns.cheogram.com/google-play}license")
        =<< XML.elementChildren payload
    googlePlaySig = mconcat $ XML.elementText
        =<< XML.isNamed (s"{https://ns.cheogram.com/google-play}licenseSignature")
        =<< XML.elementChildren payload

    We can then use the verified and extracted googlePlayUserId value to check if this user has got a bundled month before and, if not, to provide them with one during signup.

    • chevron_right

      Cheogram Android: Stickers

      blog.jmp.chat · Wednesday, 1 March, 2023 - 06:00 edit · 3 minutes

    One feature people ask about from time to time is stickers.  Now, “stickers” isn’t really a feature, nor is it even universally agreed what it means, but we’ve been working on some improvements to Cheogram Android (and the Cheogram service) to make some sticker workflows better, released today in 2.12.1-3.  This post will mostly talk about those changes and the technical implications; if you just want to see a demo of some UI you may want to skip to the video demo.

    Many Android users already have pretty good support for inserting stickers (or GIFs) into Cheogram Android via their keyboard.  However, as the app existed at the time, this would result in the sender re-uploading and the recipient re-downloading the sticker image every time, and fill up the sending server and receiving device with many copies of the same image.  The first step to mitigating this was to switch local media storage in the app to content-addressed, which in this case means that the file is named after the hash of its contents.  This prevents filling up the device when receiving the same image many times.

    Now that we know the hashes of our stored media, we can use SIMS to transmit this hash when sending.  If the app sees an image that it already has, it can display it without downloading at all, saving not only space but bandwidth and time as well.  The Cheogram service also uses SIMS to transmit hashes of incoming MMS images for this purpose as well.

    An existing Jabber client which uses the word “stickers” is Movim.  It wouldn’t make sense to add the word to our UI without supporting what they already have.  So we added support for XHTML-IM including Bits of Binary images.  This also relies on hash-based storage or caching, which by now we had.  This tech will also be useful in the future to extend beyond stickers into custom emoji.

    Some stickers are animated, and users want to be able to send GIFs as well, so the app was updated to support inline playback of animated images (both GIF and WebP format).

    Some users don’t have any sticker support in their keyboard or OS, so we want to provide some tools for these users as well.  We have added the option to download some default sticker packs (mostly curated from the default set from Movim for now) so that users start with some options.  We also built a small proxy to allow easily importing stickers intended for signal by clicking the regular “add to signal” links on eg signalstickers.com.  Any sticker selected from these will get sent without even uploading, saving time and space on the server, and then will be received by any user of the app who has the default packs installed with no need for downloading, with fallbacks for other clients and situations of course.

    If a user receives a sticker that they’d like to save for easily sending out again later, they can long-press any image they receive and choose “Save as sticker” which will prompt them to choose or create a sticker pack to keep it in, then save it there.  Pointing a sticker sheet app or keyboard at this directory also allows re-using other sticker selection UIs with custom stickers saved in this way.

    Taken together we hope these features produce real benefits for users of stickers, both with and without existing keyboard support, and also provide foundational work that we can build upon to provide custom emoji, thumbnails before downloading, URL previews, and other rich media features in the future.  If you’d like to see some of these features in action, check out this short video.

    • chevron_right

      Newsletter: JMP is 6! Leaving beta this year! And FOSSY SLIGHTLY SMILING FACE

      denver · Wednesday, 15 February, 2023 - 03:30 edit · 4 minutes

    Hi everyone!

    Welcome to the latest edition of your pseudo-monthly JMP update!

    In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

    JMP is 6 years old today!  When we launched in 2017 we had no idea exactly how far we’d go, or that we’d be making one of the most popular chat clients on F-Droid (that is Cheogram Android, which is based on Conversations).  Last year we called JMP “phone-feature-complete” and since then we’ve made all of JMP’s features even easier to use, shepherding big improvements to various Jabber clients, including Movim and Dino, while continuing to provide first-class telephony and messaging features in our flagship mobile app: Cheogram Android.

    With so many of the edges now smoothed, and a new onboarding flow almost ready to go, it’s now time to announce: JMP will be leaving beta this year!

    What does that mean?  Primarily this is our vote of confidence (as the JMP and Cheogram team) that JMP, and apps we develop such as Cheogram Android, are ready for widespread use.  While of course there will still be improvements to make, we believe it will be able to be recommended to your friends and family (especiall Android users) without reservation.

    Naturally there are a couple things to do yet to make that happen, and one of them is to put Cheogram Android in the Play Store at last.  This will be a paid (but still free-as-in-freedom) app that will include one month of JMP service.  Of course, you will still be able to get Cheogram Android from all the other places you can already get it (such as F-Droid and our own repos/APKs).

    The other main thing is to set a final post-beta monthly price for JMP.  And, while it won’t take effect until we launch later this year, we are able to now officially announce it: US$4.99/month, with incidental pricing remaining the same (i.e. extra/international minutes will remain what they are now).  Note that there will be discounts for additional JMP numbers linked to your primary JMP number, and also (before JMP leaves beta) a chance to lock in the existing pricing for a period of time.  Having never changed the price since we started JMP 6 years ago, and given the inflation and our own staffing costs since then, we feel the new price will allow JMP to remain both sustainable, and able to face new challenges and exciting opportunities going forward (like the EU’s DMA, for one).  We want to make JMP the best phone number service, and Cheogram the best gateway to everything in the world!

    Speaking of Cheogram, a JMP newsletter these days wouldn’t be complete without mention of new Cheogram Android features (2.12.1-2 released in APK form and Cheogram F-Droid repo today!):

    • it will now offer to setup Dialer integration automatically when available
    • the Call Logs (cdrs) command replaces the usage command (giving you more info)
    • the new onboarding flow is improved even more
    • admins of a Snikket instance can create a new Jabber ID and JMP number all inside the app now (see the video demo)
    • new theme: any colour you want! (requires Android 11 or higher)

    Note that the Call Logs (cdrs) command will roll out to everyone in about a week.  If you’d like to try it before then, please send a private inquiry to JMP support and we’ll activate it for you.

    Lastly, some of you may be interested to know that the JMP/Cheogram team are going to be venturing out to a conference for the first time since March 2020!  In particular, most of the JMP/Cheogram team will be attending FOSSY this year, in Portland, Oregon, USA this July 13-16.  We’ll be announcing specifics of our involvement (whether we have a booth, talks, etc.) closer to the dates.  In the meantime, just know we’ll be there, and would love to chat with any JMP/Cheogram users, prospective customers, or otherwise!

    With that, we’ll cap off our 6 years. :)  And what an exciting 6 years it’s been!  With the big launch this year, you can bet on many more to come!

    To learn what’s happening with JMP between newsletters, here are some ways you can find out:

    Thanks for reading and have a wonderful rest of your week!

    • chevron_right

      Newsletter: Threads, Thumbnails, XMR, ETH

      Stephen Paul Weber · Monday, 23 January, 2023 - 19:30 edit · 2 minutes

    Hi everyone!

    Welcome to the latest edition of your pseudo-monthly JMP update!

    In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

    This month we released Cheogram Android 2.12.1-1 which includes several new features.  One of the big ones is an interface for having threaded conversations with other Jabber users (watch the demo video).  This feature will also make it easier to reply to the right message if you use the email gateway.  The app has grown support for more media features, including an ability to show an image right away if you already have it, without waiting for a download, and blurhash based placeholders for images from MMS you have not yet downloaded.

    There is also a new user experience when receiving group texts that will actually show the sender’s name (and even avatar, if you have one set for them locally) the same way as any other group chat in the app.  This is made possible by a new draft protocol extension we adopted for part of the purpose.

    This version is based on the latest 2.12.1 from upstream, which among other things has added the ability to function as a Unified Push distributor, so if you use any compatible app you may want to check that out.

    For the JMP service, this month we shipped the ability to make top-up payments using XMR or ETH directly from the top up command.  This simplifies the flow for users of those currencies, and we hope you will find it useful.  Integrating this support into registration is also coming, but not ready quite yet.

    If you are planning to be at FOSDEM 2023, be sure to check out the realtime lounge in with the other stands.  Unfortunately no one from JMP will be there this year, but people from Snikket and other projects around the ecosystem will be present.

    To learn what’s happening with JMP between newsletters, here are some ways you can find out:

    Thanks for reading and have a wonderful rest of your week!

    • chevron_right

      Newsletter: Busy Year in 2022

      Stephen Paul Weber · Monday, 19 December, 2022 - 08:30 edit · 4 minutes

    Hi everyone!

    Welcome to the latest edition of your pseudo-monthly JMP update!

    In case it’s been a while since you checked out JMP, here’s a refresher: JMP lets you send and receive text and picture messages (and calls) through a real phone number right from your computer, tablet, phone, or anything else that has a Jabber client.  Among other things, JMP has these features: Your phone number on every device; Multiple phone numbers, one app; Free as in Freedom; Share one number with multiple people.

    Cheogram Android 2.11.0-1 has been released, including an important fix for creating new private group chats.  For some months creating such a group (a Jabber group, not a “group text”) with Cheogram Android has resulted in a public channel on many servers.  Please double-check your private groups and change settings if necessary!  This release will also be the first accepted into F-Droid with an up-to-date version of libwebrtc, so if you’ve had any issues with calls and use the F-Droid build, we recommend upgrading and trying again.  This release also adds support for tagging channels and group chats (on supporting servers, such as Snikket), better use of locales to determine what country code to prepend when dialling, a new OLED black theme, and more.

    The data plan roll out continues, accelerating in December but we know there are still many of you waiting.  Thank you so much for your patience, and to all the feedback we have received from users so far.  We are actively working on making the signup process self-serve so that the waitlist will no longer be necessary in the future.

    When JMP started we were just one part-time person.  As we grow, the legal structures that fit that time no longer do.  This fall we incorporated the MBOA Technology Co-operative to house JMP, Togethr, consulting work, and other activity.  This gives all our employees full agency in the company and gives us a firm legal footing for the future.  Nothing changes for you at this time, we’re still the same team, and for the time being you don’t even change the name you write on the cheques, nevertheless it marks a milestone in our life as a company.

    Year in Review

    This year, JMP and Snikket CIC made a deal to offer Jabber hosting as an option for JMP customers. This service is included in the regular JMP subscription and will eventually be the default option for new users during the sign-up process. JMP customers have been able to participate in a beta version of this integration, and JMP customers can contact JMP support to set up a Snikket instance directly.

    This year also saw international calling added to our list of features. JMP users are able to use as many minutes per month as they like, with approximately 120 minutes of credit to USA and Canada included by default. Customers are able to pay for additional minutes and make international calls, although users who are still paying with the old PayPal system will not have access to these features (or other features such as the data plan). We also implemented a per-calendar-month overage limit system, where customers can set their own limits to avoid unexpected charges. The default limit is currently set at $0/month.

    One of our most popular features has always been our voicemail and transcription, this year we expanded that to support multi-lingual transcriptions as well.

    We also added multi-account billing this year, an alpha for JMP use from Matrix, added two employees, created new bot commands for account management, launched Togethr to help people take control of their social media identity, added support for SMS-only ports and the option to disable voicemail, built an XMPP integration for Chatwoot, and launched the JMP data plan.

    This year saw the launch and rapid development of the Cheogram Android app, forked from Conversations and including these and other improvements:

    • Add contacts without typing @cheogram.com
    • Integrate with the native Android Phone app (optional)
    • Address book integration (optional)
    • Option to start group texts easily
    • Command UI for better interactions with our and other bots (you can even sign up entirely from within the app!)
    • Rich text message display (including stickers from Movim users)
    • Data de-duplication for files sent/received multiple times
    • Message retraction
    • Ability to edit tags on contacts and channels
    • Tag navigation widget for easier conversation management
    • Ability to copy any link in a message to the clipboard
    • F-Droid repositories for quick updates of official builds

    Blog posts this year included: How to use Jabber from SMS, Why Bidirectional Gateways Matter, Computing International Call Rates with a Trie, Privacy and Threat Modelling, SMS Account Verification, and Writing a Chat Client from Scratch.

    To learn what’s happening with JMP between newsletters, here are some ways you can find out:

    Thanks for reading and have a wonderful rest of your week!

    • chevron_right

      Writing a Chat Client from Scratch

      Stephen Paul Weber · Wednesday, 30 November, 2022 - 03:30 edit · 27 minutes

    There are a lot of things that go into building a chat system, such as client, server, and protocol.  Even for only making a client there are lots of areas of focus, such as user experience, features, and performance.  To keep this post a manageable size, we will just be building a client and will use an existing server and protocol (accessing Jabber network services using the XMPP protocol).  We’ll make a practical GUI so we can test things, but not spend too much time on polish, and look at getting to a useful baseline of features.

    You can find all the code for this post in git.  All code licensed AGPL3+.

    Use a Library

    As with most large programming tasks, if we wanted to do every single thing ourselves we would spend a lot more time, so we should find some good libraries.  There is another reason to use a library: any improvements we make to the library benefits others.  While releasing our code might help someone else if they choose to read it, a library improvement can be picked up by users of that library right away.

    We need to speak the XMPP protocol so let’s choose Blather.  We need a GUI so we can see this working, but don’t really want to futz with it much so let’s choose Glimmer.  The code here will use these libraries and be written in the Ruby programming language, but these ideas are general purpose to the task and hopefully we won’t get too bogged down in syntax specifics.

    One little language-specific thing you will need to create is a description of which ruby packages are being used, so let’s make that file (named Gemfile):

    Gemfile

    source "https://rubygems.org"
    
    gem "blather", git: "https://github.com/adhearsion/blather", branch: "develop"
    gem "glimmer-dsl-libui", "~> 0.5.24"

    Run this to get the packages installed:

    bundle install --path=.gems

    Let’s get the bare minimum: a connection to a Jabber service and a window.

    client.rb

    require "glimmer-dsl-libui"
    require "blather/client"
    
    BLATHER = self
    include Glimmer
    
    Thread.new do
    	window("Contacts") {
    		on_destroy {
    			BLATHER.shutdown
    		}
    	}
    end

    When required in this way, Blather will automatically set up a connection with event processing on the main thread, and will process command line arguments to get connection details.  So we put the GUI on a second thread to not have them block each other.  When the window is closed (on_destroy), be sure to disconnect from the server too.  You can run this barely-a-client like this:

    bundle exec ruby client.rb user@example.com password

    The arguments are a Jabber ID (which you can get from manyexistingservices), and the associated password.

    You should get a blank window and no errors in your terminal.  If you wanted to you could even look in another client and confirm that it is connected to the account by seeing it come online.

    Show a Contact List

    Let’s fetch the user’s contacts from the server and show them in the window (if you use this with a new, blank test account there won’t be any contacts yet of course, but still).

    $roster = [["", ""]]
    
    Thread.new do
    	window("Contacts") {
    		vertical_box {
    			table {
    				button_column("Contact") {
    				}
    				editable false
    				cell_rows $roster
    			}
    		}
    
    		on_destroy {
    			BLATHER.shutdown
    		}
        }.show
    end
    
    after(:roster) do
    	LibUI.queue_main do
    		$roster.clear
    		my_roster.each do |item|
    			$roster << [item.name || item.jid, item.jid]
    		end
    	end
    end

    In a real app you would probably want some kind of singleton object to represent the contacts window and the contact list (“roster”) etc.  For simplicity here we just use a global variable for the roster, starting with some dummy data so that the GUI framework knows what it will look like, etc.

    We fill out the window from before a little bit to have a table with a column of buttons, one for each contact.  The button_column is the first (and in this case, only) column definition so it will source data from the first element of each item in cell_rows.  It’s not an editable table, and it gets its data from the global variable.

    We then add an event handler to our XMPP connection to say that once the roster has been loaded from the server, we hand control over to the GUI thread and there we clear out the global variable and fill it up with the roster as we now see it.  The first item in each row is the name that will be shown on the button (either item.name or item.jid if there is no name set), the second item is the Jabber ID which won’t be shown because we didn’t define that column when we made the window.  Any updates to the global variable will be automatically painted into the GUI so we’re done.

    One Window Per Conversation

    For simplicity, let’s say we want to show one window per conversation, like so:

    $conversations = {}
    
    class Conversation
    	include Glimmer
    
    	def self.open(jid, m=nil)
    		return if $conversations[jid]
    
    		($conversations[jid] = new(jid, m)).launch
    	end
    
    	def initialize(jid, m=nil)
    		@jid = jid
    		@messages = [["", ""]]
    		new_message(m) if m
    	end
    
    	def launch
    		window("Conversation With #{@jid}") {
    			vertical_box {
    				table {
    					text_column("Sender")
    					text_column("Message")
    					editable false
    					cell_rows @messages
    					@messages.clear
    				}
    
    				horizontal_box {
    					stretchy false
    
    					@message_entry = entry
    					button("Send") {
    						stretchy false
    
    						on_clicked do
    							BLATHER.say(@jid, @message_entry.text)
    							@messages << [ARGV[0], @message_entry.text]
    							@message_entry.text = ""
    						end
    					}
    				}
    			}
    
    			on_closing do
    				$conversations.delete(@jid)
    			end
    		}.show
    	end
    
    	def format_sender(jid)
    		BLATHER.my_roster[jid]&.name || jid
    	end
    
    	def message_row(m)
    		[
    			format_sender(m.from&.stripped || BLATHER.jid.stripped),
    			m.body
    		]
    	end
    
    	def new_message(m)
    		@messages << message_row(m)
    	end
    end
    
    message :body do |m|
    	LibUI.queue_main do
    		conversation = $conversations[m.from.stripped.to_s]
    		if conversation
    			conversation.new_message(m)
    		else
    			Conversation.open(m.from.stripped.to_s, m)
    		end
    	end
    end

    Most of this is the window definition again, with a table of the messages in this conversation sourced from an instance variable @messages.  At the bottom of the window is an entry box to type in text and a button to trigger sending it as a message.  When the button is clicked, send that message to the contact this conversation is with, add it to the list of messages so that it shows up in the GUI, and make the entry box empty again.  When the window closes (on_closing this time because it’s not the “main” window) delete the object from the global set of open conversations.

    This object also has a helper to open a conversation window if there isn’t already one with a given Jabber ID (jid), some helpers to format message objects into table rows by extracting the sender and body (including format_sender which gets the roster item if there is one, uses &.name to get the name if there was a roster item or else nil, and if there was no roster item or no name just show jid) and a helper that adds new messages into the GUI.

    Finally we add a new XMPP event handler for incoming messages that have a body.  Any such incoming message we look up in the global if there is a conversation open already, if so we pass the new message there to have it appended to the GUI table, otherwise we open the conversation with this message as the first thing it will show.

    Getting from the Contact List to a Conversation

    Now we wire up the contact list to the conversation view:

    button_column("Contact") {
    	on_clicked do |row|
    		Conversation.open($roster[row][1].to_s)
    	end
    }

    When a contact button is clicked, grab the Jabber ID from the hidden end of the table row that we had stashed there, and open the conversation.

    horizontal_box {
    	stretchy false
    
    	jid_entry = entry {
    		label("Jabber ID")
    	}
    
    	button("Start Conversation") {
    		stretchy false
    
    		on_clicked do
    			Conversation.open(jid_entry.text)
    		end
    	}
    }

    And let’s provide a way to start a new conversation with an address that isn’t a contact too.  An entry to type in a Jabber ID and a button that opens the conversation.

    Adding a Contact

    Might as well add a button to the main window that re-uses that entry box to allow adding a contact as well:

    button("Add Contact") {
    	stretchy false
    
    	on_clicked do
    		BLATHER.my_roster << jid_entry.text
    	end
    }

    Handling Multiple Devices

    In many chat protocols, it is common to have multiple devices or apps connected simultaneously. It is often desirable to show messages sent to or from one device on all the others as well.  So let’s implement that.  First, a helper for creating XML structures we may need:

    def xml_child(parent, name, namespace)
    	child = Niceogiri::XML::Node.new(name, parent.document, namespace)
    	parent << child
    	child
    end

    We need to tell the server that we support this feature:

    when_ready do
    	self << Blather::Stanza::Iq.new(:set).tap { |iq|
    		xml_child(iq, :enable, "urn:xmpp:carbons:2")
    	}
    end

    We will be handling live messages from multiple event handlers so let’s pull the live message handling out into a helper:

    def handle_live_message(m, counterpart: m.from.stripped.to_s)
    	LibUI.queue_main do
    		conversation = $conversations[counterpart]
    		if conversation
    			conversation.new_message(m)
    		else
    			Conversation.open(counterpart, m)
    		end
    	end
    end

    And the helper that will handle messages from other devices of ours:

    def handle_carbons(fwd, counterpart:)
    	fwd = fwd.first if fwd.is_a?(Nokogiri::XML::NodeSet)
    	return unless fwd
    
    	m = Blather::XMPPNode.import(fwd)
    	return unless m.is_a?(Blather::Stanza::Message) && m.body.present?
    
    	handle_live_message(m, counterpart: counterpart.call(m))
    end

    This takes in the forwarded XML object (allowing for it to be a set of which we take the first one) and imports it with Blather’s logic to become hopefully a Message object.  If it’s not a Message or has no body, we don’t really care so we stop there. Otherwise we can handle this extracted message as though we had received it ourselves.

    And then wire up the event handlers:

    message(
    	"./carbon:received/fwd:forwarded/*[1]",
    	carbon: "urn:xmpp:carbons:2",
    	fwd: "urn:xmpp:forward:0"
    ) do |_, fwd|
    	handle_carbons(fwd, counterpart: ->(m) { m.from.stripped.to_s })
    end

    Because XMPP is just XML, we can use regular XPath stuff to extract from incoming messages.  Here we say that if the message contains a forwarded element inside a carbons received element, then we should handle this with the carbons handler instead of just the live messages handler.  The XML that matches our XPath comes in as the second argument and that is what we pass to the handler to get converted into a Message object.

    message(
    	"./carbon:sent/fwd:forwarded/*[1]",
    	carbon: "urn:xmpp:carbons:2",
    	fwd: "urn:xmpp:forward:0"
    ) do |_, fwd|
    	handle_carbons(fwd, counterpart: ->(m) { m.to.stripped.to_s })
    end

    This handler is for messages sent by other devices instead of received by other devices.  It is pretty much the same, except that we know the “other side of the conversation” (here called counterpart) is in the to not the from.

    message :body do |m|
    	handle_live_message(m)
    end

    And our old message-with-body handler now just needs to call the helper.

    History

    So far our client only processes and displays live messages.  If you close the app, or even close a conversation window, the history is gone.  If you chat with another client or device, you can’t see that when you re-open this one.  To fix that we’ll need to store messages persistently, and also fetch any history from while we were disconnected from the server.  We will need a few more lines in our Gemfile first:

    gem "sqlite3"
    gem "xdg"

    And then to set up a basic database schema:

    require "securerandom"
    require "sqlite3"
    require "xdg"
    
    DATA_DIR = XDG::Data.new.home + "jabber-client-demo"
    DATA_DIR.mkpath
    DB = SQLite3::Database.new(DATA_DIR + "db.sqlite3")
    
    if DB.user_version < 1
    	DB.execute(<<~SQL)
    		CREATE TABLE messages (
    			mam_id TEXT PRIMARY KEY,
    			stanza_id TEXT NOT NULL,
    			conversation TEXT NOT NULL,
    			created_at INTEGER NOT NULL,
    			stanza TEXT NOT NULL
    		)
    	SQL
    	DB.execute("CREATE TABLE data (key TEXT PRIMARY KEY, value TEXT)")
    	DB.user_version = 1
    end

    user_version is a SQLite feature that allows storing a simple integer alongside the database.  It starts at 0 if never set, and so here we use it to check if our schema has been created or not.  We store the database in a new directory created according to the XDG Base Directory specification.  There are two relevant IDs for most XMPP operations: the MAM ID (the ID in the server’s archive) and the Stanza ID (which was usually selected by the original sender).  We also create a data table for storing basic key-value stuff, which we’ll use in a minute to remember where we have sync’d up to so far.  Let’s edit the Conversation object to store messages as we send them, updating the send button on_clicked handler:

    def message
    	Blather::Stanza::Message.new(@jid, @message_entry.text, :chat).tap { |m|
    		m.id = SecureRandom.uuid
    	}
    end

    on_clicked do
    	m = message
    	EM.defer do
    		BLATHER << m
    		DB.execute(<<~SQL, [nil, m.id, @jid, m.to_s])
    			INSERT INTO messages
    			(mam_id, stanza_id, conversation, created_at, stanza)
    			VALUES (?,?,?,unixepoch(),?)
    		SQL
    	end
    	@messages << message_row(m)
    	@message_entry.text = ""
    end

    When we send a message we don’t yet know the server’s archive ID, so we set that to nil for now.  We set mam_id to be the primary key, but SQLite allows multiple rows to have NULL in there so this will work.  We don’t want to block the GUI thread while doing database work so we use EM.defer to move this to a worker pool.  We also want to store messages when we receive them live, so add this to the start of handle_live_message:

    mam_id = m.xpath("./ns:stanza-id", ns: "urn:xmpp:sid:0").find { |el|
    	el["by"] == jid.stripped.to_s
    }&.[]("id")
    delay = m.delay&.stamp&.to_i || Time.now.to_i
    DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
    	INSERT INTO messages (mam_id, stanza_id, conversation, created_at, stanza)
        VALUES (?,?,?,?,?)
    SQL

    Here we extract the server archive’s ID for the message (added by the server in a stanza-id with by="Your Jabber ID") and figure out what time the message was originally sent (usually this is just right now for a live message, but if it is coming from offline storage because every client was offline or similar, then there can be a “delay” set on it which we can use).  Now that we have stored the history of message we received we need to load them into the GUI when we start up a Conversation so add this at the end of initialize:

    EM.defer do
    	mam_messages = []
    	query = <<~SQL
    		SELECT stanza
    		FROM messages
    		WHERE conversation=?
    		ORDER BY created_at
    	SQL
    	DB.execute(query, [@jid]) do |row|
    		m = Blather::XMPPNode.import(
    			Nokogiri::XML.parse(row[0]).root
    		)
    		mam_messages << m
    	end
    
    	LibUI.queue_main do
    		mam_messages.map! { |m| message_row(m) }
    		@messages.replace(mam_messages + @messages)
    	end
    end

    In the worker pool we load up all the stored messages for the current conversation in order, then we take the XML stored as a string in the database and parse it into a Blather Message object.  Once we’ve done as much of the work as we can in we worker pool we use queue_main to switch back to the GUI thread and actually build the rows for the table and replace them into the GUI.

    With these changes, we are now storing all messages we see while connected and displaying them in the conversation.  But what about messages sent or received by other devices or clients while we were not connected?  For that we need to sync with the server’s archive, fetching messages at a reasonable page size from whatever we already have until the end.

    def sync_mam(last_id)
    	start_mam = Blather::Stanza::Iq.new(:set).tap { |iq|
    		xml_child(iq, :query, "urn:xmpp:mam:2").tap do |query|
    			xml_child(query, :set, "http://jabber.org/protocol/rsm").tap do |rsm|
    				xml_child(rsm, :max, "http://jabber.org/protocol/rsm").tap do |max|
    					max.content = (EM.threadpool_size * 5).to_s
    				end
    				next unless last_id
    
    				xml_child(rsm, :after, "http://jabber.org/protocol/rsm").tap do |after|
    					after.content = last_id
    				end
    			end
    		end
    	}
    
    	client.write_with_handler(start_mam) do |reply|
    		next if reply.error?
    
    		fin = reply.find_first("./ns:fin", ns: "urn:xmpp:mam:2")
    		next unless fin
    
    		handle_rsm_reply_when_idle(fin)
    	end
    end

    The first half of this creates the XML stanza to request a page from the server’s archive. We create a query with a max page size based on the size of our worker threadpool, and ask for messages only after the last known id (if we have one, which we won’t on first run). Then we use write_with_handler to send this request to the server and wait for a reply. The reply is sent after all messages have been sent down (sent seperately, not returned in this reply, see below), but we may still be processing some of them in the worker pool so we next create a helper to wait for the worker pool to be done:

    def handle_rsm_reply_when_idle(fin)
    	unless EM.defers_finished?
    		EM.add_timer(0.1) { handle_rsm_reply_when_idle(fin) }
    		return
    	end
    
    	last = fin.find_first(
    		"./ns:set/ns:last",
    		ns: "http://jabber.org/protocol/rsm"
    	)&.content
    
    	if last
    		DB.execute(<<~SQL, [last, last])
    			INSERT INTO data VALUES ('last_mam_id', ?)
    			ON CONFLICT(key) DO UPDATE SET value=? WHERE key='last_mam_id'
    		SQL
    	end
    	return if fin["complete"].to_s == "true"
    
    	sync_mam(last)
    end

    Poll with a timer until the worker pool is all done so that we aren’t fetching new pages before we have handled the last one.  Get the value of the last archive ID that was part of the page just processed and store it in the database for next time we start up.  If this was the last page (that is, complete="true") then we’re all done, otherwise get the next page.  We need to make sure we actually start this sync process inside the when_ready handler:

    last_mam_id = DB.execute(<<~SQL)[0]&.first
    	SELECT value FROM data WHERE key='last_mam_id' LIMIT 1
    SQL
    sync_mam(last_mam_id)

    And also, we need to actually handle the messages as they come down from the server archive:

    message "./ns:result", ns: "urn:xmpp:mam:2" do |_, result|
    	fwd = result.xpath("./ns:forwarded", ns: "urn:xmpp:forward:0").first
    	fwd = fwd.find_first("./ns:message", ns: "jabber:client")
    	m = Blather::XMPPNode.import(fwd)
    	next unless m.is_a?(Blather::Stanza::Message) && m.body.present?
    
    	mam_id = result.first["id"]&.to_s
    	# Can't really race because we're checking for something from the past
    	# Any new message inserted isn't the one we're looking for here anyway
    	sent = DB.execute(<<~SQL, [m.id])[0][0]
    		SELECT count(*) FROM messages WHERE stanza_id=? AND mam_id IS NULL
    	SQL
    	if sent < 1
    		counterpart = if m.from.stripped.to_s == jid.stripped.to_s
    			m.to.stripped.to_s
    		else
    			m.from.stripped.to_s
    		end
    		delay =
    			fwd.find_first("./ns:delay", ns: "urn:xmpp:delay")
    			&.[]("stamp")&.then(Time.method(:parse))
    		delay = delay&.to_i || m.delay&.stamp&.to_i || Time.now.to_i
    		DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
    			INSERT OR IGNORE INTO messages
    			(mam_id, stanza_id, conversation, created_at, stanza)
    			VALUES (?,?,?,?,?)
    		SQL
    	else
    		DB.execute(<<~SQL, [mam_id, m.id])
    			UPDATE messages SET mam_id=? WHERE stanza_id=?
    		SQL
    	end
    end

    Any message which contains a MAM (server archive) result will get handled here.  Just like with carbons we extract the forwarded message and import, making sure it ends up as a Blather Message object with a body.

    Remember how when we stored a sent message we didn’t know the archive ID yet?  Here we check if there is anything in our database already with this stanza ID and no archive ID, if no we will insert it as a new message, but otherwise we can update the row we already have to store the server archive ID on it, which we now know.

    And with that, our client now stores and syncs all history with the server, to give the user a full view of their conversation no matter where or when it happened.

    Display Names

    If a user is added to the contact list with a name, we already show that name instead of their address in conversations.  What if a user is not a contact yet, or we haven’t set a name for them?  It might be useful to be able to fetch any display name they advertise for themselves and show that.  First we add a simple helper to expose write_with_handler outside of the main object:

    public def write_with_handler(stanza, &block)
    	client.write_with_handler(stanza, &block)
    end

    We need an attribute on the Conversation to hold the nickname:

    attr_accessor :nickname

    And then we can use this in Conversation#initialize to fetch the other side’s nickname if they advertise one and we don’t have one for them yet:

    self.nickname = BLATHER.my_roster[jid]&.name || jid
    return unless nickname.to_s == jid.to_s
    
    BLATHER.write_with_handler(
    	Blather::Stanza::PubSub::Items.new(:get).tap { |iq|
    		iq.node = "http://jabber.org/protocol/nick"
    		iq.to = jid
    	}
    ) do |reply|
    	self.nickname = reply.items.first.payload_node.text rescue self.nickname
    end

    Inside the window declaration we can use this as the window title:

    title <=> [self, :nickname]

    and in format_sender we can use this as well:

    return nickname if jid.to_s == @jid.to_s

    Avatars

    Names are nice, but what about pictures?  Can we have nice avatar images that go with each user?  What should we display if they don’t have an avatar set?  Well not only is there a protocol to get an avatar, but a specification that allows all clients to use the same colours to represent things, so we can use a block of that if there is no avatar set.  Let’s generate the colour blocks first.  Add this to Gemfile:

    gem "hsluv"

    Require the library at the top:

    require "hsluv"
    
    $avatars = {}

    And a method on Conversation to use this:

    def default_avatar(string)
    	hue = (Digest::SHA1.digest(string).unpack1("v").to_f / 65536) * 360
    	rgb = Hsluv.rgb_prepare(Hsluv.hsluv_to_rgb(hue, 100, 50))
    	rgba = rgb.pack("CCC") + "xff".b
    	image { image_part(rgba * 32 * 32, 32, 32, 4) }
    end

    This takes the SHA-1 of a string, unpacks the first two bytes as a 16-bit little-endian integer, converts the range from 0 to MAX_SHORT into the range from 0 to 360 for hue degrees, then passes to the library we added to convert from HSV to RGB colour formats.  The GUI library expects images as a byte string where every 4 bytes are 0 to 255 for red, then green, then blue, then transparency.  Because we want a square of all one colour, we can create the byte string for one pixel and then multiply the string by the width and height (multiplying a string by a number in Ruby make a new string with that many copies repeated) to get the whole image.

    In Conversation#initialize we can use this to make a default avatar on the dummy message row then the window first opens:

    @messages = [[default_avatar(""), "", ""]]

    And we will need to add a new column definition to be beginning of the table { block:

    image_column("Avatar")

    And actually add the image to message_row:

    def message_row(m)
    	from = m.from&.stripped || BLATHER.jid.stripped
    	[
    		$avatars[from.to_s] || default_avatar(from.to_s),
    		format_sender(from),
    		m.body
    	]
    end

    If you run this you should now see a coloured square next to each message.  We would now like to get actual avatars, so add this somewhere at the top level to advertise support for this:

    set_caps(
    	"https://git.singpolyma.net/jabber-client-demo",
    	[],
    	["urn:xmpp:avatar:metadata+notify"]
    )

    Then in the when_ready block make sure to send it to the server:

    send_caps

    And handle the avatars as they come in:

    pubsub_event(
    	"//ns:items[@node='urn:xmpp:avatar:metadata']",
    	ns: "http://jabber.org/protocol/pubsub#event"
    ) do |m|
    	id = m.items.first&.payload_node&.children&.first&.[]("id")
    	next $avatars.delete(m.from.stripped.to_s) unless id
    
    	path = DATA_DIR + id.to_s
    	key = m.from.stripped.to_s
    	if path.exist?
    		LibUI.queue_main { $avatars[key] = image(path.to_s, 32, 32) rescue nil }
    	else
    		write_with_handler(
    			Blather::Stanza::PubSub::Items.new(:get).tap { |iq|
    				iq.node = "urn:xmpp:avatar:data"
    				iq.to = m.from
    			}
    		) do |reply|
    			next if reply.error?
    
    			data = Base64.decode64(reply.items.first&.payload_node&.text.to_s)
    			path.write(data)
    			LibUI.queue_main { $avatars[key] = image(path.to_s, 32, 32) rescue nil }
    		end
    	end
    end

    When an avatar metadata event comes in, we check what it is advertising as the ID of the avatar for this user.  If there is none, that means they don’t have an avatar anymore so delete anything we may have in the global cache for them, otherwise create a file path in the same folder as the database based on this ID.  If that file exists already, then no need to fetch it again, create the image from that path on the GUI thread and set it into our global in-memory cache.  If the file does not exist, then use write_with_handler to request their avatar data.  It comes back Base64 encoded, so decode it and then write it to the file.

    If you run this you should now see avatars next to messages for anyone who has one set.

    Delivery Receipts

    The Internet is a wild place, and sometimes things don’t work out how you’d hope.  Sometimes something goes wrong, or perhaps just all of a user’s devices are turned off.  Whatever the reason, it can be useful to see if a message has been delivered to at least one of the intended user’s devices yet or not.  We’ll need a new database column to store that status, add after the end of the DB.user_version < 1 if block:

    if DB.user_version < 2
    	DB.execute(<<~SQL)
    		ALTER TABLE messages ADD COLUMN delivered INTEGER NOT NULL DEFAULT 0
    	SQL
    	DB.user_version = 2
    end

    Let’s advertise support for the feature:

    set_caps(
    	"https://git.singpolyma.net/jabber-client-demo",
    	[],
    	["urn:xmpp:avatar:metadata+notify", "urn:xmpp:receipts"]
    )

    We need to add delivery status and stanza id to the dummy row for the messages table:

    @messages = [[default_avatar(""), "", "", false, nil]]

    And make sure we select the status out of the database when loading up messages:

    SELECT stanza,delivered FROM messages WHERE conversation=? ORDER BY created_at

    And pass that through when building the message rows

    mam_messages << [m, row[1]]

    mam_messages.map! { |args| message_row(*args) }

    Update the messages table to expect the new data model:

    table {
    	image_column("Avatar")
    	text_column("Sender")
    	text_column("Message")
    	checkbox_column("Delivered")
    	editable false
    	cell_rows @messages
    	@messages.clear if @messages.length == 1 && @messages.first.last.nil?
    }

    And update the row builder to include this new data:

    def message_row(m, delivered=false)
    	from = m.from&.stripped || BLATHER.jid.stripped
    	[
    		$avatars[from.to_s] || default_avatar(from.to_s),
    		format_sender(from),
    		m.body,
    		delivered,
    		m.id
    	]
    end

    Inbound messages are always considered delivered, since we have them:

    def new_message(m)
    	@messages << message_row(m, true)
    end

    And a method to allow signalling that a delivery receipt should be displayed, using the fact that we now hide the stanza id off the end of the rows in the table to find the relevant message to update:

    def delivered_message(id)
    	row = @messages.find_index { |r| r.last == id }
    	return unless row
    
    	@messages[row] = @messages[row][0..-3] + [true, id]
    end

    In the Send button’s on_clicked handler we need to actually request that others send us receipts:

    m = message
    xml_child(m, :request, "urn:xmpp:receipts")

    And we need to handle the receipts when they arrive:

    message "./ns:received", ns: "urn:xmpp:receipts" do |m, received|
    	DB.execute(<<~SQL, [received.first["id"].to_s])
    		UPDATE messages SET delivered=1 WHERE stanza_id=?
    	SQL
    
    	conversation = $conversations[m.from.stripped.to_s]
    	return unless conversation
    
    	LibUI.queue_main do
    		conversation.delivered_message(received.first["id"].to_s)
    	end
    end

    When we get a received receipt, we get the id attribute off of it, which represents a stanza ID that this receipt is for.  We update the database, and inform any open conversation window so the GUI can be updated.

    Finally, if someone requests a receipt from us we should send it to them:

    message :body do |m|
    	handle_live_message(m)
    
    	if m.id && m.at("./ns:request", ns: "urn:xmpp:receipts")
    		self << m.reply(remove_children: true).tap { |receipt|
    			xml_child(receipt, :received, "urn:xmpp:receipts").tap { |received|
    				received["id"] = m.id
    			}
    		}
    	end
    end

    If the stanza has an id and a receipt request, we construct a reply that contains just the received receipt and send it.

    Message Correction

    Sometimes people send a message with a mistake in it and want to send another to fix it.  It is convenvient for the GUI to support this and render only the new version of the message.  So let’s implement that.  First we add it to the list of things we advertise support for:

    set_caps(
    	"https://git.singpolyma.net/jabber-client-demo",
    	[],
    	[
    		"urn:xmpp:avatar:metadata+notify",
    		"urn:xmpp:receipts",
    		"urn:xmpp:message-correct:0"
    	]
    )

    Then we need a method on Conversation to process incoming corrections and update the GUI:

    def new_correction(replace_id, m)
    	row = @messages.find_index { |r| r.last == replace_id }
    	return unless row
    
    	@messages[row] = message_row(m, true)
    end

    We look up the message row on the stanza id, just as we did for delivery receipts, and just completely replace it with a row based on the new incoming message.  That’s it for the GUI.  Corrections may come from live messages, from carbons, or even from the server archive if they happened while we were disconnected, so we create a new insert_message helper to handle any case we previously did the SQL INSERT for an incoming message:

    def insert_message(
    	m,
    	mam_id:,
    	counterpart: m.from.stripped.to_s,
    	delay: m.delay&.stamp&.to_i
    )
    	if (replace = m.at("./ns:replace", ns: "urn:xmpp:message-correct:0"))
    		DB.execute(<<~SQL, [m.to_s, counterpart, replace["id"].to_s])
    			UPDATE messages SET stanza=? WHERE conversation=? AND stanza_id=?
    		SQL
    	else
    		delay ||= Time.now.to_i
    		DB.execute(<<~SQL, [mam_id, m.id, counterpart, delay, m.to_s])
    			INSERT OR IGNORE INTO messages
    			(mam_id, stanza_id, conversation, created_at, stanza, delivered)
    			VALUES (?,?,?,?,?,1)
    		SQL
    	end
    end

    The else case here is the same as the INSERTs we’ve been using up to this point, but we also check first for an element that signals this as a replacement and if that is the case we issue an UPDATE instead to correct our internal archive to the new version.

    Then in handle_live_message we also signal the possibly-open GUI:

    if (replace = m.at("./ns:replace", ns: "urn:xmpp:message-correct:0"))
    	conversation.new_correction(replace["id"].to_s, m)
    else
    	conversation.new_message(m)
    end

    We can now display incoming corrections, but it would also be nice to be able to send them.  Add a second button after the Send button in Conversation that can re-use the @message_entry box to correct the most recently sent message:

    button("Correct") {
    	stretchy false
    
    	on_clicked do
    		replace_row = @messages.rindex { |message|
    			message[1] == format_sender(BLATHER.jid.stripped)
    		}
    		next unless replace_row
    
    		m = message
    		m << xml_child(m, :replace, "urn:xmpp:message-correct:0").tap { |replace|
    			replace["id"] = @messages[replace_row].last
    		}
    		EM.defer do
    			BLATHER << m
    			DB.execute(<<~SQL, [m.to_s, @jid, @messages[replace_row].last])
    				UPDATE messages SET stanza=? WHERE conversation=? AND stanza_id=?
    			SQL
    		end
    		@messages[replace_row] = message_row(m, @messages[replace_row][-2])
    		@message_entry.text = ""
    	end
    }

    When the button is clicked we find the row for the most recently sent message, construct a message to send just as in the Send case but add the message correction replace child with the id matching the stanza id of the most recently sent message.  We send that message and also update our own local copy of the stanza both in the database and in the memory model rendered in the GUI.

    Conclusion

    There are a lot more features that a chat system can implement, but hopefully this gives you a useful taste of how each one can be incrementally layered in, and what the considerations might be for a wide variety of different kinds of features.  All the code for the working application developed in this article is available in git under AGPLv3+, with commits that corrospond to the path we took here.