close
  • chevron_right

    A little update from the Movim project

    Timothée Jaussoin · pubsub.movim.eu / Movim · Tuesday, 16 November - 21:29 edit · 1 minute

It's been a while since I have told you about Movim. I was quite busy the past few months with many other things in my personal life, but this doesn't mean that I haven't worked on Movim!

So in short, here is a few things that have been changed recently:

api.movim.eu

api.movim.eu has been upgraded and refreshed with a new Laravel backend, Google ReCAPTCHA has been replaced with hCaptcha

The Legals page will be soon upgraded. A specific ticket has been opened to allow you to report and bring ideas on how to improve it.

End of the Android application

Following our previous blog post about the state of the Android application I decided to officially archive the Android app and remove it from F-Droid.

Some work has been done to integrate Movim better as a #PWA (Progressive Web App) to still allow you to enjoy it from your mobile phone.

Security Audit

We had an extensive #SecurityAudit by Radically Open Security that did an amazing job.

The audit covered Movim itself but also api.movim.eu and all the related projects, including movim/movim_docker. The result of the audit is currently restricted but we are actively using the finding to improve the general project security:

An finally on Movim itself

  • I added a basic support of OMEMO for MUC Groups, you can already try it out on the official pod
  • And as always, bugs fixes!

That's all folks!

edhelas

  • favorite

    10 Like

    sthaan, odomar, Marzanna, drone839, samuel, joergd, debacle, matlag, Blue, aŭtunido

  • chevron_right

    Element and Movim Messengers Comparison Made Simple - ubuntubuzz.com

    Timothée Jaussoin · pubsub.movim.eu / Movim · Sunday, 3 October - 20:05 edit

A nice #comparison between #Movim (XMPP) and Element (Matrix) from similar features. Thanks !

  • chevron_right

    We are looking for some help for the Movim Android application

    Timothée Jaussoin · pubsub.movim.eu / Movim · Monday, 31 May - 20:17 edit · 2 minutes

Hi,

I am writing this little post today to ask for (a bit) of help on the official Movim #Android application that is currently available there.

I am maintaining #Movim for more than 10 years now, mostly without asking for any help from the hundreds of daily users that we have now (on the servers we're aware of, at least). I am also maintaining and updating the infrastructure hosting the website, the official pod and the #XMPP server and related services.

However Movim is a little more than that, there is also an Android app, and a desktop app (currently abandoned).

I am maintaining the Android app with the really thin knowledge that I have on this platform. Most of the code of the application is basically copy/pasted from StackOverflow and a bit hacky. Fixing and developing this app is always really time consuming as the environment, and the way of dealing with it, is really different than the knowledge that I have on the web development side.

So basically today I'm asking from some help for the official Movim Android application. If you have a little bit of experience in Android development (or if you want to learn !) and if you're willing to help me to improve the application do not hesitate to contact me through our official chatroom, directly on Github or as a comment of this post.

The app is quite simple. It is basically a WebView that shows one of the configured pods. And… that's mostly it.

What would be really helpful would be to have (non exhaustive list):

  • Another pair of eye to check and maybe refactor the few classes that are contained in the application
  • Create two flavors of the app, one "Play Store ready" and one "free of Google Play" integration (the current app cannot be published on F-Droid because there is some strong dependencies there)
  • Fix the mic/camera support and authorizations
  • See if the current notification system can be improved
  • Send an event to Movim when the app is "put in background" in a chat conversation to ensure that the notifications are re-enabled in this specific case
  • And any other nice feature that you would like to see integrated

There is no pressure or deadline, any pull requests that could help improving or fixing things in the application would be really appreciated.

I am currently dealing with Google to re-enable the application on the Play Store as well, I will keep you updated about that.

Regards,

edhelas

  • Pictures 1 image

  • visibility
  • favorite

    8 Like

    Ryu Shirogane, valeg, Arne, bung, Kefah Issa, arie, ThurahT, adb

  • 2 Comments

  • 24 August Kefah Issa

    First of all: Hats off for the amazing accomplishment. I just came across movim today and I'm pretty impressed.

    How can I learn more about the codebase? What technologies did you chose? I can see that the android app is almost identical to the web, so I assume that the web version is embedded into the android app.

    Do you have the code/issues on github?

  • 24 August Kefah Issa

    My bad! just noticed the github link on your post https://github.com/movim/movim_android
    Will look into it and come back. Thanks!

  • chevron_right

    Movim 0.19 - Ikeya is out!

    Timothée Jaussoin · pubsub.movim.eu / Movim · Saturday, 20 February, 2021 - 21:58 edit · 7 minutes

This version is a major step for Movim with a lot of improvements on the performances and many (many!) small fixes and new features.

Chat and chatrooms

A criticism that was coming once in a while is how difficult it was to join and/or create a chatroom in Movim. The 0.19 version follow the Modern XMPP Multi-user Chats recommendations that is already implemented in several other XMPP clients. This standard is bringing a common naming convention and flow to create and configure chatrooms in XMPP.

Add chatroom dialog

On top of that the join/create chatroom flow was completely redesigned. The new one is inspired by the Telegram one, with one unique "+" button to start a new one-to-one or one-to-many conversation. The global chatroom search was also directly integrated in the flow to allow you to quickly search and join one of the few thousands already listed ones.

The one-to-one chat conversations views are now cached server side. This simple trick is saving some precious milliseconds during the page load. The performance boost allowed us to simply remove the chat list placeholder (you don't need a placeholder if the content is loading instantly).

Conversations improvements

The encrypted messages are now displayed in the bubbles.

An encrypted message

You can receive messages read confirmation in group chats and in small channels (when there are less than 10 persons connected).

XEP-0201: "Best Practices for Message Threads" has been implemented. Now Movim handles replies on received messages. Useful to follow multiple topics during a conversation.

A replied message

Like images before, now videos are also embedded in conversations with the support of the WebM and H264 video format.

With this awesome feature, the Tenor API was also integrated. You can now search in millions of GIFs and publish them instantly in your conversations. The Tenor API is disabled by default, the server administrator can enable it by adding its custom API key in the Movim admin panel.

The GIF picker

All the published message URLs are also now embedded in the discussions. Now you can also easily browse the URL related pictures directly in the Movim picture preview gallery widget.

An embeded URL

Chat rooms with unread messages are put on top of the chat rooms list now.

A package of ~1500 new emojis were added to the emoji picker.

A couple of new emojis

Communities and blogs

The explore page was reorganized with a few new features. You can now filter the explore panel to return only the communities or blog articles (the same way you can filter the articles on the news page). A "More" button is also now allowing you to retrieve older articles.

The new discover page

The communities on the servers are now ordered by last updated.

But also…

The preview widget is now having a "copy to clipboard" button. Useful!

Preview and copy easily the picture link

There is a first integration of the Firebase Push notifications in Movim. This is currently in test with the Android app and will require some more improvements and adjustments before being released as a stable feature.

You can now directly drag & drop files or copy & paste pictures in Movim to trigger the upload dialog box.

The drag and drop box

Performances

Batch insertion

Lets now dive in the more technical aspects with the performances improvements.

Most of the performances bump in Ikeya are actually regarding how he database is requested. In the previous Movim version, a presence buffer was introduced to save "batches" of incoming requests in one database query and not hundred of them.

The presence is a core concept of XMPP (eXtensible Messaging and Presence Protocol). During the connection, you will receive a presence from each online contact from your contact list, similarly, when you join a chatroom, you will as well receive a presence for each member connected to the room.

So now imagine that you have a big account, with hundreds of contacts and a few dozen chat rooms with hundreds of connected users in each of them. During the login, your XMPP client will then receive thousands of presences. The presence buffer is there to "stack them" when they are received and save them in batches in the database and then notifies the UI (your browser) only once that "all those presences were saved".

This presence buffer element was greatly improved and will not only save the presences in batches now, but also handle some related data like capabilities requests (those information are used to know what your contacts XMPP clients are capable of) as well as vcard requests (your contact personal information and avatars).

In a similar aspect, the retrieved bookmarked conferences and community subscriptions are also now saved in batches in the database, saving dozens of requests.

Preload

Another important optimisation that can be done when dealing with databases is to preload some information when retrieving list of items. Lets imagine that you need to retrieve 50 messages from the database. Those messages might have related data, such as reactions, replies or attached files. It is often good to load in parallel all those data "in batches".

The wrong way
1. Get the messages A, B, E and F
2. Process the messages
  - Check if A has an attached file
  - Check if B has an attached file
  …

The good way
1. Get the messages A, B, E and F
2. Try to get files for the A, B, E and F messages
3. Once the two lists are retrieved, see if some files matches the messages

This was already done in many places in Movim (hopefully), but even after years we are still finding some small improvements that were missing. All those small improvements are now allowing Movim to retrieve full conversations and scroll the history in a few hundred milliseconds, even if the database is filled with millions of messages.

You should know that each time you open a discussion in Movim, your browser is actually asking the server about all the contact that will be displayed, there is no local cache. If you have a backend that is fast enough, you don't need complex Javascript frontends to process and store things ;)

Database connection auto-close

Each connected user on a Movim instance is having it's own specific process launched on the server. This is bringing some nice isolation and performances distribution, especially on servers with many parallel available threads (this also brings some memory consumption issues that will be improved in the upcoming versions).

For each of those launched "user-session-processes" a database connection was opened to allow them to perform database queries quickly.

The main issue there is that database servers (such as MySQL and PostgreSQL, the two supported servers for Movim) can only handle a certain limit of parallel connections. Movim is then now automatically closing unused connections after a few seconds and resume them once a new query is sent. This allows Movim to handle hundreds of parallel sessions without overloading the database server anymore.

Translations cache

The languages translations are now cached when the daemon is launched once for all. This brings a few milliseconds gain during page load. It's not much but it's an easy win!

The translations can also be manually recompiled using the following command

php daemon.php compileLanguages

Picture proxyfying and caching

Most of the Movim pictures are now proxyfied by Movim. This way the Movim pod will hide its users personal IPs and browser information when the pictures are retrieved. The proxy is also moving all the pictures URL under a specific picture/ directory. This simple trick allows server admins to create a nice caching system directly on the web-server level.

For example, if a friend in a chatroom is sharing a nice picture that is a bit heavy. The first person that will receive the picture will ask Movim, through the Movim web-server to download it and return an optimized version of it. The web-server will then keep a copy of the optimized picture. All the other users will then ask the same resource, and the web-server will simply return the cached version, without even asking anything to Movim anymore.

A specific section in the Movim setup tutorial was added to explain you how to setup and configure this cache system during the Movim deployment.

To conclude

This was quite a big version indeed, we hope that you will enjoy it. The upcoming one will be a special release and will focus on one specific feature. We will come back to it in the upcoming weeks, be patient.

A clue about the upcoming release?

As always, if you like Movim please share it around. You can always help us by directly contributing to the project and help funding us on our Patreon.

That's all folks!

  • Pictures 1 image

  • visibility
  • favorite

    11 Like

    valeg, typhoon, adb, Arne, Archer Ships, Odysseus Libre, Xabi, DebXWoody, debacle, quatta, Ryu Shirogane

  • chevron_right

    Movim | Basic Review & Beginner's Guide

    Timothée Jaussoin · pubsub.movim.eu / Movim · Sunday, 14 February, 2021 - 11:17 edit

Thanks for this awesome guide by UbuntuBuzz. With a lot of details and some ideas of improvements for the future. #Movim #review #guide

  • favorite

    9 Like

    wxyz, Federico, Yannv, U, Xabi, Matt, ericbuijs, debacle, eyome

  • 1 Comments

  • 10 April, 2021 preptorrent

    Please can you put Movim back so html tables can be written in our posts the old way?

    Sadly, Movim have now changed the ability to write in html so your tables end up garbled. What a terrible shame. I wish Movim had just left it as it was before. It was fine as html! Nowadays it forces your markup to be their Movim markdown which is simply nowhere as neatly laid out as html with tables. Also, people have long understood html.

    I feel gutted about this. I'm not a fan of markdown being mandatory in place of html. Please can you put Movim back so html can be written the old way? I used to show people how cool movim was for that. It used to be so cool.

  • chevron_right

    Movim is moving!

    Timothée Jaussoin · pubsub.movim.eu / Movim · Wednesday, 20 January, 2021 - 06:47 edit

Following our previous article, with the sudden subscription of hundred of new users both on our XMPP #services movim.eu and jappix.com and on #Movim itself it seems that we are starting to be quite limited by our current servers capacity.

You might have noticed some connection issues and downtimes the past few days that are the direct result in this recent gain of interest for Movim.

In the upcoming days we are planning to migrate the whole #infrastructure to a new server to give a bit more space for the project to grow and to allow many new users to join us.

This will involve some extra server costs. If you want to help us covering our expenses you can always join our Patreon.

By the way, we also bumped the size limit from 3Mb to 10Mb on our XMPP upload service, enjoy!

  • favorite

    10 Like

    John, U, Himesaka Noa, adb, alexisinparis, Minh Le, Yannv, purplesky, Eclipcity Global, jsm

  • chevron_right

    Good news everyone!

    Timothée Jaussoin · pubsub.movim.eu / Movim · Monday, 18 January, 2021 - 12:36 edit

So it seems that we doubled our daily registrations the past few days 🎉 .

Thanks to #WhatsApp lots of new people are discovering XMPP and its ecosystem . We also recently rolled-out an improved and simplified flow for the registration process. You can find it there Register on movim.eu and share the link around.

We will also communicate soon about some big news about #Movim itself, stay tuned 😉

  • favorite

    10 Like

    Minh Le, matlag, DebXWoody, Mathias Poujol-Rost ✅ , Matt, U, warren, adb, Miho, eyome