close
  • chevron_right

    Dino package updates in Debian

    debacle · Friday, 10 September - 02:00 edit

Warning: Upstream warned against using this version! If in doubt, and you don't want to lose your local database, stay with the official releaase, e.g. 0.2.0-3 in Debian stable!

Note, that the latest git master of #Dino (dino-im), featuring audio/video calls, is now in #Debian unstable, testing, and stable-backports. Also, the same version with enhancements for small touchscreens patched in is available in Debian experimental, mainly for use on #Mobian. Happy chatting! #Jabber #XMPP

  • favorite

    1 Like

    DebXWoody

  • Pl chevron_right

    RcppSMC 0.2.4 on CRAN: Even More GSoC !!

    pubsub.slavino.sk / planetdebian · Friday, 3 September - 01:47 · 1 minute

A brand new release 0.2.4 of the RcppSMC package arrived on CRAN earlier today, with a dual delay for CRAN closing for a well-earned break, and then being overwhelmed when reopening. Other than that the processing was again versy smooth.

RcppSMC provides Rcpp-based bindings to R for the Sequential Monte Carlo Template Classes (SMCTC) by Adam Johansen described in his JSS article . Sequential Monte Carlo is also referred to as Particle Filter in some contexts.

The package started when I put some Rcpp bindings together based on Adam’s paper and library. It grew when Adam and I supervised Leah South during the 2017 iteration of the Google Summer of Code . And … now it grew again as we have Adam, Leah and myself looking over the shoulders of Ilya Zarubin who did very fine work during the 2021 iteration of the Google Summer of Code that just concluded! So we are now GSoC squared!

This release is effectively all work by Ilya and summarized below.

Changes in RcppSMC version 0.2.4 (2021-09-01)

  • Multiple Sequential Monte Carlo extensions (Ilya Zarubin as part of Google Summer of Code 2021)

    • Provide informative user output (convergence diagnostics) for PMMH example #50 (Ilya in #50 and #52 addressing #25 , bullet point 5)

    • Support for tracking of ancestral lines for base sampler class (Ilya in #56 )

    • Support for conditional SMC via derived conditionalSampler class (Ilya in #60 )

  • Add URL and BugReports to DESCRIPTION (Dirk in #53 )

Courtesy of my CRANberries , there is a diffstat report for this release .

More information is on the RcppSMC page . Issues and bugreports should go to the GitHub issue tracker .

If you like this or other open-source work I do, you can now sponsor me at GitHub .

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.


Značky: #Debian

  • Pl chevron_right

    Stretch to Buster upgrade issues: "Grub error: symbol ‘grub_is_lockdown’ not found", missing RTL8111/8168/8411 Ethernet driver and RTL8821CE Wireless adapter on Linux Kernel 5.10 (and 4.19)

    pubsub.slavino.sk / planetdebian · Thursday, 2 September - 16:30 · 6 minutes

I have been Debian Stretch running on my HP Pavilion 14-ce0000nq laptop since buying it back in April 2019, just before my presence at Oxidizeconf where I presented " How to Rust When Standards Are Defined in C ".

Debian Buster (aka Debian 10) was released about 4 months later and I've been postponing the upgrade as my free time isn't what it used to be. I also tend to wait for the first or even second update of the release to avoid any sharp edges.

As this laptop has a Realtek 8821CE wireless card that wasn't officially supported in the Linux kernel, I had to use an out-of-tree hacked driver to have the wireless work on Stretch kernels such as 4.19, it didn't even got along with DKMS, so all compilations and installations of it, I did them manually. More reason to wait for a newer release that would contain a driver inside the official kernel .

I was waiting for the inevitable and dreading the wireless issues, but since mid-august Bullseye became stable , turning Stretch into oldoldstable, I decided that I had to do the upgrade, at least to buster.

The Grub error and the fix

Everything went quite smooth, except that after the reboot, the laptop failed to boot with this Grub error:

error: symbol ‘grub_is_lockdown’ not found

I looked for a solution and it seemed everyone was stuck or the solution was unclear .

There is even a bug report in Debian about this error, bug #984760 .

Adding to the pile of confusion my own confused solution: I tried supergrubdisk2/rescatux, it didn't work for me, it might have been a combination of me using LVM and grub-efi-amd64. I also tried to boot in rescue mode the Buster first DVD (to avoid the need for network), I was able to enter the partition, mount the EFI partition, too, but since I didn't want to mess the setup even more or depend on an external USB stick, I didn't know where should I try to write the Grub EFI config - the root partition is on an NVME storage.

When buying the laptop it had FreeDOS installed on it and some HP rescue app, which I did not wipe when installing Debian. I even forgot where or how was the EFI installed on the disk and EFI, even if it should be more reliable and simpler, I never got the hang of it.

In the end, I realized that I could via BIOS actually select manually which EFI executable should be booted into, so I was able to boot with some manual intervention during boot into the regular system.

I tried regenerating the grub configuration, installing it and also tried restoring the default proper boot sequence (and I even installed refind in the system during my fumbling), but I think somewhere between grub-efi-amd64 reconfiguration and its reinstallation I managed to do the right thing, as the default boot screen is the Grub one now.

Hints for anyone reading this in the hope to fix the same issue, hopefully it will make things better, not worse (see the text below):

1) regenerate the grub config:

update-grub2

2) reinstall grub-efi-amd64 and make Debian the default

dpkg-reconfigure -plow grub-efi-amd64

When reinstalling grub-efi-amd64 onto the disk, I think the scariest questions were to these:

Force extra installation to the EFI removable media path?

Some EFI-based systems are buggy and do not handle new bootloaders correctly. If you force an extra installation of GRUB to the EFI removable media path, this should ensure that this system will boot Debian correctly despite such a problem. However, it may remove the ability to boot any other operating systems that also depend on this path. If so, you will need to make sure that GRUB is configured successfully to be able to boot any other OS installations correctly.

and

Update NVRAM variables to automatically boot into Debian?

GRUB can configure your platform's NVRAM variables so that it boots into Debian automatically when powered on. However, you may prefer to disable this behavior and avoid changes to your boot configuration. For example, if your NVRAM variables have been set up such that your system contacts a PXE server on every boot, this would preserve that behavior.

I think the first can be safely answered "No" if you don't plan on booting via a removable USB stick, while the second is the one that does the restoring.

The second question is probably safe if you don't use PXE boot or other boot method, at least that's what I understand. But if you do, I suspect by installing refind, by playing with the multiple efi* named packages and tools, you can restore that, or it might be that your BIOS allows that directly.

I just did a walk through of these 2 steps again on my laptop and answered "No" to the removable media question as it leads to errors when the media was not inserted (in my case the internal SD card reader), and "Yes" to making Debian the default.

It seems that for me this broke the FreeDOS and HP utilities boot entries from Grub, but I still can boot via the BIOS options and my goal was to have Debian boot correctly by default.

Fixing the missing RTL811/8168/8411 Ethernet card issue

As a side note for people with computers having Realtek RTL8111/8168/8411 Gigabit Ethernet Controller and upgrading to Buster or switching to a newer kernel, please note that you might end up having the unpleasant surprise even your Ethernet card to disappear because the r8169 driver is not loader by default.

I had to add it to /etc/modules so is loaded by default:

eddy@aptonia:/ $ cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
r8169

The 5.10 compatible driver for RTL8821CE wireless adapter

After the upgrade to Buster, the oldstable version of the kernel, 4.19, the hacked version of the driver I've been using on Stretch on 4.9 kernels was no longer compatible - failed to compile due to missing symbols.

The fix for me was to switch to the DKMS compatible driver from https://github.com/tomaspinho/rtl8821ce , as this seems to work for both 4.19 and 5.10 kernels (installed from backports).

I installed it via a modification of the manual install method only for the 4.19 and 5.10 kernels, leaving the legacy 4.9 kernels working with the hacked driver. You can do the same if instead of running the provided script, you do its steps manually and you install only for the kernel versions you want, instead of the default to install for all:

I looked inside the dkms-install.sh script to do the required steps:

Copy the driver, add it to the dkms set of known drivers:

DRV_NAME=rtl8821ce
DRV_VERSION=v5.5.2_34066.20200325

cp -r . /usr/src/${DRV_NAME}-${DRV_VERSION}

dkms add -m ${DRV_NAME} -v ${DRV_VERSION}

But you just build and install them only for the select kernel versions of your choice:

dkms build -m ${DRV_NAME} -v ${DRV_VERSION} -k 5.10.0-0.bpo.8-amd64
dkms install -m ${DRV_NAME} -v ${DRV_VERSION} -k 5.10.0-0.bpo.8-amd64

Or, without the variables:

dkms build rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64
dkms install rtl8821ce/v5.5.2_34066.20200325 -k 4.19.0-17-amd64

dkms status should confirm everything is in place and I think you need to update grub2 again after this.

Please note this driver is no longer maintained and the 5.10 tree should support the RTL8821CE wireless card with the rtw88 driver from the kernel , but for me it did not. I'll probably try this at a later time, or after I upgrade to the current Debian stable, Bullseye.


Značky: #Debian

  • Pl chevron_right

    partial-borrow: references to restricted views of a Rust struct

    pubsub.slavino.sk / planetdebian · Thursday, 2 September - 16:28 · 3 minutes

tl;dr:
With these two crazy proc-macros you can hand out multipe (perhaps mutable) references to suitable subsets/views of the same struct.

Why

In Otter I have adopted a style where I try to avoid giving code mutable access that doesn't need it, and try to make mutable access come with some code structures to prevent "oh I forgot a thing" type mistakes. For example, mutable access to a game state is only available in contexts that have to return a value for the updates to send to the players. This makes it harder to forget to send the update.

But there is a downside. The game state is inside another struct, an Instance , and much code needs (immutable) access to it. I can't pass both &Instance and &mut GameState because one is inside the other.

My workaround involves passing separate references to the other fields of Instance , leading to some functions taking far too many arguments. 14 in one case. (They're all different types so argument ordering mistakes just result in compiler errors talking about arguments 9 and 11 having wrong types, rather than actual bugs.)

I felt this problem was purely a restriction arising from limitations of the borrow checker. I thought it might be possible to improve on it. Weeks passed and the question gradually wormed its way into my consciousness. Eventually, I tried some experiments. Encouraged, I persisted.

What and how

partial-borrow is a Rust library which solves this problem. You sprinkle #[Derive(PartialBorrow)] and partial!(...) and then you can pass a reference which grants mutable access to only some of the fields. You can also pass a reference through which some fields are inaccessible. You can even split a single mut reference into multiple compatible references, for example granting mut access to mutually-nonverlapping subsets.

The core type is Struct__Partial (for some Struct ). It is a zero-sized type, but we prevent anyone from constructing one. Instead we magic up references to it, always ensuring that they have the same address as some Struct . The fields of Struct__Partial are also ZSTs that exist ony as references, and they Deref to the actual field (subject to compile-type borrow compatibility checking).

Soundness and testing

partial-borrow is primarily a nontrivial procedural macro which autogenerates reams of unsafe . Of course I think it's sound, but I thought that the last two times before I added a test which demonstrated otherwise. So it might be fairer to say that I have tried to make it sound and that I don't know of any problems...

Reasoning about the correctness of macro-generated code is not so easy. One problem is that there is nowhere good to put the kind of soundness arguments you would normally add near uses of unsafe .

I decided to solve this by annotating an instance of the macro output . There's a not very complicated script using diff3 to help fold in changes if the macro output changes - merge conflicts there mean a possible re-review of the argument text. Of course I also have test cases that run with miri, and test cases for expected compiler errors for uses that need to be forbidden for soundness.

But this is quite hairy and I'm worried that it might be rather "my first insane unsafe contraption". Also the pointer/reference trickery is definitely subtle, and depends heavily on knowing what Rust's aliasing and pointer provenance rules really are. Stacked Borrows is not entirely trivial to reason about in fiddly corner cases.

So for now I have only called it 0.1.0 and left a note in the docs. I haven't actually made Otter use it yet but that's the rather more boring software integration part, not the fun "can I do this mad thing" part so I will probably leave that for a rainy day. Possibly a rainy day after someone other than me has looked at partial-borrow (preferably someone who understands Stacked Borrows...).

Fun!

This was great fun. I even enjoyed writing the docs.

The proc-macro programming environment is not entirely straightforward and there are a number of things to watch out for. For my first non-adhoc proc-macro this was, perhaps, ambitious. But you don't learn anything without trying...

edited 2021-09-02 16:28 UTC to fix a typo



comment count unavailable comments

Značky: #Debian

  • Pl chevron_right

    Reducing (sparsifying) qcow2 image of Windows10

    pubsub.slavino.sk / planetdebian · Thursday, 2 September - 03:17

Since joining Fujitsu I am permanently running a VM (kvm/qemu) with Windows 10 (unfortunately necessary). While the usaged disk space is about 50G, the actual qcow2 file had grown to over 180G, not good.

Searching the web the very promising virt-sparsify is mentioned again and again, and the man page gives hope, but as it turns out it is broken and calls qemu-img with incorrect/not-working arguments (see this bug ).

Another problem seems to be that by default the discard mode seems not to be set to unmap .

So here are the stages how I reduced the size of the qcow2 image back down to around 60G.

  1. Turn off the VM
  2. Make sure you are using VirtIO for the disk, select Discard mode: unmap
  3. Boot into Windows, and from an elevated prompt run: Optimize-Volume -DriveLetter C -ReTrim -Verbose
  4. Shut down the VM
  5. Run a dummy conversion which sparsifies the image: qemu-img convert -O qcow2 orig.qcow2 sparse.qcow2
  6. Rename/Backup the images and boot back into Windows.

That helped in my case, without any consequences (till now) for the Windows installation.


Značky: #Debian

  • Pl chevron_right

    20210901-Debian-Reunion-Hamburg-2021

    pubsub.slavino.sk / planetdebian · Wednesday, 1 September - 22:22 · 1 minute

Debian Reunion Hamburg 2021

Moin!

I'm glad to finally be able to send out this invitation for the "Debian Reunion Hamburg 2021" taking place at the venue of the 2018 & 2019 MiniDebConfs!

The event will run from Monday, Sep 27 2021 until Friday Oct 1 2021, with Sunday, Sep 26 2021 as arrival day. IOW, Debian people meet again in Hamburg. The exact format is less defined and structured than previous years, probably we will just be hacking from Monday to Wednesday, have talks on Thursday and a nice day trip on Friday.

debianreunionhh2021.png

Please read https://wiki.debian.org/DebianEvents/de/2021/DebianReunionHamburg if you intend to attend, especially

Probably having some video coverage would be very nice to have, though due to this very late announcement I'm not sure we'll really have talks and the need for video. The event is in 3.5 weeks and will take place, either as a very small hack meeting, or somewhat bigger. We certainly want videoing if we have talks - and if you could help with this that would be very great!

Last and definitly not least, financial sponsors for the event would be great. If you can support the "Debian Reunion Hamburg 2021", please contact me directly!

Now, late, after weeks of wondering if and how to do this event, I'm finally and very much looking forward to it, to meet some Debian folks at least & for some shared Debian hacking. Definitly not the 2021 event I had in mind after the 2019 one, but something I feel I can responsibly do & enjoy.

So, hoping to see some of you soon & most of you later! ;-) Sad but true, and at least something for some people. We should all do more local events. And more online events too, eg I think this is a great idea too: https://wiki.debian.org/DebianEvents/internet/2021/MiniDebConfOnlineBookworm

See you!


Značky: #Debian

  • Pl chevron_right

    FLOSS Activities August 2021

    pubsub.slavino.sk / planetdebian · Wednesday, 1 September - 01:10 · 1 minute

Focus

This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Review

Administration

  • Debian servers: expand LV, fix debbugs config
  • Debian wiki: unblock IP addresses, approve accounts
  • Debian QA services: deploy changes

Communication

Sponsors

The pyemd, pytest-rerunfailures, libpst, sptag, librecaptcha work was sponsored by my employer. All other work was done on a volunteer basis.


Značky: #Debian

  • Pl chevron_right

    Already September.

    pubsub.slavino.sk / planetdebian · Tuesday, 31 August - 23:59

Already September. Trying to update first devices to bullseye. Many raspberry pis.


Značky: #Debian