Public Movim Post Of Turkey Syria Earthquake Article

https://mov.im/?blog/preptorrent%40movim.eu/earthquakes-2023-turkey-or-syria-and-compassion-from-ham-radio-free-software-70tsza

Earthquakes between 7.4 and 7.7 (or 7.8) Magnitude (but also 6.7 Magnitude for a late shallow Earthquake) struck Turkey and Syria Monday 6 February 2023. Turkey Gaziantep then Ekinözü (a 7.5 Magnitude Earthquake struck) and the Kahramanmaras region is a struck epicentre, also in Syria Besnaya - Bseineh and Sarmada (North West Syria).

USGS (United States Geological Survey, for USA Government) Data Reports the earthquakes for time and location as 2023-02-06 01:17:35 (UTC)37.174°N 37.032°E17.9 km depth

An Internet Intensity Map from USGS shows internet activty and the map is big enough to cover where the erathquake struck.

https://earthquake.usgs.gov/realtime/product/dyfi/us6000jllz/us/1675727221027/us6000jllz_ciim_geo.jpg

USGS United States Geological Survey

https://earthquake.usgs.gov/earthquakes/eventpage/us6000jllz/executive

Disaster and Emergency Management Authority in Turkey

https://en.afad.gov.tr/about-us

HAM Radio can be used to send information by voice but also a data Mode can send Morse code or Geographical location data and Wether and so forth, and so any short text message can be sent (such as even through morse code if you so desired). The LORAWAN technologies (if you have such equipment) can also be used aside form HAM Radio for small amounts of data such as sending a start and end time-date stamp to indicate the duration of an event, along with geographical location data as longitude, latitude and altitude and then a magnet link of a torrent file for an ISO of a CDROM containing information on the topic you wish to share which could later be downloaded by the internet when it becomes avaiable (so the CDROM might have some video file stored on it and text files and PDF files and 3D-Model Files like a collada-file for software such as 'Blender' and audio files such as mp3 or Ogg Vorbis or it might be a mixed-mode CD which plays some audio in a Audio CD-Player but also has some computer data if placed into a PC computer such as MSWindows or Linux). HAM Radio exists in Syria and in Turkey and can help communications and this is important for helping people and for science data like geology and physics which might pertain more and more to the situation. Sole Linux is a debian linux software which can be used to install HAM radio software and use OCr software and FireFox Web Browser nd it can share geology and physics science information such as by installing SciLab for free. Learning Languages can be done in Liunx too. Gaussian Heat-Maps could be generated to use Kernelized classification of geological and disaster relife areas and communication HAM Radio spots possibly with LoRaWAN shown, and so OpenID built atop of OAuth2 as OIDC can allow single-sign-on and federated log-in to bring together the information such as by the Fediverse. Kernel density estimation of an hypercube could be accomkplished via the Parzen window via the Rosenblatt-Parzen method, such that OpenCL (especially OpenCL1.2 or higher) is used as an heterogenous computing method to distributed processing over low-power devices including a raspberry-pi3b (or Pizero2W) GPU or GT710 Nvidia Desktop PC GPU or even a large server so as to balance the computational efforsts as scalable. SciLab can help with this and can run on a RaspberryPi3b+ or a home Linux computer (such as the notion of a 64bit linux distro desktop PC with a GPU as good as a Nvidia GT710 installed or a AMD HD6450 or better). Machine Learning or GPU neural networks can use Kernelization methodology.

Gaussian Kernel Density Estimation Explained on Wikipedia

https://en.wikipedia.org/wiki/Kernel_density_estimation

Gaussian Kernel Method explained on Wikipedia

https://en.wikipedia.org/wiki/Kernel_method

Ham Radio and Getting Started with Data Modes

https://www.essexham.co.uk/how-to-get-started-with-data-modes

Syrian Scientific Technical Amateur Radio Society SSTARS

https://en.wikipedia.org/wiki/Syrian_Scientific_Technical_Amateur_Radio_Society

Technical Institute of Radio by a group of amateur radio enthusiasts in Damascus

https://www.qsl.net/on4ww/Syria.html

There are various free OCR (Optical character Recognition softwares such as Copyfish (linked below and OCR Space) which can be used on PC (Windows Or Linux) and smartphone app, but also Tesseract with a9t9 frontend installed to a MSWindows PC. Also consider GOCR open-source character recognition (e.g. gocr049.exe), and (GPL License) A9t9 Free OCR (probably the easiet for MSWindows if you are willing to use free Tesseract OCR Software which has a good License, BSD/Apache style). You would thereby download the (a9t9)FreeOcrWindowsDesktop-Setup.exe file.

Free-Ocr-Windows-Desktop (a9t9)FreeOcrWindowsDesktop-Setup.exe

https://github.com/A9T9/Free-Ocr-Windows-Desktop/releases

The (a9t9)FreeOcr blog and documentation hyperlinks

https://onlineocr.a9t9.com/blog/p/free-ocr-software.html

(GOCR)[https://jocr.sourceforge.net/download.html]

https://jocr.sourceforge.net/download.html

For example, these languages can be recognised in he free OCR software by scanning (or taking a photograph using a digital camera or smartphone or webcam connected to a PC). Once the text is on your computer it can br translated into other languages as text. also, text can be converted into speech or SRT files as subtitles which can be added to a DVD video you make on your computer or an Mpeg4 video (or Ogg Vorbis theora in free software like Openshot or KdenLive). Text can be "spoken by a computer" using voice synthesis software. Translations of text between languages can be helped by free software like "FireFox Translations" (hyperlinked below) which you can use in the Mozilla FireFox Web-Browser and you can use Thunderbird to read RSS feeds. Discover the web pages about the OCR Languages below. Languages to use the OCR Free software with include Turkish OCR, Arabic OCR, English OCR (*), French OCR, German OCR, Portuguese OCR, Russian OCR, Spanish OCR, Chinese OCR (Simplified and traditional characters), Hindi OCR, Korean OCR, Italian OCR

OCR Languages

Arabic OCR

https://ocr.space/arabic

English OCR

https://ocr.space/english

French OCR

https://ocr.space/french

German OCR

https://ocr.space/german

Portuguese OCR

https://ocr.space/portuguese

Russian OCR

https://ocr.space/russian

Spanish OCR

https://ocr.space/spanish

Chinese OCR

https://ocr.space/chinese

Hindi OCR

https://ocr.space/hindi

Korean OCR

https://ocr.space/korean

Japanese OCR

https://ocr.space/japanese

Italian OCR

https://ocr.space/italian

ESword software can be installed on MSWindows (WindowsXP through to Windows10) or Sword Modules can be used in "BibleTime" software for Linux (such as in the software center or synaptic for Debian), MacOSX and Windows.

BibleTime software for Linux, MacOSX and Windows

https://bibletime.info/

ESword bible Software for MSWindows

https://e-sword.net/downloads.html

Read these other links to learn more about this information for using software and how to help people such as with health or nutrition via HugelKultur by readin the articles in these links and then seeing the subsequently linked articles as blog posts hyperlinked. Free Wikibooks First Aid documentation is linked so it can help people as a known quantity book to communicate about as a topic over the internet or LoRaWAN or HAM Radio. A British Military experience wound surgery research medical science paper shows the treatment of combat wounds but some wounds from natural disasters like Earthquakes can be somewhat similar and the medical knowledge of what to expect can further assist people dealing with communicating about the terminology expected for the topic.

Public Post version of IRC Donation

https://mov.im/?blog/preptorrent%40movim.eu/how-i-made-a-12-donation-to-international-rescue-committee-t1R0E8

Public Post version of 5 GBP Pakistan Flood Donation

https://mov.im/?blog/preptorrent%40movim.eu/how-i-made-5-gbp-christmas-donation-to-disasters-emergency-committee-pakistan-be6zVh

Quick Notice: For the other money donation to Gnome Foundation, see this other link.

Money donation to Gnome Foundation

https://mov.im/?blog/preptorrent%40movim.eu/donating-5-usd-to-gnome-foundation-jp4OiJ

A guide to radio communications technologies for Linux computing for Urdu

https://mov.im/?blog/preptorrent%40movim.eu/ab82747e-5183-4fe3-a260-115144cb4846

You might have been inspired to read my article (A guide to radio communications technologies for Linux computing for Urdu) on how HAM Radio can help the floods and also Afghanistan with a RaspberryPi for SDR (Software Defined Radio, such as by using Linux LimeSuite software) using a RTL2832U dongle and various open-source software like Soapy-Remote-Server, and when plugging in (the USB ports of a Raspberry Pi3b+ using GuvcView software and "cheese") a webcam (costing btween £1 and $5 USD), with a built-in microphone like a Playstation2-Eyetoy but a better (usb-converted) microphone can help too.

Arabic English Translation of How To Learn linux SDR for rtl2832u

https://upload.movim.eu/files/72545d95562ce01c8d7e25cb0e413a679c22c9f2/l1U1VDGDU3bt/arabicSDRrtl2832u.bmp.jpg

To be knowledgeable about how your study of HAM Radio or Linux can help the people communicating around the Earthquake disaster topics and the Pakistan Floods (and other regions like Afghanistan), you may seek self-improvement as you learn about how to get linux via a torrent file and how to use a CD-Writer (and other knowledge) in this (German and English language) article linked below (German and English... Deutschland macht gute Technologien wie Knoppix Linux). Also, learn about how to use linux (and prepare your amd64 PC) using Version 5.3.8 ultimatebootcd such as to backup data and recover lost data and to partitition drives or clone an HDD (all of which can be discovered by reading and talking on the UBCD forums), and so all that information can be seen on the article linked below as "Wifi Bandwidth Management Hong Kong".

Wifi Bandwidth Management Hong Kong

https://mov.im/?blog/preptorrent%40movim.eu/0a07613c-ef42-4ed0-b3d2-2bffc33ef519

Wifi Bandwidth Management Hong Kong

Amharic Ethiopia Linux Computer ሊኑክስ ኮምፕዩተር

https://mov.im/?blog/preptorrent%40movim.eu/6a4795ca-85be-4777-85c3-624eed054e25

Amharic Ethiopia Linux Computer ሊኑክስ ኮምፕዩተር

A guide to radio communications technologies for Linux computing for Urdu

https://mov.im/?blog/preptorrent%40movim.eu/ab82747e-5183-4fe3-a260-115144cb4846

A guide to radio communications technologies for Linux computing for Urdu

Slackware Donation methods like paypal

https://www.linuxquestions.org/questions/slackware-14/donating-to-slackware-4175634729/#post5882751

Slackware Donation methods like Paypal

WikiBooks First Aid web pages including PDF download version

https://en.m.wikibooks.org/wiki/First_Aid

WikiBooks First Aid web pages including PDF download version

British Heart Foundation PDF Guide to Defibrillators

https://www.resus.org.uk/sites/default/files/2020-03/AED_Guide_2019-12-04.pdf

Here is an example of Russian prepping documentation, and although it has a radioactive theme, the inclusion of it here has nothing to do with nuclear weapons threats but rather just the fact that Russian people care about people too and as such the information can be useful for various scenarios (e.g. nuclear power-station precautions) and it is a lead for other documens on Russian prepping like First Aid or Health and so scanning the letters in tesseract OCR Optical Character Recognition software with translation software added under BSD license can allow for Mozilla Firefox web-browser to translate into english via Firefox Translations and so you can then search for more Russian language prepping material documents. Likewise, Lithuanian and Estonian prepping documents can be translated and searched in a simiar way and all are included here so as to kindly help all people affected in the Russia or Ukraine or countries nearby to Europe. Or people might want to read it all simply to see the compared style of diagrams or images purely for creative reasons. Graphic images of healthcare surgery or wound treatment (a topic which is for caring for poorly people) is a precaution advised for some viewing. The inclusion of it is not about war (although of course the authors show it has been useful in such circumstances) but instead simply for learning human anatomy in medicine and knowledge and helping people.

Russian prepping documentation scanned

https://atomic-annhilation.blogspot.com/2013/08/1986-soviet-civil-defense.html

Prepping documentation such as for Lithuania or Estonia or Russia or Ukraine 2016-aktyviu-veiksmu-gaires.pdf

https://kam.lt/wp-content/uploads/2022/03/2016-aktyviu-veiksmu-gaires.pdf

Alternative Link to Prepping documentation such as for Lithuania or Estonia or Russia or Ukraine 2016-aktyviu-veiksmu-gaires.pdf

https://upload.movim.eu/files/72545d95562ce01c8d7e25cb0e413a679c22c9f2/VuNEVLDrkCAmH9ibjqpJKeZxCwj9bVu931c4IEoX/2016_aktyviu_veiksmu_gaires_internetui_.pdf

English Language Lithuanian Prepping documenation

https://upload.movim.eu/files/72545d95562ce01c8d7e25cb0e413a679c22c9f2/lEXjkncNURxLPWEaBsIx9qPVlXq3RJAKSviYVxUp/ka_turime_zinoti_praktiniai_patarimai_en-el.pdf

Lithuanian Armoured Vehicle documentation which is also useful for places like Africa

https://www.sauliusajunga.lt/wp-content/uploads/2018/04/Sarvuotos_technikos_apibudinimas_Mokomoji_kng.pdf

Graphic Content Warning Adult supervision for The Management of Combat Wounds The British Military Experience

https://www.researchgate.net/publication/282485099_The_Management_of_Combat_Wounds_The_British_Military_Experience

Graphic Content Warning Adult supervision for TheManagementofCombatWoundsthebritishexperience.pdf

https://www.researchgate.net/profile/Steven-Jeffery/publication/282485099_The_Management_of_Combat_Wounds_The_British_Military_Experience/links/580e183208ae0360753dca6e/The-Management-of-Combat-Wounds-The-British-Military-Experience.pdf

USA Navy prepping PDF example for West Nile Virus

https://www.ready.navy.mil/Portals/86/Documents/beinformed/diseases/DISEASES_WestNile_031015JJK.pdf

USA Navy Prepping documenation portals found via here

https://www.ready.navy.mil/

SimpleSAMLPHP documentation as per OIDC

https://simplesamlphp.org/docs/1.19/simplesamlphp-install.html

YouTube OAuth Example Of Using OIDC and this can be used with SimpleSAMLPHP

https://www.youtube.com/watch?v=tBpVLq_okic

I have included a link for NUAR UK Emergency Alerts (Cabinet Office) but really I would say it could do with a low cost tweak to also allow UK government square-wave data (sounding somewhat like a dial-up modem handshake sound to the human ear) to be received via RADIO (such as a FM car radio) onto a RaspberryPi (which would be expected to have installed a LoRaWAN dongle for public responding) for software updates such as torrent magnet links and bearing co-ordinates and cronjob date-time-stamps and longditude-latitude-altitude and MIDI (which cane be used for music or PC joystick or vehicle control like a drone since it even was used in guided missles in the past) to be processed in linux accepted via microphone-in 3.5mm jack plug (via USB soundcard on the RaspberryPi3b or PiZero2W) or via rtl2832u or buetooth (such as sent from a car-stereo bluetooth link, with or without a bluetooth headset earphone-and-microphone). Having a UK Gov AppImage (both RaspberryPi and amd64 compatible) on the receiving Linux device (PC or Pi3b) linked via the aforementioned technologies (3.5mm jack or bluetooth or rtl2832u) to the car-radio would allow for password authentication of installing a shellscript for every NUAR squarewave update which need only be up to 100KB but probably only needs about 3KB data sent like a DamnSmallLinux torrent file needs, notifying the need to connect to internet to download 50MB of data later that day as a bootable ISO that can be written to CDROM. About a minute of analogue FM Radio squarewave sound should transmit (from UK gov to a public radio connected to a RaspberryPi zero2w or Pi3b at home or in a car) 150KB to 300KB of data and so the signal being sent twice for parity of that 1 minute data signal should last 2 minutes and you probably do not even need that much data. The magnet-links sent could also include private firm sponsorship adverts in some scenarios.

NUAR UK Emergency Alterts which includes software

https://www.gov.uk/guidance/national-underground-asset-register-nuar

UK Cabinet Office NUAR and UK emergency Alerts video YT channel

https://www.youtube.com/watch?v=E_Dgo8SxUnI

See all YT UK Emergency Alerts Videos Cabinet Office and NUAR

https://www.youtube.com/@CabinetOfficeUKGov

Firefox Translations

www.snipca.com/44358

Github tesseract-ocr

https://github.com/tesseract-ocr/tesseract

Tesseract OCR a9t9 Frontend for Microsoft Windows 8 and 10

https://onlineocr.a9t9.com/blog/p/free-ocr-software.html

German and English... Deutschland macht gute Technologien wie Knoppix Linux

https://mov.im/?blog/preptorrent%40movim.eu/deutschland-macht-gute-technologien-wie-knoppix-linux-WwX2DA

German and English... Deutschland macht gute Technologien wie Knoppix Linux

My comment has no hate in it and I do no harm. I am not appalled or afraid, boasting or envying or complaining... Just saying. Psalms23: Giving thanks and praise to the Lord and peace and love. Also, I'd say Matthew6.

#linux #slackware #opencl #skolelinux #debian #kde #foss #paypal #gnu #gnome #christmas #donate #charity #pakistan #afghanistan #urdu #Turkey #EarthQuake #Syria #russia #education #radiation #learning #nuar #prepping #electricity #electricial #electrician #health #safety #healthandsafety #science #mathematics #translation #coding #music

It is the sixth update to SparkyLinux, also known as “Po Tolo,” and it is powered by Debian 11 (Bullseye), which brings with it a stable, secure, and reliable experience.

Read more

#linux #debian #sparkylinux #distros

Protocol Buffers are a popular choice for serializing structured data due to their compact size, fast processing speed, language independence, and compatibility. There exist other alternatives, including Cap’n Proto , CBOR , and Avro .

Usually, data structures are described in a proto definition file ( .proto ). The protoc compiler and a language-specific plugin convert it into code:

$ headflow-4.proto
syntax = "proto3";package decoder;option go_package = "akvorado/inlet/flow/decoder";message FlowMessagev4 {  uint64 TimeReceived = 2;  uint32 SequenceNum = 3;  uint64 SamplingRate = 4;  uint32 FlowDirection = 5;$ protoc-I=.--plugin=protoc-gen-go--go_out=module=akvorado:.flow-4.proto
$ headinlet/flow/decoder/flow-4.pb.go
// Code generated by protoc-gen-go. DO NOT EDIT.// versions://      protoc-gen-go v1.28.0//      protoc        v3.21.12// source: inlet/flow/data/schemas/flow-4.protopackage decoderimport (        protoreflect "google.golang.org/protobuf/reflect/protoreflect"

Akvorado collects network flows using IPFIX or sFlow , decodes them with GoFlow2 , encodes them to Protocol Buffers , and sends them to Kafka to be stored in a ClickHouse database. Collecting a new field, such as source and destination MAC addresses, requires modifications in multiple places, including the proto definition file and the ClickHouse migration code. Moreover, the cost is paid by all users. ¹ It would be nice to have an application-wide schema and let users enable or disable the fields they need.

While the main goal is flexibility, we do not want to sacrifice performance. On this front, this is quite a success: when upgrading from 1.6.4 to 1.7.1, the decoding and encoding performance almost doubled! 🤗

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ initial.txt  │              final.txt              │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      12.963µ ± 2%   7.836µ ± 1%  -39.55% (p=0.000 n=10)
Sflow/with_encoding-12         19.37µ ± 1%   10.15µ ± 2%  -47.63% (p=0.000 n=10)

Faster Protocol Buffers encoding ​

I use the following code to benchmark both the decoding and encoding process. Initially, the Decode() method is a thin layer above GoFlow2 producer and stores the decoded data into the in-memory structure generated by protoc . Later, some of the data will be encoded directly during flow decoding. This is why we measure both the decoding and the encoding. ²

funcBenchmarkDecodeEncodeSflow(b*testing.B){r:=reporter.NewMock(b)sdecoder:=sflow.New(r)data:=helpers.ReadPcapPayload(b,filepath.Join("decoder","sflow","testdata","data-1140.pcap"))for_,withEncoding:=range[]bool{true,false}{title:=map[bool]string{true:"with encoding",false:"without encoding",}[withEncoding]vargot[]*decoder.FlowMessageb.Run(title,func(b*testing.B){fori:=0;i<b.N;i++{got=sdecoder.Decode(decoder.RawFlow{Payload:data,Source:net.ParseIP("127.0.0.1"),})ifwithEncoding{for_,flow:=rangegot{buf:=[]byte{}buf=protowire.AppendVarint(buf,uint64(proto.Size(flow)))proto.MarshalOptions{}.MarshalAppend(buf,flow)}}}})}}

The canonical Go implementation for Protocol Buffers , google.golang.org/protobuf is not the most efficient one. For a long time, people were relying on gogoprotobuf . However, the project is now deprecated . A good replacement is vtprotobuf . ³

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ initial.txt │             bench-2.txt             │
                            │   sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      12.96µ ± 2%   10.28µ ± 2%  -20.67% (p=0.000 n=10)
Netflow/without_encoding-12   8.935µ ± 2%   8.975µ ± 2%        ~ (p=0.143 n=10)
Sflow/with_encoding-12        19.37µ ± 1%   16.67µ ± 2%  -13.93% (p=0.000 n=10)
Sflow/without_encoding-12     14.62µ ± 3%   14.87µ ± 1%   +1.66% (p=0.007 n=10)

Dynamic Protocol Buffers encoding ​

We have our baseline. Let’s see how to encode our Protocol Buffers without a .proto file. The wire format is simple and rely a lot on variable-width integers.

Variable-width integers, or varints , are an efficient way of encoding unsigned integers using a variable number of bytes, from one to ten, with small values using fewer bytes. They work by splitting integers into 7-bit payloads and using the 8 ^th bit as a continuation indicator, set to 1 for all payloads except the last.

For our usage, we only need two types: variable-width integers and byte sequences. A byte sequence is encoded by prefixing it by its length as a varint. When a message is encoded, each key-value pair is turned into a record consisting of a field number, a wire type, and a payload. The field number and the wire type are encoded as a single variable-width integer called a tag.

We use the following low-level functions to build the output buffer:

• protowire.AppendTag() encodes a tag,
• protowire.AppendVarint() encodes a variable-width integer, and
• protowire.AppendBytes() append bytes as is.

Our schema abstraction contains the appropriate information to encode a message ( ProtobufIndex ) and to generate a proto definition file (fields starting with Protobuf ):

typeColumnstruct{KeyColumnKeyNamestringDisabledbool// […]// For protobuf.ProtobufIndexprotowire.NumberProtobufTypeprotoreflect.Kind// Uint64Kind, Uint32Kind, …ProtobufEnummap[int]stringProtobufEnumNamestringProtobufRepeatedbool}

We have a few helper methods around the protowire functions to directly encode the fields while decoding the flows. They skip disabled fields or non-repeated fields already encoded. Here is an excerpt of the sFlow decoder :

sch.ProtobufAppendVarint(bf,schema.ColumnBytes,uint64(recordData.Base.Length))sch.ProtobufAppendVarint(bf,schema.ColumnProto,uint64(recordData.Base.Protocol))sch.ProtobufAppendVarint(bf,schema.ColumnSrcPort,uint64(recordData.Base.SrcPort))sch.ProtobufAppendVarint(bf,schema.ColumnDstPort,uint64(recordData.Base.DstPort))sch.ProtobufAppendVarint(bf,schema.ColumnEType,helpers.ETypeIPv4)

For fields that are required later in the pipeline, like source and destination addresses, they are stored unencoded in a separate structure:

typeFlowMessagestruct{TimeReceiveduint64SamplingRateuint32// For exporter classifierExporterAddressnetip.Addr// For interface classifierInIfuint32OutIfuint32// For geolocation or BMPSrcAddrnetip.AddrDstAddrnetip.AddrNextHopnetip.Addr// Core component may override themSrcASuint32DstASuint32GotASPathbool// protobuf is the protobuf representation for the information not contained above.protobuf[]byteprotobufSetbitset.BitSet}

The protobuf slice holds encoded data. It is initialized with a capacity of 500 bytes to avoid resizing during encoding. There is also some reserved room at the beginning to be able to encode the total size as a variable-width integer. Upon finalizing encoding, the remaining fields are added and the message length is prefixed:

func(schema*Schema)ProtobufMarshal(bf*FlowMessage)[]byte{schema.ProtobufAppendVarint(bf,ColumnTimeReceived,bf.TimeReceived)schema.ProtobufAppendVarint(bf,ColumnSamplingRate,uint64(bf.SamplingRate))schema.ProtobufAppendIP(bf,ColumnExporterAddress,bf.ExporterAddress)schema.ProtobufAppendVarint(bf,ColumnSrcAS,uint64(bf.SrcAS))schema.ProtobufAppendVarint(bf,ColumnDstAS,uint64(bf.DstAS))schema.ProtobufAppendIP(bf,ColumnSrcAddr,bf.SrcAddr)schema.ProtobufAppendIP(bf,ColumnDstAddr,bf.DstAddr)// Add length and move it as a prefixend:=len(bf.protobuf)payloadLen:=end-maxSizeVarintbf.protobuf=protowire.AppendVarint(bf.protobuf,uint64(payloadLen))sizeLen:=len(bf.protobuf)-endresult:=bf.protobuf[maxSizeVarint-sizeLen:end]copy(result,bf.protobuf[end:end+sizeLen])returnresult}

Minimizing allocations is critical for maintaining encoding performance. The benchmark tests should be run with the -benchmem flag to monitor allocation numbers. Each allocation incurs an additional cost to the garbage collector. The Go profiler is a valuable tool for identifying areas of code that can be optimized:

$ gotest-run=__nothing__-bench=Netflow/with_encoding\> -benchmem-cpuprofileprofile.out\> akvorado/inlet/flow
goos: linuxgoarch: amd64pkg: akvorado/inlet/flowcpu: AMD Ryzen 5 5600X 6-Core ProcessorNetflow/with_encoding-12             143953              7955 ns/op            8256 B/op        134 allocs/opPASSok      akvorado/inlet/flow     1.418s$ gotoolpprofprofile.out
File: flow.testType: cpuTime: Feb 4, 2023 at 8:12pm (CET)Duration: 1.41s, Total samples = 2.08s (147.96%)Entering interactive mode (type "help" for commands, "o" for options)(pprof)web

After using the internal schema instead of code generated from the proto definition file, the performance improved. However, this comparison is not entirely fair as less information is being decoded and previously GoFlow2 was decoding to its own structure, which was then copied to our own version.

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ bench-2.txt  │             bench-3.txt             │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12      10.284µ ± 2%   7.758µ ± 3%  -24.56% (p=0.000 n=10)
Netflow/without_encoding-12    8.975µ ± 2%   7.304µ ± 2%  -18.61% (p=0.000 n=10)
Sflow/with_encoding-12         16.67µ ± 2%   14.26µ ± 1%  -14.50% (p=0.000 n=10)
Sflow/without_encoding-12      14.87µ ± 1%   13.56µ ± 2%   -8.80% (p=0.000 n=10)

As for testing, we use github.com/jhump/protoreflect : the protoparse package parses the proto definition file we generate and the dynamic package decodes the messages. Check the ProtobufDecode() method for more details. ⁴

To get the final figures, I have also optimized the decoding in GoFlow2 . It was relying heavily on binary.Read() . This function may use reflection in certain cases and each call allocates a byte array to read data. Replacing it with a more efficient version provides the following improvement:

goos: linux
goarch: amd64
pkg: akvorado/inlet/flow
cpu: AMD Ryzen 5 5600X 6-Core Processor
                            │ bench-3.txt  │             bench-4.txt             │
                            │    sec/op    │   sec/op     vs base                │
Netflow/with_encoding-12       7.758µ ± 3%   7.365µ ± 2%   -5.07% (p=0.000 n=10)
Netflow/without_encoding-12    7.304µ ± 2%   6.931µ ± 3%   -5.11% (p=0.000 n=10)
Sflow/with_encoding-12        14.256µ ± 1%   9.834µ ± 2%  -31.02% (p=0.000 n=10)
Sflow/without_encoding-12     13.559µ ± 2%   9.353µ ± 2%  -31.02% (p=0.000 n=10)

It is now easier to collect new data and the inlet component is faster! 🚅

Značky: #Debian

OpenSSH 9.2 and 9.2p1: security fixes and other improvements were released this week, giving system administrators and users the latest in secure remote access protocols. Read more

#linux #debian #ubuntu #bsd #openssh #ssh #dev #sysadmin #devops #developers

Welcome to the first report for 2023 from the Reproducible Builds project!

In these reports we try and outline the most important things that we have been up to over the past month, as well as the most important things in/around the community. As a quick recap, the motivation behind the reproducible builds effort is to ensure no malicious flaws can be deliberately introduced during compilation and distribution of the software that we run on our devices. As ever, if you are interested in contributing to the project, please visit our Contribute page on our website.

News

In a curious turn of events, GitHub first announced this month that the checksums of various Git archives may be subject to change, specifically that because:

… the default compression for Git archives has recently changed. As result, archives downloaded from GitHub may have different checksums even though the contents are completely unchanged.

This change (which was brought up on our mailing list last October) would have had quite wide-ranging implications for anyone wishing to validate and verify downloaded archives using cryptographic signatures. However, GitHub reversed this decision, updating their original announcement with a message that “We are reverting this change for now. More details to follow.” It appears that this was informed in part by an in-depth discussion in the GitHub Community issue tracker .

The Bundesamt für Sicherheit in der Informationstechnik (BSI) (trans: ‘The Federal Office for Information Security’) is the agency in charge of managing computer and communication security for the German federal government. They recently produced a report that touches on attacks on software supply-chains ( Supply-Chain-Angriff ). ( German PDF )

Contributor Seb35 updated our website to fix broken links to Tails ’ Git repository [ … ][ … ], and Holger updated a large number of pages around our recent summit in Venice [ … ][ … ][ … ][ … ].

Noak Jönsson has written an interesting paper entitled The State of Software Diversity in the Software Supply Chain of Ethereum Clients . As the paper outlines:

In this report, the software supply chains of the most popular Ethereum clients are cataloged and analyzed. The dependency graphs of Ethereum clients developed in Go, Rust, and Java, are studied. These client are Geth, Prysm, OpenEthereum, Lighthouse, Besu, and Teku. To do so, their dependency graphs are transformed into a unified format. Quantitative metrics are used to depict the software supply chain of the blockchain. The results show a clear difference in the size of the software supply chain required for the execution layer and consensus layer of Ethereum.

Yongkui Han posted to our mailing list discussing making reproducible builds & GitBOM work together without gitBOM-ID embedding . GitBOM (now renamed to OmniBOR ) is a project to “enable automatic, verifiable artifact resolution across today’s diverse software supply-chains” [ … ]. In addition, Fabian Keil wrote to us asking whether anyone in the community would be at Chemnitz Linux Days 2023 , which is due to take place on 11th and 12th March ( event info ).

Separate to this, Akihiro Suda posted to our mailing list just after the end of the month with a status report of bit-for-bit reproducible Docker/OCI images . As Akihiro mentions in their post, they will be giving a talk at FOSDEM in the ‘Containers’ devroom titled Bit-for-bit reproducible builds with Dockerfile and that “my talk will also mention how to pin the apt/dnf/apk/pacman packages with my repro-get tool.”

The extremely popular Signal messenger app added upstream support for the SOURCE_DATE_EPOCH environment variable this month. This means that release tarballs of the Signal desktop client do not embed nondeterministic release information. [ … ][ … ]

Distribution work

F-Droid & Android

There was a very large number of changes in the F-Droid and wider Android ecosystem this month:

On January 15th, a blog post entitled Towards a reproducible F-Droid was published on the F-Droid website , outlining the reasons why “F-Droid signs published APKs with its own keys” and how reproducible builds allow using upstream developers’ keys instead. In particular:

In response to […] criticisms, we started encouraging new apps to enable reproducible builds . It turns out that reproducible builds are not so difficult to achieve for many apps. In the past few months we’ve gotten many more reproducible apps in F-Droid than before . Currently we can’t highlight which apps are reproducible in the client, so maybe you haven’t noticed that there are many new apps signed with upstream developers’ keys.

(There was a discussion about this post on Hacker News .)

In addition:

• F-Droid added 13 apps published with reproducible builds this month. [ … ]

• FC Stegerman outlined a bug where baseline.profm files are nondeterministic , developed a workaround, and provided all the details required for a fix. As they note, this issue has now been fixed but the fix is not yet part of an official Android Gradle plugin release .

• GitLab user Parwor discovered that the number of CPU cores can affect the reproducibility of .dex files. [ … ]

• FC Stegerman also announced the 0.2.0 and 0.2.1 releases of reproducible-apk-tools , a suite of tools to help make .apk files reproducible. Several new subcommands and scripts were added, and a number of bugs were fixed as well [ … ][ … ]. They also updated the F-Droid website to improve the reproducibility-related documentation. [ … ][ … ]

• On the F-Droid issue tracker, FC Stegerman discussed reproducible builds with one of the developers of the Threema messenger app and reported that Android SDK build-tools 31.0.0 and 32.0.0 (unlike earlier and later versions) have a zipalign command that produces incorrect padding .

• A number of bugs related to reproducibility were discovered in Android itself. Firstly, the non-deterministic order of .zip entries in .apk files [ … ] and then newline differences between building on Windows versus Linux that can make builds not reproducible as well. [ … ] (Note that these links may require a Google account to view.)

• And just before the end of the month, FC Stegerman started a thread on our mailing list on the topic of hiding data/code in APK embedded signatures which has been made possible by the Android APK Signature Scheme v2/v3 . As part of this, they made an Android app that reads the APK Signing block of its own APK and extracts a payload in order to alter its behaviour called sigblock-code-poc .

Debian

As mentioned in last month’s report , Vagrant Cascadian has been organising a series of online sprints in order to ‘clear the huge backlog of reproducible builds patches submitted’ by performing NMUs ( Non-Maintainer Uploads ). During January, a sprint took place on the 10th, resulting in the following uploads:

• Chris Lamb:

  • critcl ( #963600 )
  • log4cpp ( #1020662 )
  • logapp ( #1010845 )
  • nanomsg ( #1001853 )

• Holger Levsen:

  • netkit-rsh ( #1020798 )
  • wcwidth ( #1005408 )

• Vagrant Cascadian:

  • mc ( #828683 )
  • gtk-sharp3 ( #989965 & #989966 )

During this sprint, Holger Levsen filed Debian bug #1028615 to request that the tracker.debian.org service display results of reproducible rebuilds, not just reproducible ‘CI’ results.

Elsewhere in Debian, strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month, version 1.13.1-1 was uploaded to Debian unstable by Holger Levsen, including a fix by FC Stegerman ( obfusk ) to update a regular expression for the latest version of file(1) [ … ]. ( #1028892 )

Lastly, 65 reviews of Debian packages were added, 21 were updated and 35 were removed this month adding to our knowledge about identified issues .

Other distributions

In other distributions:

• Bernhard M. Wiedemann published another monthly report for reproducibility within openSUSE , as well as a belated report for December 2022 .

• It was announced that Fedora Rawhide now ‘clamps’ file modification types to SOURCE_DATE_EPOCH . [ … ]

• Finally, an existing tool called rpmreproduce was (re-)discovered this month, which claims that “given a buildinfo file from a RPM package, [it can] generate instructions for attempting to reproduce the binary packages built from the associated source and build information.”

diffoscope

diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb made the following changes to diffoscope , including preparing and uploading versions 231 , 232 , 233 and 234 to Debian:

• No need for from __future__ import print_function import anymore. [ … ]
• Comment and tidy the extras_require.json handling. [ … ]
• Split inline Python code to generate test Recommends into a separate Python script. [ … ]
• Update debian/tests/control after merging support for PyPDF support. [ … ]
• Correctly catch segfaulting cd-iccdump binary. [ … ]
• Drop some old debugging code. [ … ]
• Allow ICC tests to (temporarily) fail. [ … ]

In addition, FC Stegerman ( obfusk ) made a number of changes, including:

• Updating the test_text_proper_indentation test to support the latest version(s) of file(1) . [ … ]
• Use an extras_require.json file to store some build/release metadata, instead of accessing the internet. [ … ]
• Updating an APK-related file(1) regular expression. [ … ]
• On the diffoscope.org website , de-duplicate contributors by e-mail. [ … ]

Lastly, Sam James added support for PyPDF version 3 [ … ] and Vagrant Cascadian updated a handful of tool references for GNU Guix . [ … ][ … ]

Upstream patches

The Reproducible Builds project attempts to fix as many currently-unreproducible packages as possible. This month, we wrote a large number of such patches, including:

• Bernhard M. Wiedemann:

  • argparse (date-related issue)
  • asciimatics (build failure)
  • asyncpg (fails to build in 2032)
  • cpython (fails to build in 2038)
  • django (fails to build in 2038)
  • grandorgue ( .zip -related issue)
  • libarchive (fails to build in 2038)
  • libarchive (fails to build in 2038)
  • librcc (date)
  • mbedtls (fails to build in 2023)
  • mozilla-nss (fails to build in 2023)
  • ocaml-rpm-macros (fix fallout from an RPM-related change)
  • perl HTTP::Cookies (fails to build in 2038)
  • python-aiosmtplib/python-trustme (fails to build in 2038 due to SSL certificate)
  • python-bmap (fails to build in 2024)
  • python-compileall2 (fails to build in 2038)
  • taskwarrior ( python-tasklib fails to build in 2038)
  • taskwarrior (fix fails to build in 2038)
  • wrk (hash ordering issue)
  • xemacs (fails to build in 2038 stuck)

• Chris Lamb:

  • #1027988 filed against click .
  • #1027992 filed against towncrier .
  • #1028051 filed against unifrac-tools .
  • #1028310 filed against hamster-time-tracker .
  • #1028515 filed against accel-config .
  • #1029295 filed against python-miio .
  • #1029297 filed against python-graphviz .

• Vagrant Cascadian:

  • #1029227 filed against ectrans .
  • #1029303 filed against fiat-ecmwf .
  • #1029307 filed against node-katex .
  • #1029800 filed against aribas .
  • #1029801 filed against tbox .
  • #1029807 filed against borgbackup2 .
  • #1029809 filed against dnf-plugins-core .
  • #1030057 filed against refpolicy .

• FC Stegerman:

  • Several patches for file(1) (which is used by reproducible builds tools like diffoscope and strip-nondeterminism ) that improve detection of various file formats are now included in the Debian packaging. [ … ]

Testing framework

The Reproducible Builds project operates a comprehensive testing framework at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In October, the following changes were made by Holger Levsen:

• Node changes:

  • Add three new nodes hosted at the Oregon State University Open Source Lab including integrating them into the DNS, maintenance and monitoring systems. [ … ][ … ][ … ][ … ][ … ]

• Debian-related changes:

  • Only keep diffoscope ’s HTML output (ie. no .json or .txt ) for LTS suites and older in order to save diskspace on the Jenkins host. [ … ]
  • Re-create pbuilder base less frequently for the stretch , bookworm and experimental suites. [ … ]

• OpenWrt-related changes:

  • Add gcc-multilib to OPENWRT_HOST_PACKAGES and install it on the nodes that need it. [ … ]
  • Detect more problems in the health check when failing to build OpenWrt. [ … ]

• Misc changes:

  • Update the chroot-run script to correctly manage /dev and /dev/pts . [ … ][ … ][ … ]
  • Update the Jenkins ‘shell monitor’ script to collect disk stats less frequently [ … ] and to include various directory stats. [ … ][ … ]
  • Update the ‘real’ year in the configuration in order to be able to detect whether a node is running in the future or not. [ … ]
  • Bump copyright years in the default page footer. [ … ]

In addition, Christian Marangi submitted a patch to build OpenWrt packages with the V=s flag to enable debugging. [ … ]

If you are interested in contributing to the Reproducible Builds project, please visit the Contribute page on our website. You can get in touch with us via:

• IRC: #reproducible-builds on irc.oftc.net .

• Twitter: @ReproBuilds

• Mastodon: @reproducible_builds@fosstodon.org

• Mailing list: rb-general@lists.reproducible-builds.org

Značky: #Debian

This post is very late, but better late than never! I want to take a look back at the work that was done on FreedomBox during 2022.

Several apps were added to FreedomBox in 2022. The email server app (that was developed by a Google Summer of Code student back in 2021) was finally made available to the general audience of FreedomBox users. You will find it under the name “Postfix/Dovecot”, which are the main services configured by this app.

Another app that was added is Janus, which has the description “video room”. It is called “video room” instead of “video conference” because the room itself is persistent. People can join the room or leave, but there isn’t a concept of “calling” or “ending the call”. Actually, Janus is a lightweight WebRTC server that can be used as a backend for many different types of applications. But as implemented currently, there is just the simple video room app. In the future, more advanced apps such as Jangouts may be packaged in Debian and made available to FreedomBox.

RSS-Bridge is an app that generates RSS feeds for websites that don’t provide their own (for example, YouTube). It can be used together with any RSS news feed reader application, such as TT-RSS which is also available in FreedomBox.

There is now a “Privacy” page in the System menu, which allows enabling or disabling the Debian “popularity-contest” tool. If enabled, it reports the Debian packages that are installed on the system. The results can be seen at https://popcon.debian.org , which currently shows over 400 FreedomBoxes are reporting data.

A major feature added to FreedomBox in 2022 is the ability to uninstall apps. This feature is still considered experimental (it won’t work for every app), but many issues have been fixed already. There is an option to take a backup of the app’s data before uninstalling. There is also now an “operations queue” in case the user starts multiple install or uninstall operations concurrently.

XEP-0363 (HTTP File Upload) has been enabled for Ejabberd Chat Server. This allows files to be transferred between XMPP clients that support this feature.

There were a number of security improvements to FreedomBox, such as adding fail2ban jails for Dovecot, Matrix Synapse, and WordPress. Firewall rules were added to ensure that authentication and authorization for services proxied through Apache web server cannot be bypassed by programs running locally on the system. Also, we are no longer using libpam-tmpdir to provide temporary folder isolation, because it causes issues for several packages. Instead we use systemd’s sandboxing features, which provide even better isolation for services.

Some things were removed in 2022. The ez-ipupdate package is no longer used for Dynamic DNS, since it is replaced by a Python implementation of GnuDIP. An option to restrict who can log in to the system was removed, due to various issues that arose from it. Instead there is an option to restrict who can login through SSH. The DNSSEC diagnostic test was removed, because it caused confusion for many users (although use of DNSSEC is still recommended).

Finally, some statistics. There were 31 releases in 2022 (including
point releases). There were 68 unique contributors to the git
repository; this includes code contributions and translations (but not
contributions to the manual pages). In total, there were 980 commits to the git repository.

Značky: #Debian

Prologue

For over a year now Fre(i)e Software GmbH (my company) is involved in Ubuntu Touch 20.04 developemt. The development effort currently is handle by a mix of paid and voluntary developers/contributors under the umbrella of the UBports Foundation. We are approachint the official first release of Ubuntu Touch 20.04 with rapid pace.

And, if you are a non-Englisch native speaker, we'd like to ask you for help...

light+love
Mike (aka sunweaver at debian.org, Mastodon, IRC, etc.)

Internationalization (i18n) of Ubuntu Touch 20.04

The UBports team has moved most of the translation workflows for localizing Ubuntu Touch over to Hosted Weblate:

• https://hosted.weblate.org/projects/lomiri/
• https://hosted.weblate.org/projects/ayatana-indicators/
• https://hosted.weblate.org/projects/ubports/
• https://hosted.weblate.org/projects/open-store-io/

To contribute to the UBports projects you need to register here:

• https://hosted.weblate.org/accounts/register/

The localization platform of all UBports / Lomiri components is sponsored by Hosted Weblate via their free hosting plan for Libre and Open Source Projects. Many thanks for providing this service.

Translating Lomiri

The translation components in the Lomiri project have already been set up and are ready for being updated by translators. Please expect some translation template changes for all those components to occur in the near future, but this should not hinder you from starting translation work right away.

Translating Lomiri will bring the best i18n experience to Ubuntu Touch 20.04 end users for the core libraries and the pre-installed (so called) Core Apps.

Translating Ubuntu Touch Apps

For App Developers (apps that are not among the Core Apps) we will now offer a translation slot under the UBports project on hosted.weblate.org:

• https://hosted.weblate.org/projects/ubports/

If you are actively maintaining an Ubuntu Touch app, please ask for a translation component on hosted.weblate.org and we will set up your app's translation workflow for and with you. Using the translation service at Hosted Weblate is not a must for app developers, it's rather a service we offer to ease i18n work on Ubuntu Touch apps.

To get translations for your app set up on Hosted Weblate, please get in touch with us on https://t.me/ubports , please highlight @sunweaver (Mike Gabriel), @BetaBreak (Raoul Kramer), @cibersheep and @Danfro with your request.

Značky: #Debian

In 2022 I read 34 books (-19% on last year).

In 2021 roughly a quarter of the books I read were written by women. I was determined to push that ratio in 2022, so I made an effort to try and only read books by women. I knew that I wouldn't manage that, but by trying to, I did get the ratio up to 58% (by page count).

I'm not sure what will happen in 2023. My to-read pile has some back-pressure from books by male authors I postponed reading in 2022 (in particular new works by Christopher Priest and Adam Roberts). It's possible the ratio will swing back the other way, which would mean it would not be worth repeating the experiment. At least if the ratio is the point of the exercise. But perhaps it isn't: perhaps the useful outcome is more qualitative than quantitative.

I tried to read some new (to me) authors. I really enjoyed Shirley Jackson ( The Haunting of Hill House , We Have Always Lived In The Castle ). I Struggled with Angela Carter's Heroes and Villains although I plan to return to her other work, in particular, The Bloody Chamber . I also got through Donna Tartt's The Secret History on the recommendation of a friend. I had to push through the first 15% or so but it turned out to be worth it.

I finally read (and loved) The Handmaid's Tale , which I had never read despite loving Atwood.

My top non-fiction book was The Nanny State Made Me by Stuart Maconie. I still read far more fiction than non-fiction. Or perhaps I'm not keeping track of non- fiction as well. I feel non-fiction requires a different approach to reading: not necessarily linear; it's not always important to read the whole book; it's often important to re-read sections. It might not make sense to consider them in the same bracket.

My favourite novels this year were Sea of Tranquility by Emily St. John Mandel, a standalone sort-of sequel to The Glass House but in a very different genre; and The This by Adam Roberts, which was equally remarkable. The This has an interesting narrative device in the first third where a stream of tweets is presented in parallel with the main text. This works well, and does a good job of capturing the figurative river of tweet-like stuff that is woven into our lives at the moment. But I can't help but wonder how they tackle that in the audiobook.

Značky: #Debian

We are getting towards the freeze for Debian/Bookworm so the current state of packages isn’t going to change much before the release. Bugs will get fixed but missing features will mostly be missing until the next release.

Anarcat wrote an excellent blog post about using Wayland with the Sway window manager [1] . It seems pretty good if you like Sway, but I like KDE and plan to continue using it. Several of the important utility programs referenced by Anarcat won’t run with KDE/Wayland and give errors such as “Compositor doesn’t support wlr-output-management-unstable-v1”. One noteworthy thing about Wayland is the the Window manager and the equivalent to the X server are the same program so KDE has different Wayland code than Sway and doesn’t support some features. The lack of these features limits my ability to manage multiple displays and therefore makes KDE/Wayland unsuitable for many laptop uses. My work laptop runs Ubuntu 22.04 with KDE and wouldn’t correctly display on the pair of monitors on a USB-C dock that’s the standard desktop configuration where I work.

In my previous post about Wayland [2] I wrote about converting 2 of my systems to Wayland. Since then I had changed them back to X because of problems with supporting strange monitor configurations on laptops and also due to the KDE window manager crashing occasionally which terminates the session in Wayland but merely requires restarting the window manager in X. More recently I had a problem with the GPU in my main workstation sometimes not being recognised by the system (reporting no PCIe device), when I got a new one I couldn’t get X to work with the error “Cannot run in framebuffer mode. Please specify busIDs for all framebuffer devices” so I tried Wayland again. Now in the later stage of the Bookworm development process it seems that the problem with the KDE window manager crashing has been solved or mitigates and there is a new problem of the plasmashell process crashing. As I can restart plasmashell without logging out that’s much less annoying. So now my main workstation is running on Wayland with a slower GPU than I previously had while also giving a faster user experience so Wayland is providing a definite performance benefit.

Maybe for Trixie (the next release of Debian after Bookworm) we should have a release goal of having full Wayland support in all the major GUI systems.

• [1] https://anarc.at/software/desktop/wayland/
• [2] https://etbe.coker.com.au/2022/03/25/wayland/

Značky: #Debian