Whether it is carpentry, auto mechanics, electrical engineering, or computer science, you always hear the same saying: use the right tool for the right job. The wrong tool can make any solution difficult and could introduce new problems.

I've previously written about some of the big problems with the C2PA solution for recording provenance and authenticity in media. However, I recently came across a new problem based on their decision to use X.509 certificates and how they are used. Specifically: their certificates expire. This has some serious implications for authentication and legal cases that try to use C2PA metadata as evidence.

X.509 Background

Whether it's HTTPS or digital signatures, the terms "certificates", "X.509", and "x509" are often used interchangeably. While there are different types of digital certificates, most use the X.509 format. The name "X.509" refers to a section in the ITU standards. The different parts in the standard are called "Series". There are Series A, D, E, F, ... all the way to Z. (I don't know why they skipped B or C.) Series A covers their organization. Series H specifies audiovisual and multimedia systems. Series X covers data networks and related security. Within Series X, there are currently 1,819 different sections. "X.509" refers to Series X, section 509. The name identifies the section that standardizes the digital certificate format for public and private key management. In general, when someone writes about certificates, certs, X.509, or x509, it's all the same thing.

X.509 technology has been around since at least November 1988. That's when the first edition of the specification was released. It's not a perfect technology (they're on the ninth major revision right now), but it's good enough for many day-to-day uses.

I'm not going to claim that X.509 is simple. Public and private key cryptography is very technical and the X.509 file format is overly complicated. Years ago, I wrote my own X.509 parser, for both binary (DER) and encoded text (PEM) formats, and with support for individual certs and chains of certs. You really can't comprehend the full depth of complexity until you try to implement it your own parser. This is why almost everyone relies on someone else's existing library. The most popular library is OpenSSL .

At an overly-simplified view, each X.509 certificate contains (at minimum) a public key, and/or a private key, and/or a chain of parent keys. (A parent cert can be used to issue a child cert, creating a chain of certificates.) Data encrypted with the public key can only be decoded with the private key, and vice versa. Usually the private key is not distributed (kept private) while the public key is shared (made public).

X.509 Metadata

Buried inside the X.509 format are parameters that identify how the certificate can be used. Some certs are only for web sites (HTTPS), while others may only be for digitally signing documents.

Most certs also include information about the issuer and the issuee (the 'subject'). These are usually encoded text notations, such as:

• CN: Common Name
• OU: Organizational Unit
• O: Organization
• L: Locality (City or Region)
• ST: State or Province Name
• C: Country Name

So you might see a cert containing:

Subject: /CN=cai-prod/O=Adobe Inc./L=San Jose/ST=California/C=US/emailAddress=cai-ops@adobe.com

Issuer: /C=US/O=Adobe Systems Incorporated/OU=Adobe Trust Services/CN=Adobe Product Services G3

Depending on the X.509 parser, you might see these fields separated by spaces or converted to text (e.g., Country: US, Organization: Adobe Inc., etc.)

The reason I'm diving into X.509 is that C2PA uses these certs to digitally sign the information. As I previously demonstrated with my forgeries , you can't assume that the Subject or Issuer information represents the actual certificate holder. We trust that every issuing certificate provider, starting with the top-level CA provider, is trusted and reputable, filling in the information using validated and authenticated credentials. However, that's not always the case.

• 2011: Fraudulent certs were issued by real CA provider for Google, Microsoft, Yahoo, and Skype
  
• 2012: The trusted and reputable CA provider "TURKTRUST" issued unauthorized certs for Google, Microsoft, and others. (Even though the problem was identified, it was nearly a year later before TURKTRUST was revoked as an untrusted and disreputable CA provider.)
  
• 2013: Google and Microsoft again discovered that fraudulent certs were issued by a real CA provider for some of their domains.
  
• 2015: A trusted and reputable CA provider in China was caught issuing unauthorized certificates for Google. As crypto-expert Bruce Schneier wrote , "Yet another example of why the CA model is so broken."

This is far from every instance. It doesn't happen often, but news reports every 1-2 years is enough to recognize that threat is very realistic. Moreover, if the company (like Leica) isn't as big as Google or Microsoft, then they might not notice the forged certificate.

X.509 Dates

Inside each cert should also be a few dates that identify when the certificate is valid:

• Not Before : A cert may be created at any time, but is not valid until after a specific date. Often (but not always), this is the date when the certificate was generated.
  
• Not After : Certs typically have an expiration date. After that date, the cert is no longer valid. With LetsEncrypt, certificates are only good for 3 months at a time. (LetsEncrypt provides a helper script that automatically renews the cert so administrators don't forget.) Other certificate authorities (CA servers) may issue multi-year certificates. The trusted CA providers (top of the cert chain) often have certs that are issued for a decade or longer.

With C2PA, every cert that I've seen (both real and forged) include these dates. Similarly, unless it is a self-signed certificate, every web server has these dates. In your desktop browser, you can visit any web site and click on the little lock in the address bar to view the site's certificate information. It will list the issuer, subject, and valid date ranges.

Expired Certs and Signing

On rare occasions, you might encounter web sites where the certificate has expired. The web browser detects the bad server certificate and displays an error message:


For a web site:

• If you don't know what's going on, then the error message stops you from visiting a site that uses an invalid/expired certificate.
  
• The temporary workaround is to ignore the expired date, use the certificate anyway, and visit the web site. (NEVER do this if the web site requires any kind of login.)
  
• The long-term solution requires the web administrator to update the certs for the site.

But what about digital signatures, such as those issued with DocuSign or C2PA? In these cases, the public certificate is included with the signature. Since the public cert is widely distributed, they can't just reach out across the internet and update all of them.

An invalid certificate must never be used to create a new signature, but it can be used to validate an existing signature. As noted by GlobalTrust (a trusted certificate authority), there are a few situations to consider when evaluating the time range of a cert used for a digital signature:

Case #1: Post-date : This occurs when the current time is before the Not Before date. (You almost never encounter this.) In this case, we can prove that an invalid certificate was used to sign the data. The signature must be interpreted as invalid, even if the certificate can be used to confirm the signature. (An invalid certificate cannot be used to create a valid signature.)

Case #2: Active : If the current date falls between the Not Before and Not After dates, then the certificate's time range is valid. If the signature checks out, then the signature is valid.

Case #3: Expired : This is where things get a little complicated. As noted by GlobalTrust, "Expired certificates can no longer produce valid signatures -- however, signatures set before expiry remain valid later. Software that issues an error here does not work in accordance with the standard."

This means two things:

1. You are not supposed to use an expired certificate to sign something after it has expired. An expired certificate is invalid for signing, and any new signatures must be treated as invalid.
   
2. If you have an expired certificate that can validate a signature, then the signature is valid.

If (like most of the world) you use OpenSSL to generate a digital signature, then OpenSSL checks the dates. The library refuses to sign anything using an expired certificate. And since Adobe's c2patool uses OpenSSL, it also can't sign the C2PA metadata using an expired certificate.

Except... There's a nifty Linux command-line tool called "faketime". ( sudo apt install faketime ) This program hijacks any system calls to the time() or clock() functions and returns a fake time to the calling application. To backdate the clock by 2 years for a single application, you might use:

faketime -f -2y date # display the date, but it's backdated 2 years
faketime -f -2y openssl ... # force openssl to sign something as if it were 2 years ago
faketime -f -2y c2patool ... # force c2patool to sign using an expired certificate

This way, you can generate a signature that appears valid using an expired certificate. After the expiration date, the backdated signature will appear valid because it can be verified by an expired certificate. (Doh!)

(Using faketime is much easier than taking your computer offline, rolling back the system clock, and then signing the data.)

This means that a malicious user with an expired certificate can always backdate a signature. So really, we need to change Case #3 to make it more specific:

Case #3: Expired and Previously Verified : If the cert is currently expired and you, or someone you trust, previously verified the signature when it wasn't expired, then the signature is still valid.

Case #4: Expired and Not Previously Verified (by someone you trust) : What if you just received the document and the validating certificate is already expired? In this case, you can't tell if someone backdated the signature using something like 'faketime'.

Even if you trust the signing authority, or even if other certificate issuers in the chain are still valid, that doesn't mean that someone didn't backdate the expired certificate for signing. In this case, you cannot trust the signature and it must be treated as invalid .

Hello, Adobe

The reason I'm bringing up all of this: Adobe's certificate that they used for signing C2PA metadata expired on 2024-02-01 23:59:59 GMT. At FotoForensics, I have over a hundred examples of files with C2PA metadata that are now expired. I also have copies of a few dozen pictures from Microsoft that are about to expire (Not After 2024-03-31 18:15:27 GMT) and a few examples from other vendors that expire later this year.

• In theory, the signer can always authenticate their own signature. Even after it expires, Adobe can authenticate Adobe, Microsoft can authenticate Microsoft, etc. This is because Adobe can claim that only Adobe had access to the cert and nobody at Adobe backdated the signature. (Same for Microsoft and other companies.)
  
• If the signer is trusted, such as DocuSign (used for signing legal documents), then we can trust that it was previously validated. (This is Case #3 and we can trust the signature.)

But what about the rest of us? I'm not Adobe, Microsoft, Leica, etc. Maybe I don't trust all of the employees at these companies. (I shouldn't be require to trust them.) Consider this picture (click to view it at FotoForensics or Adobe's Content Credentials ):

• If you only trust the C2PA signature, then you have an expired certificate that validates the signature. Adobe's Content Credentials doesn't mention that the certificate is expired.
  
• If you look at the timestamps, it claims that it was signed by Adobe on 16-Jan-2024; 16 days before the certificate expired. This means you had about two weeks to validate it before it expired. That's not very long.
  
• If you look at the metadata, you can see that it was heavily altered. Taking the metadata at face value identifies a Nikon D810 camera, Adobe Lightroom 7.1.2 for Windows, and Photoshop 25.3 for Windows. The edits happened over a two-year time span. (That's two year between creation and the digital signing with C2PA's authenticity and provenance information.) You can also identify a lot of different alterations, including adjustments to the texture, shadows, lighting, spot healing, etc. (And all of that is just from the metadata!)

This picture is far from a "camera original".

I know a few companies that have said that they want to just relying on the C2PA validation step to determine if a picture is altered. If the C2PA signature is valid, then they assume the picture is valid. But in this case, the non-C2PA metadata clearly identifies alterations. Moreover, the C2PA signature is only able to be validated using an expired certificate. We trust that it was not backdated, but given all of the other edits, is that trust misplaced?

Keep in mind, this specific example is using Adobe. However, many other companies are considering adopting the C2PA specification. Even if you trust Adobe, do you really trust every other company, organization, and creator out there?

New Certs

According to the sightings at FotoForensics, Adobe updated their certificate days before it expired. The new certificate says Not Before 2024-01-11 00:00:00 GMT and Not After 2025-01-10 23:59:59 GMT. That means it's good for one year.

However, even though Adobe's new cert has a Not Before date of 2024-01-11, it wasn't rolled out until 2024-01-23. It took 12 days before FotoForensics had its first sighting of Adobe's new cert. Most companies generate their new cert and immediately deploy it. I'm not sure why Adobe sat on it for 12 days.

In the example picture above, the metadata says it was last modified and cryptographically signed on 2024-01-17, which is 6 days after the new cert was created. If I didn't suspect that Adobe did a slow release of their new certificate, then I'd probably be wondering why it was signed using the old cert after the new cert was available.

The problem of expiring certs will also be problematic when cameras, such as Leica , Canon , and Nikon , begin incorporating C2PA technology. Here are some of the issues:

• Standalone cameras often have clocks that are unset or off by minutes or hours. (Do you want to capture that spontaneous photo, or set the clock first?) Unlike cellphone cameras, many DSLR cameras don't set the time automatically and lose time when the batteries are removed or replaced. If the clock isn't right, then the cert may not be valid for signing.
  
• If the cert is expired, then the camera owner can always backdate the time on the device. This permits claiming that a new photo is old and backing up the claim with a C2PA signature. Even if you trust Canon's cert, you cannot extend that trust to the camera's owner.
  
• The few standalone camera certs I've seen so far have only been valid for a year. That means you will need to update the firmware on your camera at least annually in order to get the new certs. I don't know any photographers who regularly check for firmware updates. New firmware also increases the risk of an update failing and bricking the camera or changing functionality that you previously enjoyed. (When doing a firmware update, vendors rarely change "just one thing".)
  
• Many DSLR cameras lose vendor support after 1-2 years. ( Canon is one of the few brands that offers 5 years for most cameras and 10 years on a few others.) Your new, really expensive camera may not receive new certs after a while. Then what do you do? (The answer? Stop using that camera for taking signed photos!)

Expired and Revoked

There's one other irony about C2PA's use of certificates. In their specification ( Section 1.4: Trust ), they explicitly wrote:

C2PA Manifests can be validated indefinitely regardless of whether the cryptographic credentials used to sign its contents are later expired or revoked.

This means that the C2PA decision makers probably never considered the case of an expired certificate being backdated. Making matters worse, C2PA explicitly says that revoked certificates are acceptable . This contradicts the widely accepted usage of certificates. As noted by IBM :

When x.509 certificates are issued, they are assigned a validity period that defines a start and end (expiration) date and time for the certificate. Certificates are considered valid if used during the validity period. If the certificate is deemed to be no longer trustable prior to its expiration date, it can be revoked by the issuing Certificate Authority (CA). The process of revoking the certificate is known as certificate revocation. There are a number of reasons why certificates are revoked. Some common reasons for revocation are:

• Encryption keys of the certificate have been compromised.
• Errors within an issued certificate.
• Change in usage of the certificate.
• Certificate owner is no longer deemed trusted.

In other words, if a C2PA signature is signed using an explicitly untrusted certificate, then (Section 1.4) the signature should still be considered "valid". This is a direct conflict with the X.509 standard.

C2PA's Section 16.4: Validate the Credential Revocation Information , includes a slight contradiction and correction, but doesn't improve the situation:

When a signer’s credential is revoked, this does not invalidate manifests that were signed before the time of revocation.

It might take a while for a compromised certificate to become revoked. According to C2PA, all signatures made by a known-bad-but-not-yet-revoked certificate are valid. (Ouch! And my coworker just remarked: "It's not like an election can occur in the intervening time. Oh wait, did I say the bad part out loud?")

If you are generating a forgery and have a revoked certificate, you can always backdate the signature to a time before the revocation date. According to C2PA, the signature must be accepted as valid.

With X.509 certs and web sites, revocation is very rare. However, this needs to change if these certs are embedded into devices for authoritative signatures. Some of the new cameras are embedding C2PA certificates for signing. If your camera is lost, stolen, or resold, then you need to make sure the certificate is revoked.

Legal Implications

[Disclaimer: I am not an attorney and this is not legal advice.]

The C2PA specification lists many use cases . Their examples include gathering data for open source intelligence (OSINT) and "providing evidence". Intelligence gathering and evidence usage typically come down to legal purposes. (Ironically, during a video chat discussion that I had with C2PA and CAI leadership last December, they mentioned that they had not consulted with any experts about how C2PA would be used for legal cases.)

I've occasionally been involved in legal cases as a subject matter expert or unnamed technical consultant. From what I've seen, legal cases often drag on for years. Whether it's an ugly divorce, child custody battle, or burglary charge, they are never resolved in months. (All of those TV crime shows and legal dramas, where they go from crime to arrest to trial in days? Yeah, that kind of expedited timeline is fiction.) I sometimes see cases where evidence was collected but went untouched for years. I've also seen a lot of mishandled evidence. What most people don't realize, is that usually a named party in the lawsuit produces the evidence; it's not collected through law enforcement or a trusted investigator. Also, it's usually not handled properly so there are plenty of opportunities for evidence tampering. (As a subject matter expert, I enjoy catching people who have tampered with evidence.)

So, consider a hypothetical case that is based on photographic evidence with C2PA signatures:

• The files were collected, but the signatures were never checked.
  
• Due to issues with evidence handling, we can't even be certain when they were digitally signed, how they were handled, or how they were collected. There is plenty of opportunity for evidence tampering.
  
• By the time someone does check the files, the certificates have expired. They may have expired prior to being documented as evidence.

This is the "Case #4" situation: expired and not validated by a trusted party. Now it's up to the attorneys to decide what to do with an expired digital signature. If it supports their case, they may ignore the expiration date and use it anyway. If it doesn't support their client's position, then they might get the evidence thrown out or flagged as tampered, spoiled , altered, or fabricated. C2PA's bad decision to use X.509 certificates with short expiration windows helps with any " motion to suppress ", " suppression of evidence ", or " exclusion of evidence ."

Without C2PA, we have to rely on good old forensics for tamper detection. With these methods, it doesn't matter when the evidence was collected or who collected it as long as there is no indication of tampering. C2PA adds in a new way to reject otherwise legitimate evidence.

The same concerns apply to photojournalism. If a picture turns up with an expired certificate, does that mean it is just an old photo? Or could it be fabricated and backdated so that it appears to have a valid signature from an expired certificate?

There are many ways to do cryptography and many different frameworks for managing public and private keys. The use of X.509 certificates with short expiration dates, and the acceptance of revoked certificates, just appears to be more bad decisions by C2PA. The real problem here isn't the use of X.509 -- that's just a case of using the wrong tool for the job. Instead, the bigger problem is leaving the signing mechanism in the hands of the user. If the user has malicious intent, then they can use C2PA to sign any metadata (real or forged) as if it were real and they can easily backdate (or post-date) any signature.

---

Značky: #FotoForensics, #Authentication, #Programming, #Network, #Politics, #Forensics

Each January, I try to clean up around the office. This includes getting old papers off my desk and emptying my email box. Similarly, I'm reviewing my long-running research projects. Between FotoForensics and my honeypots, my servers are constantly running a wide range of research projects:

• Some of these research projects have been short-lived. For example, I was seeing a bunch of images with an unknown EXIF tag identified as "0x9999". Using the public service's data, I gathered a bunch of pictures with the same EXIF 0x9999 tag and realized that the field contained JSON information from Xiaomi smartphones. I informed the developers of ExifTool, and as of ExifTool 12.72 , ExifTool decodes the field.
  
• I often use the data I collect for regression testing. Whenever any of my dependencies releases a new version, or when I make a significant code change, I run a month's worth of pictures (or more) through the code to see how the output changes. This allows me to account for any unexpected issues, either in my code or in the dependency. (And more than once, these large-scale tests have identified new bugs. We've been able to address these before any public release.)
  
• I have some long running research projects where I have been collecting data for years. These allow me to identify long-term trends, significant short-term changes, and how manufacturers adopt new standards over time.

This week, I turned off one of my longest running projects: X-Wap-Profile. It tracks a web browser header that used to be popular, but seems obsolete today.

What's this X-Wap-Profile stuff?

Each time your web browser sends a request to the web server, it sends a large number of HTTP header fields. These fields describe the browser, client OS, and other client settings. Consider this HTTP client header:

POST /upload-file.php HTTP/1.1
Host: fotoforensics.com
Connection: keep-alive
Content-Length: 257624
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: https://fotoforensics.com
X-Wap-Profile: http://wap1.huawei.com/uaprof/HUAWEI_RIO-UL00_UAProfile.xml
User-Agent: Mozilla/5.0 (Linux; Android 5.1; HUAWEI RIO-UL00 Build/HUAWEIRIO-UL00) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/39.0.0.0 Mobile Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary91du4bKYR6ZB05sH
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,en-US;q=0.8
X-Requested-With: com.android.browser

This HTTP client header includes a lot of information:

• The user was uploading a file to FotoForensics (POST /upload-file.php, content-length 257,624 bytes).
  
• The client's device was configured for Spanish from Spain (language es-ES ) and English from the United States (en-US). The "en-US" is default for virtually all web browsers. If the server supports internationalization, then it should try to respond in Spanish. Otherwise, it should default to American English.
  
• The client supports two types of compression: gzip and deflate. The server can compress the response using either algorithm.
  
• The "User-Agent" identifies an older Android device (Android 5.1) on a Huawei Rio smartphone.
  
• The user is using the default android browser (com.android.browser). In this case, it's Mobile Chrome version 39.

And then there's the X-Wap-Profile. This is part of the User Agent Profile ( uaprof ) specification from the Wireless Application Protocol Forum (WAP). This standard dates back to October 2001.

This HTTP header is supposed to specify a URL that provides more detailed device information. The URL should return a well-defined XML file with details like the screen size (this Huawei's resolution is 1920x1080), native audio, video, and image format support (amr, aac, wav, midi, jpeg, png, H.264, etc.), and much more. Some device profiles identify the amount of memory or CPU model. Others identify the OS version, software versions, and whether they can be upgraded.

While this data provides details about the device, none of this information is personal to the user; this isn't a privacy issue. However, I can totally see how a malicious server could use this information to customize a hostile response to a vulnerable client. I mean, the uaprof data often includes protocols and version numbers; I'm sure there are known vulnerabilities for most unpatched systems. (I'm surprised I haven't heard of this attack vector being exploited by anyone.)

Applied User-Agent Profiles

In theory, the web server can request the X-Wap-Profile data and optimize the response for the client. At FotoForensics, I've been collecting this X-Wap-Profile data for years. If I know the device's make and model, then I can immediately look up the CPU, RAM, screen size, and more. Originally I had been hoping to use it to identify more information about the client and optimize what I could easily present. (Why bother trying to send them a 1024x800 image when the device has an 800x600 display?)

While the theory sounds good, in practice, servers usually ignore the uaprof data. My server collected it but never used it for customizing responses. I don't know any servers or services that actually used this data.

Why does everyone ignore it?

• Using this data requires retrieving it from a URL. However, different servers use different timeouts. Some may not respond immediately or may take a long time before failing. These delays make real-time retrieval impractical, especially if the user is waiting on the response. Any delays will result in a bad user experience. And remember: users won't know that your server reply is delayed because of a slow dependency; they'll just notice that your system is slow.
  
• Although the XML format is well-defined , many URLs returned poorly formatted XML, non-standard XML values, or other formats like HTML or JSON. This becomes a parsing nightmare for the server. For example, "http://phone.hisense.com/khfw/rjxz/201208/P020120809608224982262.xml" used to be a valid URL. (It doesn't work anymore.) To properly view this file, you needed to render the HTML web page (with JavaScript enabled) and then copy the rendered text into the XML profile.
  
• Many vendors are inconsistent. For example, Sony Ericsson used different vendor names in different uaprof files: "SONYERICSSON", "Sony Ericsson Mobile Communications", "Sony Mobile Communications", "SonyEricsson", "Sony Ericsson Mobile Communications AB", and "Ericsson Mobile Communications AB". Asus used "ASUSTeK COMPUTER INC." and "ASUS", and Lenovo alternated between "Lenovo Mobile", "LenovoMobile", and "Lenovo". The number of vendors that were consistent is significantly less than the number that changed their names arbitrarily.
  
• A few vendors think this data should be "proprietary". For example, the LG LS990 (http://device.sprintpcs.com/LG/LS990-Chameleon/LS990ZV4.rdf, back when it was online) had a comment field that said, "Elements that are not required have been omitted from this template." The uaprof file listed the screen size as "0x0" and the CPU as "NotDisclosed". I find this secrecy ironic since LG made these specs public on their web site. (The LS990 uses a 2560x1440 display and a Qualcomm Snapdragon 801 Quad-Core Processor.)
  
• Often, the URL never existed. For example, as far as I can tell, "http://support.acer.com/UAprofile/Acer_A510_JZO54K_Profile.xml" and "http://www.sonyericsson.com/UAProf/T68R401.xml" were never valid URLs, even when the device was new.
  
• Some vendors never bothered to participate. (Yes, I'm talking about Apple.)
  
• Even if the URL was valid, some URLs returned information for the wrong device. This appeared to happen when the manufacturer just copied over another device's profile and forgot to update the data for the new device.
  
• If you didn't retrieve the uaprof data when the device was new, then the URL would eventually go away. (This is one of the big reason I had been collecting them for years.) In the last year, nearly every uaprof URL that I've seen has been offline. Even ones that used to work no longer work. (It's rare to find one that still works today.)
  
• Most uaprof URLs use hostnames, but some contain hard-coded IP addresses. This is particularly the case for vendors in China and Japan. With hostnames, the vendor can move the site to different network addresses and the URL will still work. But with hard-coded addresses, it means that the vendor must never change the network address, upgrade to IPv6, or alter their network architecture. Not surprisingly, most of these URLs with hard-coded IP addresses fail.
  
• And speaking of URLs... I saw a few instances of the same device using different URLs. Since the URL should be hard-coded, this detail grabbed my attention. I ended up finding a few users who had traveled between countries like India and China, Australia and China, or Russia and China. I knew each was the exact same devices because the picture's metadata included the device's unique serial number. When they were in their home country (India, Australia, etc.), I would see the expected X-Wap-Profile URL. (The URL is hard-coded and set by the web browser.) But when they went to China, the URL would change. Sometimes the URL would work and sometimes it didn't. It appears that the Great Firewall of China was changing the X-Wap-Profile URLs in real-time during the web request. (And switching my server to HTTPS did not deter these alterations. It appears that China was intercepting and altering HTTPS traffic.)

While lots of devices used to support these X-Wap-Profile URLs for identifying additional device profiles, that really isn't the case today. The only time I see these URLs now is when someone uses a really old smartphone. Newer smartphones don't include the X-Wap-Profile header.

Archiving the Old

I'm certainly not the only person who was collecting these URLs. Open Mobile Alliance has a long list of X-Wap-Profile URLs. (While they are marked as 'valid', almost all are no longer online.) There's also a SourceForge project with a list of URLs (again, most are no longer online).

For my own research, I had a script that was watching every incoming HTTP header and harvesting any new X-Wap-Profile URLs. After 12 years, this script had collected 10,702 unique uaprof URLs, with the last one being retrieved on 20-June-2023: http://www-ccpp.tcl-ta.com/files/8050D_HD.xml . It isn't that I had stopped collecting them after last June. Rather, the script hadn't seen any new ones in over six months.

I don't know if the Wireless Application Protocol Forum or Open Mobile Alliance (the groups behind this specification) are defunct, or if mobile device vendors just decided to stop supporting this specification. In either case, it's no longer being used. For now, I'm not going to delete what I've collected, but I am going to archive it and disable the collection script. Maybe there will be a purpose for this data someday in the future.

While this uaprof collection script never took many resources, there's no reason to continue running it when it's not collecting anything. I think a lot of the internet is on auto-pilot, with old scripts that are running even though nobody is looking at the results. This month, I'm trying to do my part by removing obsolete processes.

---

Značky: #FotoForensics, #Network, #Forensics, #Programming

As someone with insomnia, I typically sleep 3-6 hours a night. About once every 2 weeks, I might sleep 7-8 hours, but that's really unusual. My college roommates couldn't understand how I could function on so little sleep. It isn't a medical issue. (I don't wake up needing the bathroom and I don't wake up tired.) I wake up fully rested. Insomnia is just one of those things that runs in my family; one of my parental units often sleeps only a few hours a night.

For years, I tried to find ways to make my sleep more regular. (If I'm going to sleep 5 hours, then let's make it consistently 5 hours.) I finally found a great solution: naps. For the last few years, I've been taking naps once a day, usually in the middle of the afternoon. Just 15-30 minutes is great for recharging myself. And best of all, my nighttime sleeping is now consistently at 6 hours. It doesn't matter when I go to bed. I will wake up 6 hours later without any kind of alarm. (It's almost to the minute.)

The problem with naps is that I would lay down and my brain would be buzzing with ideas. I couldn't turn it off and wouldn't easily fall asleep. Fortunately, I found a solution that works for me: brown noise.

The Color of Noise

Many years ago, my late friend Nurse (Brad Smith) gave a fascinating talk about Weaponizing Lady Gaga . Basically, different sounds can directly influence your brain. If you were to include these psychosonic frequencies in a concert, you could make the crowd peaceful, irritated, revved up, or more susceptible to suggestions. (And yes, some commercials embed these frequencies to influence buyers. I'm not going to link to them, but you might see a medication commercial that's supposed to make you feel better while they play a calming psychosonic frequency. You'll associate 'calm' and 'feeling better' with the medication.)

A simple version of this concept is basic background static. You can buy noise machines that make sounds like rain or ocean waves. The actual type of noise is often named after colors and are based on specific noise frequency distribution properties.

• White noise has a uniform frequency spectrum. It's great at drowning out low volume background noise. If your office has a bothersome ventilation fan or buzz from fluorescent lighting, then some very low volume white noise can drown it out.
  
  
• Green noise focuses on the middle frequency ranges; fewer high and low notes. It often has a calming effect on people. (Hospital waiting rooms and police holding cells should play it at a very low volume. Playing it just above the hearing threshold is very effective. Blasting it loudly doesn't make people calmer.)
  
• Pink noise has a linear distribution, with more lower frequencies than high frequencies. This often sounds like a heavy rain. Some people report that this really helps them sleep.
  
  
• Brown noise (also called brownian noise) uses a really steep frequency distribution. Lots of low frequencies and almost no high frequencies. If pink noise kind of helps you sleep, then brown noise will knock you out cold.
  

Build Your Own Sleep Machine

While I knew about these different noise patterns, I never really gave them much thought. Until one day I noticed something. My better half and I sometimes watch streaming TV together. During the ad breaks, there would be an ad for a table-top pink noise and brown noise generator. I couldn't help but notice that every time it played a few seconds of brown noise in the commercial, she would yawn.

Of course, I'm not the type of person who would shell out money to buy a brown noise speaker system. Instead, I used the Linux audio program "sox" to generate about 15 minutes of brown noise as a music file.

My first attempt just sounded like static to me. So I did a little tweaking with the sox audio filters.

My second attempt gave me a very distinct calming sensation. So I did a few more adjustments.

I knew the third attempt worked because I woke up 30 minutes later, still sitting at the keyboard. Totally well rested. The mp3 had ended 15 minutes before I woke up.

I added this song to my "favorites" playlist and now I use it every time I take a nap. Seriously:

1. Lay down in a semi-dark room and hit play on "Brown".
   
2. Try not to think for just two minutes. (For me, turning off all of the thoughts in my head is like trying to hold my breath. I can't do it for very long.) Don't quietly repeat to yourself "don't think". Don't think about the next things you need to do. Don't listen for the settling of the house, or anything else. Just focus on the brown noise.

For me, I'm typically out cold after 1-3 minutes. The mp3 lasts for 15 minutes. I usually end up napping for 15-30 minutes, but occasionally for an entire hour. The sleep duration definitely varies per person. Test it on yourself when you have a quiet afternoon. Maybe set an alarm if you don't want to sleep too long.

I even use this when traveling by plane. Right after the flight attendant gives the safety presentation, I put on the headphones and hit play . I'm usually asleep before liftoff and I wake up after we reach cruising altitude.

Try it!

I've packaged up the mp3 files as an 'album' and uploaded it to my music server. On the album, I have 15 minutes of white, green, and brown noise. I also have some chicken sounds on the album because those help you snap out of it when the song ends.

---

Značky: #Network, #Programming, #Forensics

A month ago, I wrote a very detailed and critical review of the Coalition for Content Provenance and Authenticity (C2PA) specification for validating media metadata. My basic conclusion was that C2PA introduces more problems than it tries to solve and it didn't solve any problems.

Although a few people have left comments on the blog, I have had a near daily stream of new contacts writing in. Some are curious developers, while others are people tasked with evaluating or implementing C2PA. The comments have come from small organizations, Fortune-500 companies, and even other CAI and C2PA members. (Since some of them mentioned not having permission to publicly comment, I'm treating them all as anonymous sources.)

Despite having different areas of interest, they all seem to ask variations of the same basic questions. (It's so frequent, that I've often been cut-and-pasting text into replies.) In this blog entry, I'm going to cover the most common questions:

1. Are there any updates?
2. How bad is C2PA's trust model?
3. Isn't C2PA's tamper detection good enough?
4. Is C2PA really worse than other types of metadata?
5. What's the worst-case scenario?

For people interested in problem solving with forensic tools, I have a challenge in the 5th item.

Question 1: Are there any updates?

There have been a couple of updates to C2PA since my blog came out. Some appear to be unrelated to me, and some are directly related.

• C2PA released version 1.4 of their specifications. Unfortunately, it doesn't address any of the problems with the trust model. (Since this happened a few days after my blog, it's an unrelated update.)
  
• Some of the bugs submitted to various C2PA github projects suddenly became actively addressed by the C2PA/CAI developers. (Including some submitted by me.) While this is progress, none have been fixed yet. In most cases, the bugs seem to have been copied from the public github locations to Adobe's private bug management system. (This means that C2PA development is really only happening inside Adobe.) Unfortunately, none of these bugs address C2PA's trust model.
  
• Last week I had a productive discussion with people representing C2PA and CAI. I'm not going to disclose the discussion topics, except to say that there were some things we agreed on, and many things we did not agree on.
  
  Excluding myself and my colleague, everyone on the call was an Adobe employee. In my previous blog , I mentioned that CAI acts like a façade to support C2PA. Between all of the key people being at Adobe and all development going through Adobe, I'm now getting the distinct impression that CAI and C2PA are both really just "all Adobe all the time." It does not appear to be the wider community that C2PA and CAI describe.
  
• A few days after the conference call, Adobe made a change to CAI's Content Credentials web site. Now self-signed certificates (and some of the older non-test certificates) are flagged as untrustworthy: "This Content Credential was issued by an unknown source."
  
  
  The only way around this requires paying money to a certificate authority for a "signing certificate". (At DigiCert, this costs $289 for one year.) Switching to a fee-based requirement dramatically reduces the desirability for smaller companies and individual developers to adopt C2PA. And yet, adding in a fee requirement does nothing to deter fraud. (Fraud is a multi-billion dollar per year industry. The criminals can afford to pay a little for "authentication".)
  
  As an aside, "HTTPS" adoption had a similar pay-to-play limitation until Let's Encrypt began offering free certificates. Today, HTTPS is the norm because it's free. Unfortunately, those free certificates are for web domains only and not for cryptographic signatures .

Question 2: How bad is C2PA's trust model?

My previous evaluation identified different ways to create forgeries, but it only superficially covered the trust model.

Back in the old days of network security, we relied on "trust", but "trust" was rapidly shown to be vulnerable and insecure. "Trust" doesn't work very well as a security mechanism; it's not even "better than nothing" security. "Trust" is used for legal justification after the fact. We trust that someone won't ignore the "No Soliciting" sign by your door. The sign doesn't stop dishonest solicitors. Rather, it establishes intent for any legal consequences that happen later.

In networking, "trust" was replaced with " trust but verify ", which is better than nothing. Today, network security is moving from "trust but verify" to " zero trust ".

C2PA is based heavily on the "trust" and not "trust but verify". (And definitely not "zero trust".) With C2PA:

• We trust that the metadata accurately reflects the content. (Provenance, origin, handling, etc.) This explicitly means trusting in the honesty of the person inserting the metadata.
  
• We trust that each new signer verified the previous claims.
  
• We trust that a signer didn't alter the previous claims.
  
• We trust that the cryptographic certificate (cert) was issued by an authoritative source.
  
• We trust that the metadata and cert represents an authoritative source. While the cert allows us to validate how it was issued ("trust but verify"), it doesn't validate who it was issued to or how it is used.
  
• We trust the validation tools to perform a proper validation.
  
• We trust that any bad actors who violate any of these trusts will be noticed before causing any significant damage. ( Who will notice it? We trust that there is someone who notices, somewhere to report it, and someone who can do something about it. And we trust that this will happen quickly, even though it always takes years to revoke a pernicious certificate authority .)

All of this trust is great for keeping honest people honest. However, it does nothing to deter malicious actors.

C2PA is literally based on the honor system. However, the honor system is the definition of having no security infrastructure. (We don't need security because we trust everyone to act honorably.)

Question 3: Isn't C2PA's tamper detection good enough?

Putting strong cryptography around unverified data does not make the data suddenly trustworthy. (It's lipstick on a pig !)

Other than the cryptographic signature, all of C2PA's metadata can be altered without detection. (Yes, C2PA also uses a hash, like sha256, as a secondary check, but it's trivial to recompute the hashes after making any alterations.) The cryptographic signature ensures that the data hasn't changed after signing. C2PA calls this tamper evident , while other C2PA members describe it as being tamper resistant or tamper proof . Regardless of the terminology, there are four possible scenarios for their tamper detection: unsigned, invalid, valid, and missing.

• Case 1: Unsigned data
  This cases occurs when the C2PA metadata exists but there is no cryptographic signature. As I've mentioned, everything can be easily forged. Even though the C2PA metadata exists, you cannot explicitly trust it since you don't know who might have changed it.
  
  The C2PA specification permits including files as a component of another file. This unsigned case can easily happen when incorporating media that lacks C2PA metadata. In my previous blog entry , I pointed out that CAI's example with the flowery hair contains a component without a signature.
  
  
  
  
  What this means: If there is no cryptographic signature, then we cannot trust the data. We must use other, non-C2PA methods to perform any validation.
  
• Case 2: Invalid signature or checksum
  When the C2PA metadata has an invalid checksum or signature, it explicitly means something was changed. However, you don't know what was changed, when it happened, or who did it.
  
  Keep in mind, this does not mean that there is intentional fraud or a malicious activity. Alterations could be innocent or unintentional. For example, importing a JPEG into the Windows Photo Gallery ( discontinued but still widely used) automatically updates the metadata. This update causes a change, making the C2PA signature invalid. The same kind of unintentional alteration can also happen when attaching a picture to an outgoing email. (As opposed to importing a picture into the iPhone Photo Library, which simply strips out C2PA metadata.)
  
  If the signature is invalid, then C2PA says we cannot trust the data. However, the important aspects of the data may not have changed and may still be trustworthy. We must use other, non-C2PA methods to determine the cause and perform any validation.
  
• Case 3: Valid signature and checksums
  There are two different ways that a file can contain valid signatures and checksums:
  
  (A) While we don't know if we can trust the data, we know we can trust that it wasn't changed after being signed.
  or
  (B) The data was changed after being signed and any invalid signatures were replaced, making it valid again. In this case, the data and signatures are untrusted.
  
  In both cases (A and B), the signatures and checksums are valid and we cannot trust the data. Moreover, we can't distinguish unaltered (A) from intentionally altered (B). Since we cannot use C2PA for trust or tamper detection, we must use other, non-C2PA methods to perform any validation.
  
• Case 4: No C2PA metadata
  This is currently the most common case. Does it mean that the picture never included C2PA metadata, or that the metadata was removed (stripped)?
  
  CAI's Content Credentials service addresses this problem by performing a similar picture search (perceptual search). Most of the time, they find nothing. However, even if they do find a visually similar file that contains C2PA metadata, it doesn't mean the data in the search result is trustworthy or authentic. (See Cases 1-3.)

In every case:

• The data is untrusted.
• Intentional tampering cannot be detected.
• The data must be validated through other, non-C2PA means.

What does C2PA provide? Nothing except computational busywork. C2PA's complexity and use of buzzwords like "cryptography" and "tamper evident" makes it appear impressive, but it currently provides nothing of value.

Question 4: Is C2PA really worse than other types of metadata?

Given that C2PA does not validate, why do we need yet another standard for storing the exact same information?

The metadata information provided by C2PA is typically present in other metadata fields: EXIF, IPTC, XMP, etc. However, C2PA provides the same information in an overly complicated manner, requiring 5 megs of libraries and four different complex formats to process. In contrast, EXIF and IPTC are simple formats, easy to implement, require few resources, and come in very small libraries. Even XMP (Adobe's XML-based format that sucks due to a lack of consistency) is a better choice than C2PA's JUMBF/CBOR/JSON/XML.

A few people have written to me with comments like, "C2PA has the same long-term integrity issue as other metadata" or "Isn't C2PA as trustworthy as anything else?"

My reply is a simple "No." C2PA is much worse:

1. Regular metadata doesn't claim to be tamper-evident.
   
2. Regular metadata doesn't use cryptographic signatures.
   
3. Regular metadata doesn't have the backing of tech companies like Adobe, Microsoft, and Intel that give the false impression of legitimacy.

Other types of metadata (EXIF, IPTC, etc.) can be altered and should be validated though other means. C2PA gives the false impression that you don't need to validate the information because the cryptographic signatures appear valid.

Even the converse is problematic. Because the metadata can be altered without malicious intent (see the previous Case 2), an invalid C2PA signature does not mean the visual content, GPS, timestamps, or other metadata is invalid or altered. Everything else could be legitimate even with an invalid C2PA signature.

Unlike other types of metadata, C2PA's cryptographic signatures act as a red herring . Regardless of whether the signature is valid, invalid, or missing, forensic investigators should ignore C2PA's signatures since they tell nothing about the data's validity, authenticity, and provenance.

Question 5: What's the worst-case scenario?

This is the most common question I've received. I usually just explain the problem. But for a few people, I've manufactured examples of the worst-case scenario for the questioner to evaluate. In each of these instances, I used the questioner's own personal information to really drive home the point.

Since lots of people are reading this blog entry, I'm going to use Shantanu Narayen as the fictional example of the worst-case scenario. Narayen is the current chair and CEO of Adobe; his personal information in this example comes from his Wikipedia page. (I'm not doxing him.)

In my opinion, the worst-case scenario is when the data is used to frame someone for a serious crime, like child pornography. Here's the completely fictitious scenario:

Last month, Shantanu Narayen got into an online argument with a very vindictive person. The vindictive person acquired a bunch of child pornography and used C2PA to attribute the pictures to Narayen. The pictures were then distributed around the internet.

It doesn't take long for the pictures to be reported to the National Center for Missing and Exploited Children (NCMEC). NCMEC sees the attribution to Narayen and immediately passes the information to the FBI and California's Internet Crimes Against Children (ICAC) task force. A short while later, the police knock on Narayen's door.

I'm not going to include a real picture of child porn for this demonstration of a fictional situation. (That would be gross, illegal, and irrelevant to this example.) Instead, I used a teddy bear picture (because it's "bearly legal").

For this evaluation, ignore the visual portion and assume the picture shows illegal activity. Here's one of those forged pictures!


Now, you get to play the role of the investigator: Prove Narayen didn't do it.

You'll probably want to:

• Download the image. (Save it as 'bearlylegal.jpeg'.) To make sure it wasn't modified between my server and your evaluation, the file is 306,924 bytes and the SHA1 checksum is 04857686607b05b9fe3efef11fa6a11cd68e51df.
  
• Evaluate it using whatever metadata viewers you have available.
  
  
  • If you don't have a starting place, then you can use my own online forensic services, like FotoForensics and Hintfo . But don't feel like you need to use my tools.
    
  • From the command line, try exiftool or exiv2 .
    
  • Most graphical applications have built-in metadata viewers, including Mac's Preview, Windows file properties ("Details" tab), Gimp, and Photoshop. (Just be careful to not hit 'save' or make any changes to the file's metadata.)
  
  
• Use Adobe's command-line c2patool to evaluate the C2PA metadata. With c2patool, you can view what sections are valid and the x.509 certificate contents:
  c2patool -d bearlylegal.jpeg
c2patool --certs bearlylegal.jpeg | openssl x509 -text | less
  
• View it at Adobe/CAI's Content Credentials online service. While this web service currently isn't as informative as the command-line c2patool, it does provide information about the file's C2PA metadata.

If you try solving this problem, either with software or as a conceptual exercise, I'd love to hear your results! I hope people include information like:

• What's your background? Inquisitive amateur, student, software engineer, law enforcement, legal, professional investigator, or something else?
• What tools did you use?
• What were some of your observations?
• What other issues should we consider?
• Do you think you could prove innocence or reasonable doubt?
• Let's turn the problem around: If you were working for the prosecution, do you think you could get a conviction?

I look forward to hearing how people did!

[Spoiler Alert]

* If you want to evaluate this problem on your own, come back to this section later.

Worst-Case Results

Given all of these findings, there are a few other things to consider:

• Law enforcement, attorneys, and the courts are not very technical. If the file's metadata says it's his name and it has a valid signature, then it's valid until proven otherwise. Remember: digital signatures are legal signatures in a court of law; they are accepted as real until you can prove tampering. (And saying "I didn't do that" doesn't prove tampering.)
  
• Narayen can say "that's not my picture!" While Leica may be able to verify that the cert isn't legitimate, Leica is in Germany and Narayen is in the United States. That makes serving a subpoena really difficult. In general, law enforcement (and everyone else who isn't Leica) cannot verify this. Also, Leica is a big company and can have multiple certs. Maybe they just don't remember creating this one or maybe it was part of a limited beta test.
  
• C2PA's trust model assumes that a forged certificate will be eventually noticed. However, Narayen doesn't have months or years to wait for the forged certificate to be reused with some other victim. And that's assuming it is reused; it doesn't have to ever be reused.
  
• You can try to explain that the cert only validates that the data existed and hasn't been changed since signing. But the courts see the words "authentication" and "provenance" and "certified" and "verified" and "validated". The evidence clearly says it is attributed to Narayen.
  
• You might see flaws in the authenticated forgery. The prosecution will claim that the flaws are evidence that Narayen tried and failed to hide his identity. (As many people in the legal system have said, " We don't catch the smart ones. ")
  
• While the C2PA tools don't identify any problems, you might notice problems if you use other tools. In that case, why even bother with C2PA? But that's a rhetorical question and the answer carries no weight in court. The fact is, the metadata identifies Narayen and the cryptographic signature for authentication and provenance says the metadata can be trusted.
  
• Worse: even if the signature appears fake, it doesn't mean Narayen didn't do it. Remember: there is other metadata besides C2PA's metadata that names Narayen. There are also multiple pictures naming him, and not just one photo.
  
• This is a very technical topic. Really technical explanations and jargon will quickly confuse the courtroom. Remember: The judge still thinks a FAX machine is a secure data transmission system and most jury members don't know the difference between "Google" and "the Internet". Assuming you can identify how it was forged, communicating it to the judge and jury is an uphill battle. (Creating a forgery in a live demo would be great here. However, while live demos are common in TV crime shows, they almost never happen in real life. Also, live demos can go horribly wrong, as demonstrated by the OJ Simpson trial's "if the glove fits" fiasco. No sane attorney would permit you to perform a live demo.)
  
• Even if you, as the expert, can explain how it was forged, your testimony will just appear to be you trying to discredit the certified authenticated provenance information. Remember: C2PA claims to be tamper-evident. But in this case, everything checks out so there is no evidence of tampering.
  
• Most people can't afford, don't have access, and/or don't know an expert witness who can determine if the C2PA metadata is legitimate. As the CEO of Adobe, using his own employees as experts carries little or no weight. (Any expert from Adobe is clearly biased since Narayen can fire them at any time.) Law enforcement and the courts will assume: If it says it's from Narayen and the tamper-evident C2PA says there is no evidence of tampering, then it's from Narayen.
  
• Let's pretend that you can identify the forgery, demonstrate that C2PA does not work, and communicate it clearly to the court. It's now your word against every technical expert at Microsoft, Intel, Sony, Arm, DigiCert, and a dozen other big tech companies. Who has more credibility? Hundreds of highly paid security and cryptography professionals or you? And remember: these big companies have their reputations on the line. They have a financial incentive to not be proven wrong

As an online service provider, I've interacted closely with NCMEC and ICACs for over a decade, and with attorneys and law enforcement for even longer. I can tell you that the prosecution won't spend much effort here. The cops will knock on his door. Narayen won't be able to convince the courts how it happened, and he'll either be found guilty or his attorney will convince him to take a plea deal.

Narayen's only option in this fictional scenario is to demonstrate that C2PA does not verify the metadata, the metadata can be altered, anyone can sign anything, and C2PA doesn't provide validated provenance and authenticity -- even though it's in the name: C2 PA . I'm not a lawyer, but I think this option also could show that every company currently selling products that feature C2PA are actively engaged in fraud since they know what they are selling doesn't work.

Either C2PA doesn't work and Narayen walks free, or C2PA works and this forgery sends him to jail. That's the worst case scenario, and it's very realistic.

Truth or Consequences

If this fictional child porn example seems too extreme, then the same application of fake C2PA metadata works with propaganda from wars (Ukraine, Gaza, etc.), insurance fraud, medical fraud, fake passport photos, defective merchandise claims at Amazon and Etsy, altered photojournalism, photo contests, political influences, etc. At FotoForensics, I'm already seeing known fraud groups developing test pictures with C2PA metadata. (If C2PA was more widely adopted, I'm certain that some of these groups would deploy their forgeries right now.)

To reiterate:

• Without C2PA: Analysis tools can often identify forgeries, including altered metadata.
  
• With C2PA: Identifying forgeries becomes much harder. You have to convince the audience that valid, verifiable, tamper-evident 'authentication and provenance' that uses a cryptographic signature, and was created with the backing of big tech companies like Adobe, Microsoft, Intel, etc., is wrong.

Rather than eliminating or identifying fraud, C2PA enables a new type of fraud: forgeries that are authenticated by trust and associated with some of the biggest names on the tech landscape.

A solution for assigning authentication and provenance would go a long way toward mitigating fraud and misrepresentation. Unfortunately, the current C2PA specification is not a viable solution: it fails to authenticate, is trivial to forge, and cannot detect simple intentional tampering methods. It is my sincerest hope that the C2PA and CAI leadership tries to re-engage with the wider tech community and opens discussions for possible options, rather than making sudden unilateral decisions and deploying ad hoc patches (like requiring pay-to-play) that neither address the basic problems nor encourage widespread adoption.

---

Značky: #Forensics, #Network, #Security, #FotoForensics, #Programming

I usually follow a simple mantra: if it can be automated then it should be automated. However, while automation may seem like a great solution, it introduces a new set of perils. Unexpected critical points, untested conditions, and unforeseen failures can lead to new problems.

At FotoForensics , Hintfo , and my other services, I use a lot of automation. I have tools that detect and respond to network attacks, terms of use violations, server monitoring, and basic system management. However, I never just deploy an automation and walk away. I never just assume that it works.

When I release new automated solutions, I usually spend as much time testing, monitoring, and adjusting the new system as I do developing it. Sometimes new problems are identified quickly and are easy to resolve. Some issues take a while to surface, and some have no easy solutions. A few times, I've had automated scripts fail so badly during the initial testing that they had to go back to the drawing board or were treated as a failure and a good learning experience.

Unfortunately, my overly-cautious approach to software deployment seems to be the exception and not the norm. Often, companies deploy automated solutions without any deep review.

What's for dinner?

At our local grocery store, they removed more than half of the checkout aisles and replaced them with self-checkout kiosks. The space required by 3 manual checkout counters can fit six kiosks. In addition, one employee can (frantically) manage all six kiosks rather than one employee per counter. When I visit the grocery store, they usually have 10 of 16 kiosks open with little or no wait. (They open the other six during the busy periods.) And while they still have eight counters available, usually only 1 or 2 are open for the people who don't want to use the self-checkout kiosks.

On the positive side:

• Automation seems to reduce lines at the grocery store. I rarely have to wait for the kiosk. In contrast, grocery stores that don't have automated kiosks usually has long lines even when the store doesn't seem very crowded.
  
• Around here, every store has a "Help wanted" sign. Automation allows the store to operate with fewer employees.
  
• I'm not advocating the replacement of employees with machines. However, the existing employees can be moved to other essential tasks, like restocking shelves and helping customers.

Unfortunately, there is a down side to checkout counter automation:

• Saving money on employees does not translate into lower prices. Because there is less human oversight, it's easier to steal from these automated kiosks. Why pay extra for that "organic" apple when you can choose the lower-cost generic (inorganic?) apple from the kiosk's menu? Or maybe scan one box of Mac and Cheese but put two in the self-bagging area? I suspect that the increase in theft is part of the reason that prices have increased even through employee-related expenses has decreased.
  
• These machines are not always easy to use and customers often need assistance. At my grocery store, self-scanning coupons is usually more miss than hit.

Some stores are starting to re-evaluate their automated self-checkout systems. A few are even removing them altogether.

The problem isn't that the kiosks don't work. Rather, while automation solves some problems, it creates opportunities for new problems.

Neither snow nor rain nor heat nor gloom of night

The grocery store kiosks usually work well. They are mostly easy to use. (Even with coupon scanning problems and the occasional double-scan, I'd give them very high marks.) However, not all kiosk are designed well.

For example, every few years, my bank installs a new ATM machine. Sometimes the machine interfaces are really easy to use, while other versions are confusing and take twice as long to use. You can tell when the ATM interface is bad because more people go into the bank for assistance.

And then there's the US Post Office. It's the holiday season and the Post Office has long lines of people waiting to mail gifts and packages. Our Post Office has a couple of automated self-service kiosk (SSK) machines, but almost nobody is using them. These SSKs have no waiting line and are located right next to the long line of people who are waiting for assistance. (You can't miss these machines!)

It isn't that the SSKs are down or out-of-order. Rather, the user interfaces are designed so incredibly poorly, that people would rather spend 45 minutes waiting in line for human assistance. The few people using these SSKs seem to fall into one of three categories:

• They spent a lot of time earlier learning how to use them, so they know what selections to use.
  
• They think they might battle the confusing interface faster than waiting in the long line. (Sometimes this gamble pays off, sometimes it doesn't.), or
  
• They have a friend holding their place in line, just in case they can't figure out how to use the SSK. (Seriously. You often hear someone say "can you hold my place in line for a moment?" before trying their luck at an SSK.)

Then again, if you succeed in using the Post Office's SSK, you might get asked by a few strangers to help them, too.

Et tu, Etsy?

Etsy is an online marketplace for handmade goods and is the epitome of automation gone wrong. For example, earlier this year , Etsy's rolled out a filter to remove any Etsy listings that also appeared on the Chinese (cheap / knock-off goods) site called Temu. The problem is that some Temu accounts appear to steal images and product descriptions from Etsy. As a result, legitimate Etsy sellers are having their handmade products delisted. In some cases, Etsy even shuts down the legitimate Etsy shop. A few examples from the Etsy forums:

• " Etsy removed my handmade ceramic listing ": A user has repeatedly had their handmade bottles removed. Keep in mind, the bottles look handmade, are signed by the seller, and there are videos showing how she makes the bottles. Another Etsy seller responded:
  

  I had a listing removed two months ago and after opening a support request with videos showing my process I was able to get it reinstated after two weeks of waiting. However, the exact same listing was removed again last week for the same reason so even if you prove you’re handmaking the item, they may still remove it in the future. I’m hoping if they reinstate a listing it doesn’t still count as a strike against your shop since it seems this will continue to happen (in my experience)

  
  
• " China has stole my product and photos! I can't get Etsy to help. ": A seller laments that their product photos are being stolen by knock-off sellers from China.
  
• " Etsy destroyed my store. They are responsible for their own downfall. ": Again, legitimate handmade items were delisted by Etsy because knock-off sellers copied her photos.
  
• " I need some help with handmade policy violation - at wits end ": The seller uses an engraver and uv printer and is having custom items removed. One of the replies identified the likely cause: "The bots are taking over, so they may not be sophisticated enough to distinguish between similar silver tone keychains over a pure white background with similar text/sayings, etc."

A few Etsy users have posted advice for getting products or account reinstated, but your success at any of these tips is really hit-or-miss.

Etsy's Amber Alert

This isn't the only automation and filtering problem at Etsy. Back in 2016, a child died after choking on amber teething beads that had been purchased on Etsy.

I'm not going to dive into this loss of a child or even the merits of the court case. Rather, my focus is on Etsy's reaction: they banned products that included the word "amber". The initial ban appears to have been automated and does not pay attention to the context. "Amber color?" Nope. "Amber is my first name?" Nope. This forum discussion really describes the problem well (my bold for emphasis):

Listings deactivated for using the word amber to describe glass bead colour
by TheBeckoningCat Post Crafter ‎05-14-2022 10:36 AM

Received a deactivation notice from Etsy regarding the word "amber" appearing in my description. I cannot get thru to Support by any means. In my description of the bracelet, I mentioned that the Swarovski squares were of an "amber color". Swarovski, to my knowledge, does not offer any crystals made from amber. I have since changed the description to "honey brown color". Perhaps in the future, when your "bot" or algorithm comes up with results for people selling "amber" a human actually review them to determine whether actual amber is being sold. I just did a simple search for the word "amber" and came up with over 132,000 results, just in the US alone. The people who are selling Amber Heard T-shirts will have to revise their listings as Johnny Depp's ex-wife.

---

Reply by BlackberryDesigns Conversation Maker ‎05-14-2022 11:50 AM

I would get rid of the word Swarovski , also. I got hit on that one from my website. Apparently since making the shift and removing their crystals for sale to the artists, they are doubling down on their brand name and do not want it associated with us little guys.

---

Reply by JDTotesnDolls Community Maker ‎03-14-2022 04:17 PM

The bot is designed to look for the word amber and not to make decisions on whether or not it is a color or an actual bead.

Contact Etsy and ask them to manually review the listings and with the removal of the word, they may okay a relist,

Unfortunately, Etsy's use of the ban-hammer seems inconsistent at best . The Etsy bots appear to use poorly defined rules that are applied irregularly and enforced occasionally, resulting in confusion among their sellers. Adding to the problem, the only real solution is to talk to a human in their support group, and that seems like a Herculean task.

The problems related to bad bots and automated filtering is more than just an inconvenience for users. Some Etsy sellers have reported that scammers are using Etsy's poor filtering services as a front for blackmailing sellers. For example:

( I'm the bot in charge of security ) HELP
by KummLeather Inspiration Seeker ‎09-15-2023 04:49 PM

hello, I received such a message, does anyone have information about this?

"Hello, my name is Jackson, I'm the bot in charge of security. You have received a complaint that you are a scammer, in order that your account was not blocked send me your name and surname, as well as your e-mail.
You will need to be verified today, in the worst case your account will be blocked.
Send me your first name, last name and e-mail!!!!!"

---

Etsy support bot
by Bellayall Inspiration Seeker 09-15-2023 03:56 PM

I just received a message from someone claiming to be an Etsy support bot. He said that the product I shared was reported and asked for my name, surname and email address.I'm new to Etsy and didn't know it was a scam so I gave her what he wanted and then he blocked me. Is this a problem please help

"Is this a problem"? Yes. Yes it is.

Choke Points

A heavy dependency on automation also creates central points of failure in critical systems. Earlier this year, a failed update by the FAA caused thousands of flight cancellations across a wide range of airlines. A few months later, Southwest had their own software failure that caused them to cancel thousands of flights.

Keep in mind, these were not cyber attacks. These were just system failures in critical systems. These single points of failure were associated with the automation of their day-to-day operations. This is why a single computer outage can stop both automated and manual operations.

Ideally, critical services should have failover backups. Ideally, the operators should have tested for these scenarios and had viable workarounds. However, automation often results in unidentified choke points and a lack of immediate options.

This limitation is not restricted to critical infrastructures. Our local library has had a few instances where the self-checkout book kiosks have gone offline. One librarian said the kiosk outage was related to their new weekly backup system. Fortunately, they do have a workaround: They put up "out of order" signs and manually handle checkouts at the front desks.

Welcoming Our New Robot Overlords

Unfortunately, I'm seeing more instances of people relying on automation without testing. I'd cite specific instances of people mistakenly relying on ChatGPT, but there were too many references to choose from! Here's a few examples:

• Legal : An attorney used ChatGPT without checking the results. It made up fake citations. The attorney ended up getting sanctioned by the court.
  
• Legal : A different attorney was suspended after using ChatGPT and not noticing that it was citing fake cases.
  
• Legal : Another attorney was fired after using ChatGPT.
  
• Medical : Researchers found that ChatGPT was usually wrong when responding to a medical question. (It was correct 10 out of 39 times.)
  
• Summarization : When asked to generate automated summaries, ChatGPT often makes up results.
  
• Factuality : Poynter reported that, when fact-checking results, ChatGPT sometimes reached accurate conclusions, but "struggled to give consistent answers, and sometimes was just plain wrong."

Just because a solution is deployed, doesn't always mean it's a perfect solution.

A few companies have managed to get automation to work quite well. For example, I recently needed to return an item to Amazon. I went down to my local Whole Foods (where Amazon has a kiosk) and was able to effortlessly follow the instructions to return the item. (The hardest part was opening the cellophane bag that they provided.)

I also drove through Kansas this summer. Part of the trip included a tollway. I had affixed the free tollway pass to my car a month earlier. As a result, I managed to bypass the long line of cars at the tollbooth and I never came to a stop. This was easy and much less expensive than paying at the booth.

It seems to me that the success of the automation really comes down to the company's focus. Companies that are consumer-focused, like Amazon, the Kansas tollway, and the grocery store, have fewer usability problems but introduce new potential venues for fraud. In contrast, profit and process-oriented automated, like Etsy and the Post Office, seem to generate more problems than they attempt solve.

---

Značky: #Forensics, #Programming, #AI, #Network

I recently presented a short version of my " No-NOC Networking " talk to a room full of managers C-level executives, policy makers, and technical evangelists. These are not the people in a network operation center (NOC), but it did include people who manage the people in the NOCs. For this presentation, I lowered the technical level to someone who hasn't taken a basic networking class in years. The talk was very positively received.

Most of the questions were exactly as I expected, like "Where can I get your software?" and "What were those Linux commands to turn off scanner responses?" (The answer to both is at the Nuzzle web site). But there were also a few people who had the same kind of unexpected request: Where can I learn more?

They weren't asking about learning more ways to deter network attackers. Rather, they were asking about where they could learn more about security in general. A few people confided that they weren't sure if they were too old to learn.

Full stop: You are never too old to learn about security. Whether you are looking for a late-in-life occupational change or just a new hobby, you can always learn this as a new skill.

Where to Start?

I often hear people talking about starting with a certification, like CISSP , CEH (certified ethical hacker), or some of the SANS courses. However, I think that's like diving into the deep end of the pool. (As an aside: this isn't a recommendation for any of these certifications. As someone who tracks down cyber bad guys for a living, I've seen more unethical behavior from people with CEH credentials than any other group!)

Before you get too deep or start paying for some kind of education, start by finding out what you like. There isn't just one kind of "computer security". What's your interest? Here's a short sample of different cyber areas:

• Cryptography (great for math and puzzle people)
• Network security
• Policy
• Reverse-engineering
• Social engineering
• Red team (offense)
• Blue team (defense)
• Hardware and IoT (internet of things)
• Software (fuzzing is fun!)
• Physical security, including lock-picking
• Anonymity and privacy
• AI and counter-AI (yes, there's a security element)

This is far from everything. Most of these categories include forensics (detection) and anti-forensics (avoiding detection).

Don't know which one to choose? Try them all and find what you like! (Personally, I like the weird stuff, like file format forensics and packet-level network forensics. But I've also worked with everything else in this list.)

Groupies

Besides trying to learn on your own, there are tons of conferences . Most weeks have multiple conferences worldwide. While conferences usually require paid admission, a few are free or even online.

There are also tons of meet-up groups. Many of these groups are part of larger organizations. For example:

• The Open Worldwide Application Security Project (OWASP) started with a focus on web-security best practices. However, it has evolved. Today, it mostly focuses on policies and processes for securing generalized applications. The organization has individual satellite chapters that hold monthly meetings all over the world.
  
• DEF CON is a huge hacker conference that is held once a year. However, it has spawned lots of smaller " DEF CON groups " that meet monthly. Most are identified by their telephone area code. For example, Denver is area code 303, so their local DEF CON group is " DC303 ". Unlike OWASP, the DC groups usually have topics that span the entire spectrum -- software, hardware, social engineering, etc. Often, they include live demonstrations and how-to information. Some of these groups meet in person, while others are online.
  
  If you're new to DC groups and you don't like one topic, then wait a minute and there will be a different topic. The only warning: the content can be extremely detailed and discussions may quickly go over your head.

Most of these groups are friendly, helpful, and welcoming to new people. Also, most of them look down on using technology for evil, malicious, or illegal purposes. You're not going to learn how to compromise your ex's Facebook account or how to steal money from an online transaction.

Can't find a local group? Try using Meetup . Search for your city and the "Technology" category. In Fort Collins (where I am), we have "Women Who Code", "Fort Collins Internet Professionals" (FCIP), Northern Colorado Hackers (NoCo Hackers), a couple of Python developer groups, web developer groups, and more. And keep in mind, Fort Collins isn't a "big city"; big cities have even more groups. Unless you're out in the middle of nowhere (sorry, Casper, Wyoming ), there's probably something nearby.

Hands On

Beyond groups, many organizations offer various games where you can try tools, techniques, and methods in a controlled and safe environment. (I liken it to how cats sharpen their claws.) The games often include different skill levels, from newborn novice to guru expert. In my opinion, the real-world problems are nowhere near as difficult as the harder games.

So how can you find these games?

If you ask in any of the social groups (OWASP, DC, etc.) then someone is bound to provide some suggestions. But even without group participation, there are lots of 'capture the flag' (CTF) opportunities out there. These include challenges and puzzles that award points for completion. Some are meant for individuals, while others permit teams. (Often, teams are looking for new members. It's usually easy to find a team that will take on a new person.) Some of the better-known CTFs include:

• CTF101 : The challenges include forensics, cryptography, and more.
  
• PicoCTF (login required) and Hacker101 CTF (login required) includes practice levels, challenges, and competitions.
  
• The National Cyber League and Try Hack Me both provide more formal learning environments.

If you really enjoy these CTF games, then there are competitive teams. Many of the larger conferences have CTF contests with prizes for the winners.

Personally, I find game play to be a great way to teach and test knowledge. At my FotoForensics service, I include a few ' Challenges ' (Tutorials → Training → Challenges) where people can try to evaluate pictures in a controlled environment.

New Tricks for Old Dogs

Just as there are different security focuses, there are also different ways to learn. Regardless of whether you prefer self-paced, hands-on, one-on-one, or a classroom environment, there are plenty of options. After you find your interest and get a taste of the technologies, then you can start focusing on formal certifications and professional education... or you can be an informed amateur.

Most companies, universities, and news outlets focus on cybersecurity as a career . (OMG! 650,000 cyber jobs are now vacant !) However, it doesn't have to be a career. These topics have benefits even in small amounts. With a little practice, you will start noticing fraud and scams, identifying poor security practices, and distinguishing the real threats from hype. The fundamentals used to include reading, writing, and arithmetic. Then it expanded to some computer literacy. Today, a little computer security knowledge is becoming a fundamental requirement. It's time to start learning!

---

Značky: #Forensics, #Programming, #Security, #Conferences, #Network

A few blog entries ago, I wrote about problems with helping other people with their computers. (Most of the problems were due to the software and not the people.) This turned into a discussion about automating the monitoring of the elderly and making a minimal " Are you okay? " system. My solution uses a very simple script to monitor their computer for signs of activity since they all use their computers fairly regularly. If they change their habits and miss a check-in window, then it triggers an alert.

While monitoring the computer is a good start, it's still not ideal. Recently I've been evaluating miniature embedded systems and homemade IoT devices for simple automation tasks. These included Raspberry Pi miniature computers as well as Arduino embedded controllers. In the comments to my blog entry, Matt and Jon suggested that I look at the ESP32. It's the same concept as an Arduino, but it has built-in WiFi and bluetooth. (Now I just needed a reason to try it out.)

One of my friends has an elderly parent who lives alone. While this person regularly uses the computer, my friend is concerned about his father falling and going unnoticed for hours. We worked out an inexpensive solution: a network-based motion sensor. (Woo hoo! A reason for getting an ESP32 and a guinea pig for testing!) Here's the idea: we will place 2 of these wifi motion sensors in my friend's elderly parent's home. They are going to be located near the kitchen and the main hallway. These are areas that he walks by often. As a motion sensor, it will be triggered each time his father goes to the kitchen, walks to the bathroom, TV room, bedroom, etc. ("Or you could just call me. I like it when you call." Nope -- remote wireless sensors!)

First Attempt

An Arduino costs about $15 USD but needs a network adapter that costs another $15. In contrast, I purchased my first three ESP32 controllers for about $6 USD each (three CPUs plus development boards for $22 total). Unfortunately, I had to return them because each was faulty. Here's the link to the item at Amazon . However, I do not recommend these. While the development boards are fine, each ESP32 had the same problem: bad ground.


The ESP32 (ESP32-WROOM-32, 38-pin) has three ground pins (in black).

• Good ground . The ground pin in the top right (opposite corner from 5V) is good. Use it for everything.
  
• Bad ground . The ground pin on the left (6 pins from the bottom) is bad. With the power off, I hooked a meter from good-ground to bad-ground. It does have connectivity, but there's a slight delay as the resistance drops to zero. This suggests that there's a capacitor or inductor or something sitting in front of the ground pin. When power is first applied, this pin has zero connectivity for a few microseconds.
  
  I noticed that the device wasn't running the program when power was supplied over USB. I had to press the reset button before the program would run. Other people came up with complicated workarounds (using capacitors and resistors) that effectively press the restart button a moment after power is applied.
  
  Another person noticed that, if you want to use external power, then you need to use good-ground and not bad-ground. Otherwise, it won't boot. I took his idea and tested it: If you jumper good-ground to bad-ground, then it boots properly when power is applied over USB. As a power source, the integrated USB port appears to be hooked to bad-ground. Don't use bad-ground.
  
• Ugly ground . The third ground pin is on the right, 7 pins from the top. According to my meter, this pin has zero connectivity. At all. It's a floating ground pin. At best, it won't work for you. And depending on your electronics, you might end up not having it work, frying something, or causing a fire. Do not use!

If the developers can't get ground right, then who knows what else is wrong with the hardware. I returned it.

I want to emphasize that not every ESP32 has this problem. This appears to be a bad batch. (Most likely, someone bought up a bunch of known-bad chips and tried to sell them for cheap on Amazon.) However, this experience is bad enough to scare me away from anything with this form factor. Fortunately, there are lots of other options.

Attempt #2

A couple of my friends suggested that I look at the M5Stamp-S3 from a company called M5Stack. This is a postage-stamp size ESP32 controller with built-in WiFi and bluetooth. It also has a built-in multi-color LED. And best of all, the version that I got had all the pins already soldered in, so I can plug-and-play without soldering. (I didn't get it from M5Stack because they use a Chinese credit card processor who is associated with fraud and potentially unsafe . Instead, I paid $0.50 more and got it from DigiKey . And since the price for shipping didn't increase with quantity, I ordered three of them.)


This is a device that I can definitely recommend. It was simple to hook up to the sensor rig that I had originally created for the ESP32. (The pins are in different places, so be sure to redo the wiring.)

The Sensor

I configured the M5Stamp as a controller for a wireless motion sensor. For the sensor, I used a tiny microwave doppler radar switch. While it uses microwave frequencies, it's so low power that you won't fry anyone. The microwaves go through wood, drywall, doors, etc. The only things that really stop it are metal and water. Humans are basically large bags of water and create a strong microwave reflection. As you walk past the sensor, it triggers for 2 seconds.

The thing that I like about this sensor is that you don't have to drill a hole in the wall. Just mount it on the wall near an electrical outlet. It has a range of over 9 feet (3 meters). However, I'm currently seeing brief flashes of activity when there shouldn't be any. (Either there's a ghost in my house that is triggering the motion sensor, or there's a little chatter in the electronics.) I probably just need a little shielding or to filter the chatter in software. But I think this is really close to being ready. (Maybe another 1-2 weeks.)

Here's the first attempt at the box:

• The orange block is the M5Stamp.
  
• The M5Stamp is plugged into an 11x5 mini breadboard.
  
• The breadboard is sitting on top of a piece of cardboard with aluminum tape on one side. The aluminum tape blocks any RF noise from bothering the motion sensor and the cardboard prevents the metal tape from shorting out the electronics on the sensor.
  
• Under the aluminum and cardboard is the motion sensor. (Just the pins are peeking out the top.)
  
• The electronics are mounted in a box. I have a piece of foam (not shown) that keeps everything from moving around. (The box is much larger than needed because it was originally designed for the ESP32-WROOM, which is a much larger controller chip. The next version will be smaller.)
  
• The entire thing plugs into a USB cable and power plug.

When it's done, it will just need to be plugged in and hung on the wall at about ankle level. (You can't see it in this photo, but there's a hook hole for a nail at the top of the box.) It can easily be hidden out of view.

My program for the M5Stamp does a few things:

1. It connects to the WiFi network.
   
2. It watches the motion sensor.
   
3. When there is any activity, the on-board LED turns on. No activity turns the LED off.
   
4. I track when the sensor was triggered and maintain a delay of 20 seconds for the motion event. This way, small chattering is ignored until there is at least 20 seconds of inactivity.
   
5. At the start of each motion event, it uses the WiFi to contact a reporting web page. It only reports that activity was seen near the sensor.

Now, if there isn't activity for a few hours, then either (A) the person isn't home (check their iPhone and friend tracker), (B) the person is asleep (check the clock and don't panic at night), or (C) the person has fallen and needs help. Moreover, it's more reliable than waiting for the person to use their computer and trigger any proof-of-life script. It's also non-intrusive; while we know there is movement in the house, we don't know what he's doing or where he's going. (It's not like putting a camera in every room in the house.)

Other Uses

Using a wireless motion sensor is a great solution if you don't have pets. However, dogs and cats will trigger the sensor. I mentioned this to a friend of mine and he loved the idea for his elderly dog ! He currently uses multiple PIR (passive infrared) sensors to alert him when the dog walks around at night. (If he doesn't get up when the dog gets up, there's going to be a messy accident.) A single WiFi motion sensor would cover the entire area without being as directional as a PIR.

Another friend of mine lives with a parent who has dementia. He's planning on tracking every external door of the house, "just in case she makes another escape attempt." (It sounds funny, but it's really a serious problem.)

Version 2

After I finish the first version, I'm planning on making a second one for myself. It's going to monitor the front door because UPS, FedEx, Amazon, and the postal service have all decided to never ring the doorbell. They walk up, drop off packages, and run away. I'm usually in the house. If I'm near the door, I might hear the thump of a box and realize there's a package. I do have a camera by the door, but it can take up to 60 seconds before my phone beeps. (The notification is great if I'm next to my phone, but often my phone is in a different part of the house.)

The next wireless monitor will watch the front of the house. If anyone approaches the doorstep, it will let me know immediately. I'm also going to have a second sensor inside the doorway. This way, someone walking out of the house will trigger the inside sensor first, allowing me to know when someone is leaving and not trigger the doorbell. In contrast, someone walking up to the door first will trigger the door sensor. And since it's microwave, I don't have to worry about drilling a hole in the wall or mounting something outside in a weather-resistant case.

With these inexpensive and customized IoT systems, I might end up with a smarter home that works the way I want it to work -- and at a fraction of the price of a mass-produced solution.

---

Značky: #Network, #Security, #Programming

In my last blog entry , I mentioned about helping other people with their computers. While I didn't mention anyone's age, a lot of the feedback has been related to elderly relatives. (Only some of the people I tried to help were elderly.) These comments led to some very interesting discussions about monitoring the elderly.

Whether it's your parents, grandparents, distant relatives, or nearby neighbors, we all know someone who is elderly. Unless you live with them, you probably check up on them during the occasional visits, while on walks around the block, or through phone calls, emails, texting, and social media apps. I know many people (myself included) who have scheduled weekly calls to check in and chat. The problem is, if we don't hear from them, we just assume they are busy. We usually don't get concerned until after days or weeks pass.

My biggest fear is to learn that someone had an accident, like falling, and remain unnoticed for days. I remember reading an article about a man who died and nobody noticed for 7 years. That's when his auto-pay bank account ran out of funds for utilities. Another deceased person went unnoticed for 8 years .

I've been chatting with friends about different kinds of "proof of life" monitoring. Personally, I don't want to install a camera in every room of someone's house. That's too invasive. But at the same time, friends and family should know that the person is moving around and doing the expected day-to-day things. An alert should be triggered whenever the daily routine is disrupted. My friends and I have come up with a few solutions.

Solution #1: Panic Button

Devices like Life Alert, LifeCall, Lifeline, and invisaWear are wearable devices that you can press if you have an emergency. However, they are large, bulky, and unflattering to wear. The button also don't work if you can't reach it or are unconscious.

The TV commercials for these devices always show a happy elderly person receiving the device, and then using it while in distress. I'll tell you from first-hand experience: I'd rather have dental surgery than try to convince an elderly person to carry the device around every day, "just in case they need it."

A few decades ago, these panic buttons were good solutions. But there are much better options today.

Solution #2: Apple Watch

The Apple Watch includes a fall detector. If you fall, it sounds an alert. If you don't stop the alert, then it uses the built-in cellphone to call for help.

The Apple Watch is a great (but expensive) out-of-the-box solution. No technical programming needed and it's easy enough that even my non-techie elderly friends can use it. As an emergency monitoring system, this is a bare-minimum solution. However, it has some serious limitations. For example:

• If you fall, then it must be a hard fall. If you hit a sofa, slide down a wall, or partially catch yourself, then it won't detect the fall.
  
• If you fall and land on the watch, it could be damaged. (For example, falling onto hard concrete can break the watch.) A broken Apple Watch won't call for help.
  
• If you're not wearing the watch, then it won't detect the fall. This includes falling while getting out of bed, slipping in the shower, or not wearing the watch while it is charging.
  
• Speaking of charging... The watch needs to be regularly recharged. If you forget and it loses power, then it won't help you.
  
• The watch needs cellular connectivity to call for help. One of my friends has a cellular blind spot in the kitchen, between the refrigerator and the oven. (If you want good reception, move away from the kitchen.)

Let's say it's a soft fall and you are conscious. You can always use the phone to dial for help, right? Well, not necessarily. What if you broke one or both arms when you hit the ground? Then you can't easily touch or navigate the watch's interface. If the user has "Hey Siri" enabled, then it may not register when your voice has a lot of stress. (It doesn't recognize screaming.)

The worst case? A soft fall as you lose consciousness. The watch is no help here.

Solution #3: Daily Pattern Monitoring

Rather than watching for a life-impacting event, I've been thinking about detecting the absence of an event. For example, when I travel, I always have my laptop with me. If I get to the hotel too late at night, I might not call home (I don't want to wake anyone up). However, I am guaranteed to check my email when I get to the hotel. I've recently configured my laptop to trigger a "proof of life" URL every time it goes online. This way, my friends and family who want to know if I made it safely can always check to see if my laptop was turned on.

Similarly, I have a couple of elderly friends who always check their computers. I've recently modified their Windows configuration to trigger a proof-of-life URL anytime the computer is logged in (including when screensaver is deactivated). If they don't use their computer at least once a day, then it will trigger an alert.

To do this, I just needed to create three things: (1) a VisualBasic script to trigger the proof-of-life URL, (2) a task scheduler event that runs the script when there is a login event, and (3) a receiving service that looks for missed events.

Keep in mind, these are the steps I used. I wouldn't be surprised if there was a better or easier option.

1. Open the Command Prompt. This will default to your home directory (C:\users\ name \). Create a directory called 'Scripts' for holding the script to call when the event is triggered. ( mkdir Scripts )
   
2. Create the script. I used 'Notepad' to create a simple visual basic script. This script calls 'curl' to trigger a URL that will record the activity. I saved the script as \users\ name \Scripts\Proof-Of-Life.vbs (the suffix ".vbs" is important.) Here's the source code:
   

   Set oShell = CreateObject("Wscript.Shell")
   
   Dim strArgs
   
   strArgs = "cmd /c curl -A Eddie+Desktop https://server/life-track.php"
   
   oShell.Run strArgs, 0, false

   Change the URL to point to your own web server, and set the user-agent string (-A) to identify which computer is doing the reporting. The script's 'run' parameters will trigger the tracking URL silently (without having a terminal window briefly popup). In effect, the user won't notice that this happened.
   
   To verify that you did this part correctly, you should be able to open the File Folder, navigate to \users\ name \Scripts\, and run the Proof-Of-Life.vbs file by double-clicking on it (or right-click and select "Open" from the menu). If everything works correctly, nothing will appear to happen on the desktop. However, the remote web server will see a request for "/life-track.php" coming from the computer.
   
3. Open the Windows 'Task Scheduler'. (Go to the Start menu and just type 'Task Scheduler'. It will appear at the top of the list.) This is where things get complicated. For tracking logins, you will select "Create Task" and then fill in the tabs:
   
   
   • General . Give this task a name and description. I called mine "Proof of Life Login". (Caveat: You can't change the name. To change it, delete the task and recreate it with the new name.) Select 'Run only when user is logged on". You don't need to change any of the other default values.
     
   • Triggers . Select "New" to create a new trigger. From the top drop-down menu for "Begin the task", select "On workstation unlock". I also configured it to "Stop task if it runs longer than" 30 minutes. (It should only take a second to run.)
     
   • Actions . Select "New" and "Start a program". Select your "C:\users\ name \Scripts\Proof-Of-Life.vbs" program.
     
   • Conditions . I uncheck the Power settings since I want it to run no matter regardless of whether it's on battery or AC. I also selected the Network with "Any connection".
     
   • Settings . This vbs program should take a second to run. For "Stop the task if it runs longer than", select the minimum time: 1 hour.

With all of these entered, click "OK". Now you should be able to activate the screensaver (Win-L to lock). When you unlock the screensaver, it will immediately trigger the tracking URL.

On the server side, I created a life-track.php script that (1) validates the user based on the User-Agent string value, and (2) logs the information in an SQLite database. I record the user, date and time, and IP address. I also have a cronjob that checks the SQLite database every few hours to make sure that the user triggered the URL. If no user was seen, then it sends an alert email to me. (My first response will be to check if they are supposed to be home. Then call them, and if that fails, then issue a welfare check on them.)

As far as privacy goes, the only people who knows about the data are myself and the person who said I could monitor them. The monitoring is also not intrusive: I don't know what they are doing at the computer, or even how long they are on the computer. I only know when a proof-of-life was last observed. If the person is injured or missing, then I'll know within a few hours. Worst case, they will be on the floor for up to 16 hours (night check through next morning check), but that's much better than having nobody know there's a problem.

Alternate Use

For my laptop, I have a similar Windows Task called "phone home". It runs automatically whenever the laptop connects to any network. For this script, I used a custom event trigger:

• Begin the task: On an event
• Log: Microsoft-Network-NetworkProfile/Operational
• Source: (leave blank)
• Event ID: 10000 (that's when the network is up)

When I get to the hotel, I use my laptop to connect to the WiFi, and it automatically triggers a proof-of-life.

(This has the added benefit of tracking my laptop if it is ever stolen. If the thief turns it on and connects to any wireless network, it will immediately and silently call home.)

Configuring Windows Tasks is not intuitive. There are tons of event names and numeric identifiers, and very little documentation. A good start is to look in the "Event Viewer". Every event is logged and lists both the log file and the numeric code.

More Options

Looking for a change in the daily pattern of life is a great option for monitoring someone's welfare. If they end up falling or being incapacitated, then they may be down or hurt for a few hours, but it won't be for days or weeks before someone notices.

Besides tracking login access and network connectivity, there are other great uses for these types of monitors. For example, I have one Windows computer that I only use with one client. I can use these triggers to monitor both start and stop times, allowing me to automatically track my billable hours. (Why estimate to the nearest 15 minutes when I can see the exact times that the computer was in use?)

The tracking doesn't even need to be a global system event. I showed this to one of my coworkers and they immediately put in a tracker around their social media apps. The event contacts the tracking server each time the program starts and stops. Now they know exactly how much time they are wasting online.

The tracking URLs don't even need to be accessible over the internet. I could have my script contact a local embedded device, like a Raspberry Pi, Arduino, or ESP32, that runs a simple web server. The micro computer can then trigger some event or activity. Personally, I might make one that beeps every hour, so I remember to get up and move around a little. (It's not healthy to sit in one place for hours.) Or maybe have it automatically adjust the room lighting and temperature when it sees that I'm working.

The Dark Side of Tracking

While these technologies can be used to track the welfare of elderly friends, they can also be abused. A stalker with access to your computer can use this technique to monitor when you are at the computer. Employers could use them to determine when you are not working.

Fortunately, you can use the Task Scheduler to see what other tasks are currently on the system. If you see an unexpected task, you can easily disable or delete it.

On my own systems, I noticed that Google and Microsoft both added event tasks to check for updates. I modified those so that they only run on my home network. (Woo hoo! No more "auto update" while giving a presentation at a conference!) Personally, I don't care how high the risk is that the patch wants to fix; I don't want updates when I'm traveling. When I'm on the road, the risk from a malicious or failed update is almost always worse than the problem being patched.

---

Značky: #Privacy, #Network, #Programming