Eliminating PHP polyfills

The Symfony project has recently created a set of pure-PHP polyfills for both PHP extensions and newer language features. It allows developers to add requirements upon those functions or language additions without increasing the system requirements upon end users. For the most part, I think this is a good thing, and valuable to have. We've done similar things inside MediaWiki as well for CDB support, Memcached, and internationalization, just to name a few.

But the downside is that on platforms where it is possible to install the missing PHP extensions or upgrade PHP itself, we're shipping empty code. MediaWiki requires both the ctypes and mbstring PHP extensions, and our servers have those, so there's no use in deploying polyfills for those, because they'll never be used. In September, Reedy and I replaced the polyfills with "unpolyfills" that simply provide the correct package, so the polyfill is skipped by composer. That removed about 3,700 lines of code from what we're committing, reviewing, and deploying - a big win.

Last month I came across the same problem in Debian: #911832. The php-symfony-polyfill package was failing tests on the new PHP 7.3, and up for removal from the next stable release (Buster). On its own, the package isn't too important, but was a dependency of other important packages. In Debian, the polyfills are even more useless, since instead of depending upon e.g. php-symfony-polyfill-mbstring, the package could simply depend upon the native PHP extension, php-mbstring. In fact, there was already a system designed to implement those kinds of overrides. After looking at the dependencies, I uploaded a fixed version of php-webmozart-assert, filed bugs for two other packages. and provided patches for symfony. I also made a patch to the default overrides in pkg-php-tools, so that any future package that depends upon a symfony polyfill should now automatically depend upon the native PHP extension if necessary.

Ideally composer would support alternative requirements like ext-mbstring | php-symfony-polyfill-mbstring, but that's been declined by their developers. There's another issue that is somewhat related, but doesn't do much to reduce the installation of polyfills when unnecessary.

Kiwix in Debian, 2018 update

It's been a relatively productive year for the packaging of Kiwix, an offline Wikipedia reader, in Debian. My minimum expectations for the Buster release scheduled in mid-2019 are that all necessary C/C++ libraries to build the latest versions of Kiwix are in Debian.

  • libzim is at v4.0.4, and basically ready to go.
  • libkiwix is prepared for v3.0.3, but is waiting on the FTP masters to approve the new version that is in the NEW queue (fifth oldest binary-NEW package as of this post...). Once that is approved, I have the v3.1.1 upgrade prepared as well.
  • ctpp2 was a bit of a struggle, but is ready to go as well. However, it looks like the upstream website has vanished (it was inactive for years), so it's good that Kiwix is planning to replace ctpp2 in the future.

There are three main user facing packages that are in the pipeline right now:

  • zimwriterfs is waiting in the NEW queue. I'm hopeful that we can get this included in time for Debian Buster.
  • kiwix-tools is a bundle of command-line utilities, the most useful of which is kiwix-serve. There's one upstream issue relating to how JavaScript code is embedded, but the packaging prep work is basically done. It's blocked on the new libkiwix anyways.
  • kiwix-desktop or just "Kiwix" is the most interesting thing for typical users who want to read offline content, but it's probably the farthest away. I spent some time this week figuring out how to get it to compile, and mostly got it to work. Probably something to aim for the next Debian release. There's also work underway to provide it as a flatpak, which will benefit even more people.

I also created a "Kiwix team" in the Debian Package Tracker, so you can subscribe to different notifications for all of the Kiwix-related packages (though it might be spammy if you're not very interested in Debian internals).

And, thank you to the Kiwix developers for always being receptive to my bug reports and patches - and providing extra chocolate and stickers :-)


Human Rights Day 2018

This summer I participated in a study abroad trip where we went to Greece and Italy to report on the refugee crisis. In Catania, Sicily, they had a celebration for World Refugee Day that happened earlier in the week. One of the most memorable and overwhelming moments was when three refugees started reading the Universal Delcaration of Human Rights:

Watch on YouTube

Article I: All human beings are born free and equal in dignity and rights. They are endowed with reason and conscience and should act towards one another in a spirit of brotherhood.

Article III: Everyone has the right to life, liberty and the security of person.

Today is the 70th anniversary of the signing of the UDHR, and it seems more relevant and necessary now than ever.

The importance of Artemis Fowl

My bookshelf

Artemis Fowl, sitting right of center on my "favorite books" shelf.

Nearly two decades later, the Artemis Fowl movie is finally happening. It's hard for me to overstate how important Artemis Fowl has been to me. One of my friends asked me if I saw the trailer today and I pretty ecstatically said yes. Artemis Fowl Confidential, a website I registered with back in 2008, sent me an email as soon as it was released. Immediate nostalgia.

I read the original Artemis Fowl sometime in elementary school, by the time the final book, Artemis Fowl: The Last Guardian, was released, I had already graduated high school.

Sometime in eighth grade I joined the Wikipedia WikiProject Artemis Fowl - a group of editors dedicated to improve Wikipedia's coverage of Artemis Fowl related articles. I went through the archives and even found the original post, welcoming me to the project. Those were my first friends on Wikipedia...Calvin, Icy, Laptopdude. Miss y'all.

And at some point I learned templates, creating Template:AF Cite Book. Then that turned into the first ever bug I would file in Wikimedia Bugzilla (I still have bug 2700 memorized for some reason).

And then that Wikipedia thing spiraled out of control, and somehow I ended up with an actual, real, job. Definitely due to other things, but just a little bit thanks to Artemis Fowl.

Thanks Eoin, and here's to the next twenty years of Artemis Fowl!

Improving quality and maintenance of election result boxes on Wikipedia

I've spent a decent amount of time reading Wikipedia articles about Senate, House, and state races this week. And...there were inconsistencies. Specifically in the election box result templates:

Before I updated the article

That's what it looked like before I updated the article to use my template instead. There are a few different issues. First, Mike Thompson isn't marked as the incumbent. And second, the total number of votes is wrong - if you do the math, it adds up to 292,091. There are two more cosmetic issues: 1) the % column should go to one decimal point, and 2) the empty turnout field should be hidden since we don't have that data available.

The fixed version

That's the fixed version, that's using my template. So what's different? The main thing that most editors will notice is the amount of wikitext it took to generate my version: {{election box US auto|California|2016|United States Representative District 5|Mike Thompson link=Mike Thompson (California politician)}}. Compare that to what was needed previously. I think it's a pretty big improvement. Oh, and if you set the year to a comma separated list, like "2012,2014,2016", it'll generate all three boxes at once, so it becomes even easier to use.

This will also reduce the maintenance burden significantly. These boxes are copied to other articles, including Mike Thompson (California politician), which is using the wrong styles and missing the 2016 general election entirely, and on United States House of Representatives elections in California, 2016, which actually looks correct!

This is all being generated by a single Lua module called Module:Election box US auto, and a tabular data spreadsheet that's available on Commons. The Lua code isn't the cleanest, but it proves that we can replace things that were manually maintained with smarter templates that do most of the heavy lifting. To the best of my knowledge, this appears to be the first usage of the new tabular data system in English Wikipedia articles. I've updated the California's 1st congressional district through 8th district articles to use the "auto" template so far.

What's next? I'm going to import the MIT Election Data for the US House dataset to Commons so we can start using this in more articles outside of California soon (waiting on bot approval). The only thing that's missing from that dataset is incumbency data - it doesn't indicate whether the candidate was running as an incumbent (so if you know of incumbency data, please let me know!). And once we get the House in good shape, we can move onto the Senate and then state races. Aaaand I've even had someone ask me about expanding this to other countries, which should totally be doable! Anything is possible with Lua+tabular data.

On Jim Acosta

Two weeks ago, CNN Chief White House correspondent Jim Acosta came to my university and received the 2018 William Randolph Hearst Award for his work. I'm not really the biggest fan of CNN, but I've been impressed with Acosta's work, and getting to hear him speak was a real treat. He talked about what he endured at Trump rallies, and after hearing that, learning that his press pass was revoked honestly wasn't that surprising. The only way freedom of the press works is if journalists like Acosta ask the tough questions, and hold those in power responsible.

Here's my favorite clips of the student interviews of Acosta, which happened the day after pipe bombs were sent to CNN headquarters:

Jim Acosta discusses the October 2018 attempted bombing of CNN headquarters at SJSU
Full quality video available on Wikimedia Commons.

Jim Acosta discusses his trip back to Cuba at SJSU
Full quality video available on Wikimedia Commons.

Jim Acosta discusses why he became a journalist at SJSU
Full quality video available on Wikimedia Commons.

I also uploaded a full copy of the student interview, and a full copy of Acosta's speech (which I haven't had time to cut into smaller segments yet).

P.S.: If you ever get the chance to meet him, ask to see his socks, they're great.