in Tech

tagged freedom, git

Self-hosted git

As part of using only free software, I also started thinking about the various non-free services I am dependent upon. One of those I had already started working on replacing was Github. Github is currently the canonical source location for a lot of my various projects, including some that aren't even on my laptops.

So, alternatives. First I considered a hosted service that runs free software, but those don't really exist any more. Gitorious has shut down, and it turns out that GitLab has gone open core.

Alright then, self-hosted git it is. I tried out and evaluated two projects: cgit and gogs.

cgit is a web viewer for git repositories written in C. I like the UI, having used it while browsing Fedora and Linux kernel repositories. The basic set up of it was pretty simple, I downloaded and unzipped it, set up some Apache CGI rules, and bam, it was running. I imported 2 git repositories, and they showed up right away. I started trying to enable some other features like syntax highlighting, and that's where it stopped being easy to work with. I tried both Pygments and a Perl highlighter, neither worked. Around this point I got bored and gave up.

gogs is a full blown clone of Github's features written in Go. The UI is extremely similar to Github, so it was very easy to figure out. Set up was a little tricky, I had to create a "git" user for it to run as, and then fiddle with setting up an Apache proxy rule so /git proxies to localhost:3000 (I originally started out in a sub-path instead of a full sub-domain). After that, I was able to import a few Github repos directly, and clone them. Yay! It also has a mirror feature that can synchronize with an external repository every hour. I found a gogs-migrate tool that claimed to set up mirrors of all your Github repos in a gogs installation, but I couldn't get it to work. I ended up writing my own Python version called gogs-mirror. And for bonus points, I submitted an upstream pull request to improve an interface message.

Currently I have gogs running at git.legoktm.com. All of my non-forked Github repos are mirrored there, and it also is the canonical source of gogs-mirror. The next step will be to switch the mirroring, so that the canonical source lives on git.legoktm.com, and Github is a mirror. I'll also want to update links to those repositories on places like PyPI, various wikis, etc. More to come!

in Tech

tagged freedom, fsf, stickers

I joined the FSF today

Wow, two blog posts in one day. Okay, I was close and just didn't finish in time.

As the title alludes, I became an FSF member today yesterday. Aaand then I proceeded to buy their Super Sticker Mega Multi Pack. ^.^

I thought this would also be a good time to audit my usage of non-free software...at least what's installed on my computer:

  • Dropbox - mainly used to automatically back up photos from my phone to 3 different computers. I can probably replace it with Syncthing.
  • Google Hangouts - I had to install some plugin rpm for it to work. Since I need it for work, I don't really have an option here.
  • PHPStorm - I kept hoping that JetBrains would release an open source version like they did for PyCharm...but they haven't. Our MediaWiki open source license was also never renewed so I've been using the free EAPs, which I'm definitely not comfortable with. I tried out GNOME builder for a Go project, and didn't really like it. I might try out Atom at some point.

I also have my old MacBook, which at this point is mostly used for playing music or watching videos (all using free software!). I'm getting tired of Apple's updates and all the "OMG log into iCloud" nagging, so I'll probably replace it with some flavor of GNU/Linux at some point (see what I did there! ;)).

in Tech

tagged calgames, frc, golang, robots, stickers

CalGames 2015: An adventure into go, networking, and robots

This past weekend I volunteered at the 2015 CalGames tournament, which played the 2015 FIRST Robotics Competition game, Recycle Rush. This year CalGames had a few new things going on, and one of them was using a new field management system (FMS), called Cheesy Arena. Cheesy Arena was written by Team 254 for their own offseason event and is branded as a "field management system that just works".

It's also written in go. I didn't know go. I still don't know go that well. Myself and Lee were in charge of making two major modifications to the system:

  • Use the 2015 round robin elimination system instead of the traditional bracket
  • Create an audience display to display the eliminations rankings

In retrospect, we should have lobbied during the rule change process to use the bracket system since the code was already written for that. Neither of us were involved in the conversation that early on, so we never had that opportunity though. And while it would have definitely made my life easier, I'm very fortunate it didn't happen because I got the opportunity to play with and learn some go.

So, the actual changes. Integrating into Cheesy Arena was actually pretty simple, the function to generate the next rounds in the bracket was called after every match was saved, so we replaced that function with our own that generated the schedule. To figure out what matches needed to be generated, we counted how many elimination matches were already scheduled, and how many had been played. If 0 had been scheduled, we generated the quarterfinals schedule. If only 8 had been scheduled and played, we sorted all the alliances by their cumulative score over the two matches, and advanced the top four. Our code was designed to be as uninvasive as possible, so we had to reverse engineer what alliances teams were on, since the mapping was only stored in the other direction (what teams are on each alliance). We also did not implement any of the tie breaking rules, and didn't really know what would happen if we encountered a tie. Lee and I quickly wrote some code to handle that while at the event Friday afternoon, but we never tested or deployed it. Luckily we never needed it :-).

With the backend of match schedule generation taken care of, we still needed a frontend to display the rankings. The audience displays were implemented with an HTML template, CSS, JavaScript animations and transitions, and websockets to communicate with the backend. Since my JavaScript is much stronger than go, I decided to have the backend export a list of all the teams, their scores, and how many matches they had played over websockets and do the sorting and table generation in JavaScript. I mostly grabbed code from the scoring display, and fiddled with the CSS so it looked like how I wanted it to, and was rather pleased with the result (screenshot). This was finished at about 11pm Thursday evening.

During setup of the event Friday morning, I got to learn about how the networking was configured. Cheesy Arena was running on a small Ubuntu Trusty machine, and was connected to a switch that all the driver stations were plugged into. There was another switch that provided internet access for pushing data to The Blue Alliance. All the laptops used for match controlling and scoring were connected to Cheesy Arena over ethernet cables, it was only accessible over WiFi for the FTA tablets that showed which robots had connected to the FMS. All of the critical networking equipment and Cheesy Arena server were on a dedicated UPS in case someone unplugged the field (it has happened in the past), while matches were running.

And...the event was mostly uneventful! All of the issues with robots not connecting properly were some kind of issue with the robot or an improperly configured driver station. Mostly it was people FORGETTING TO TURN ON THEIR ROBOTS. Really, how hard is it??

When it came time for eliminations, the backend code worked flawlessly, and scheduled all the matches properly. We did run into a really silly bug wih the rankings display bug though. When writing it, I was under the assumption that we would be using reasonably recent versions of Chromium, so I used the newly introduced String.prototype.startsWith function, because it made the code easier to read since it is the same as Python. Unfortunately the program that was managing the overlays and video stream was using an embedded Chromium 37, which didn't contain this function, so it would throw JavaScript errors. Quickly switching to String.prototype.indexOf. Once that was fixed, everything worked perfectly.

In the end, we only had one field fault in the very last finals match :(, which was due to an overload on the network, resulting in a "christmas tree" effect with robots flickering from green to red and back as they kept losing connections.

Overall, I had a lot of fun learning go. My experience with it was that it was very very hard to write crashy code in go. Whether it was trivial typos or glaring type errors, go build would refuse to complete if I had an issue in my code. I was only able to get go to crash once in all of my development, and that was due to a database level error that I suppressed. I'm not a huge fan of the go error handling model, but it very heavily enforces the "explicit is better than implicit" philosophy I love from Python.

I used "we" a lot above, none of this would have been possible without help from Alex M., Clayton, Mr. Mitchell, Lee, Novia, wctaiwan, Yulli, and all the other volunteers. I hope to be back next year!

Also, new laptop sticker ^.^