Deadlock: Dead simple encryption

It’s been over a year! I have written the occasional blogpost on indiebiotech.com in that time, but even that blog suffers. If I’m honest, and to provide flavour for the rest of this article, every time I was sitting at my keyboard and might otherwise have been motivated to write a post on something, I wrote programs instead. Why? Because, as a friend cautioned me once, “programming is like crack to a problem-solving mind”. Writing can be powerful, or simply cathartic, but it’s thrilling to create something and see it work.

Most recently, I wrote a piece of software which implemented the protocol of minilock.io, a chrome plugin by the maker of Crypto.cat which provides secure file encryption for sending to others across the internet. I called my version deadlock, and it’s available here, here or by typing (in Linux with a recent version of Python installed) “sudo pip3 install deadlock”.

deadlock's icon

Why is something like this important? Allow me to frame it like this; if you want to send something privately, you could try making a zip-file with a password, sending the password to the recipient through a secure channel (what secure channel?) and then sending the file. But there are so many holes in that scheme; how do you get the passphrase to your friend securely, if you’re worried about sending files securely? Surely someone big enough to be listening on one channel (your internet connection) should be assumed to spy on the others (your phone)? Is it important that the file-list of encrypted zip-files is still visible to anyone?

Encryption of files to recipients in a secure way that does not rely on any trusted channels is actually a solved problem; so-called asymmetric cryptography has been around for a long time, and free, trustworthy implementations of these systems are now decades old. The chief problem for the lay person is that such schemes have been implemented for technical users who understand the threats and the solutions they face at a deep level; when attempted by non-technical users, these systems frequently fail badly and leave users open to observation by dangerous adversaries (fascist governments, overweening employers, etc.).

This “user experience” (UX) problem has plagued well-known systems like PGP to the point that many privacy advocates, myself included, will not recommend the use of PGP to journalists, solicitors, whistleblowers or human rights advocates, let alone friends and family. Something designed for the non-technical which provides no-frills, sensible-defaults asymmetric encryption has been long in coming.

These days, post-Snowden etcetera, privacy is becoming chic at last. Sadly, most of the new privacy platforms emerging are complete snake-oil; they are usually closed-source (which means the programmer has something to hide from you, e.g. it is ineffective at best, outright spyware at worst), their protocol specifications are missing, poorly documented or open but worryingly ignorant, or they implicitly trust the programmers or providers to protect you (such as “private email servers” in nations that routinely imprison people for refusing to invade the rights of others).

There are a few good systems, and one of the ones I’ve taken an interest in is miniLock.io. miniLock is a plugin for Chrome written entirely in Javascript. When run, it prompts the user for an email address and a secure passphrase (it will helpfully suggest high-security passphrases if you lack inspiration), and uses these to generate a miniLock “ID”; a string of ~45 random-seeming characters which can be used by others to send securely encrypted files to you.

The magic of asymmetric encryption means that you can safely post your ID anywhere without fear; the ID is *only* useful for encrypting files to you, and cannot be used to decrypt files. Only you, with your secure passphrase, can decrypt files send to your ID.

And, after generating this ID, miniLock offers a friendly interface to do just that; to encrypt files to others, and to decrypt files sent to you. You can encrypt to more than one person at once, so multi-party communication and file-sharing is practical using miniLock.

However, as impressive as miniLock is, its indelible tie to Chrome was too limiting for me. For starters, I don’t use Chrome or recommend it to others; the default settings amount to spyware anyway (everything you visit or see is sent to Google), so basing security software on top seems counterproductive. Also, as a plugin, miniLock has a great interface but is poorly accessible to other software, so it can’t easily be used to extend other parts of my computer experience. I think miniLock could be interesting as a preprocessor for sending and receiving email, or as a way to secure stuff shared through “cloud” folders like Dropbox (sorry, Condoleeza Rice!), but miniLock can’t be those things as a Chrome plugin.

So, I decided to write a new client for miniLock, in my favourite language; Python! Python 3 is a modern, cross-platform, flexible, rapid-to-write and easy to maintain language with huge library support. It’s perfect for applications like this, and it can be written into a text-only application (easily looped into email or dropbox, for example) or as a graphical user interface like the chrome plugin provides.

I won’t bore the reader with the intricate details of the process. Suffice to say that, because Python is a well-established and well-loved language, there were already implementations of the component algorithms and functions I needed; BLAKE2, Scrypt and NaCl. There was a Python 2 version of the password-assessment routine used in miniLock, too, so I decided to port it to modern Python and include it, too. Combining these into what would become deadlock, my Python implementation of miniLock, then took only a few days of off-and-on work.

The result is deadlock, and is considerably less user-friendly than miniLock. User-friendliness is already serviced quite well at this point by miniLock, my immediate goal was instead to create a Python module and terminal application that I and others could experiment with easily. deadlock can be installed on any system with a modern distribution of Python (that is, version 3.2 or greater, with the pip package manager) which has a C compiler for the core algorithms, by simply issuing (on a Debian-like flavour of Linux) `sudo pip3 install deadlock`.

Once installed, deadlock is available as a Python module (though bear in mind the API is not frozen and I may change public functions at this point without warning) and a terminal script by the same name. The script allows you to encrypt and decrypt files, prompting you for an email and passphrase each time and encrypting to you plus an arbitrary number of recipients.

For example, to encrypt a file to the user ID “JjmYYngs7akLZUjkvFkuYdsZ3PyPHSZRBKNm6qTYKZfAM” (that’s me!), you would type:

deadlock encrypt “sillycatpicture.jpg” JjmYYngs7akLZUjkvFkuYdsZ3PyPHSZRBKNm6qTYKZfAM

This will prompt you for your email and passphrase, use them to generate your ID on-the-fly (it is not stored in normal usage, as with miniLock), and encrypt the file to you and I as a new file with a random filename ending in “.minilock”.

Either of us (by default you are also a recipient on stuff you encrypt, you see) can later decrypt the file by issuing (assuming the random name is “66fc4601b498.minilock”):

deadlock decrypt 66fc4601b498.minilock

..which will again prompt you for an email and password, and will decrypt the file and save it to “sillycatpicture.jpg”. Note that thanks to the way miniLock’s protocol is written, even the filename is not possible to obtain unless the file is encrypted to you, so nobody knows you’re only sending me silly cat pictures!

deadlock includes a few features missing (in some cases by design) from miniLock, including a local address-store which would, for example, allow you to substitute “cathal” for “JjmYYngs7akLZUjkvFkuYdsZ3PyPHSZRBKNm6qTYKZfAM“, a private-ID store which, though lacking entirely in any measure of security, allows trivial encryption and decryption without re-entering passphrases, and auto-zipping of directories when encrypting, allowing you to encrypt a folder without preparation.

Depending on your needs, this is useful or superfluous. Right now, if you want user-friendly encryption, use minilock.io. However, I have high hopes that I and others can use deadlock to integrate this type of encryption into other sorts of activity, like aforementioned dropbox preprocessing, email sending and receiving, etcetera; so hopefully deadlock will be of use to you someday soon, as part of something more user-friendly. If you’re a hacker or a technical user, I think deadlock will speak for itself as a terminal application and Python module.

Enjoy!

Been Busy!

It’s been ages since I’ve blogged. I guess now is a nice time to get back on track, both here and on indiebiotech.com. I have been far from idle! However, much of my work has been biotech-related, which belongs yonder on indiebiotech, so I’ll just share some of my non-biotech pet projects here for now.

TinyStatus

In response to ongoing attempts to regulate “Unfettered Commentary” in Ireland, I wrote a peer-to-peer microstatus server/client in 30 lines of Python, in the hopes of making a similar point to the 15-line TinyP2P app written in response to P2P regulation attempts in 2004. That little gem is included in the Repo because the original post is no longer online except on Archive.org. (My extra lines are spent on local database management and try/except functions to prevent trivial network crashing)

The code is over on Gitorious/Github, if you’re interested. I sometimes host a test server and announce on Twitter when I do, but to date nobody has ever posted. One guy ran a curl script to “DDOS” me with nonsense HTTP GET requests for laughs, which is how the internet works and gave me a giggle.

Still, it was very instructive learning how to compress code in that way, particularly in a language that really doesn’t want you to make ugly code; Python’s philosophy includes “Explicit is better than Implicit”, so ugly oneliners are discouraged. However, Python also doesn’t stop you from doing things you want to do, so it ended up working well in the end. Of course, it was a PITA to debug, because when you use mostly lambda functions, the error messages when things go wrong are inscrutable. To get it working, I had to refactor the code into normal multi-line functions first, and then individually revert each to ugly one-liners. Finding ways to crunch code was helped in part by this thread on Stackoverflow, but moreso by decoding TinyP2P and figuring out how it worked. The annotated version of TinyP2P I made is included in the repo, too.

When I have more time to waste on this, I plan to rewrite the whole thing in explicit and more-efficient “normal” Python. There’s plenty that could be improved if brevity isn’t the aim, after all; it’s exceptionally inefficient right now.

Of course, it’s no replacement for a “real” microstatus system yet, because there’s no form of user authentication to prevent anyone from posting in the name of another; for that to work in a P2P model, there’d need to be a form of asymmetric key authentication coupled with efficient key exchange to rapidly establish a “globally unique” key:name mapping, which implies an efficient distributed datastore; a DHT. As the Python standard library contains neither asymmetric encryption (why?) nor a DHT implementation, this wasn’t an option, and in any case it’d have added another 10 lines even in a perfect world.

A DHT-based asymmetric key Microstatus client is something I’d like to see written, though, and perhaps I’ll waste some time on that someday when there’s enough to spend. I’d like to learn more about DHTs by hacking one up; there aren’t any implementations in Python 3 that I can find so far.

Leaving Google

My efforts to leave Google have stepped up, thanks to my establishing a stable email address to use elsewhere.

After dickering around with self-hosting and discovering that there are almost no turn-key open-source email systems in existence that wouldn’t be a pain to debug, and after deciding that reliability was a big concern in Ireland (land of the blackouts), and after discovering how horrid the anti-spam environment is on small self-hosters, I gave up on that idea entirely.

So, I turned to the hosts of my two blogs, 1984hosting.com to host my email for me, and initially it didn’t work; their help docs were technically incorrect and the port numbers I was trying wouldn’t (at the time) work with encryption, so I gave up and waited. A few months later though, I was able to set up IMAP and SMTP with encryption on mail.1984.is, their mail subdomain (also their webmail domain for on-the-go access), and started using my new address at cathalgarvey.me.

I’m after redirecting all my subscriptions and changing all my big accounts to the new address, and today I’m setting an autoreply to redirect people using my old address. It’s remarkably easy if you’re patient: just forward mail from the old address and change subscriptions you’ve forgotten about as they arrive. Set your signature to include your new address for a while, then set an autoreply when most or all of your frequent mailers have already switched.

If you want to change to Icelandic hosting, too, just register a domain for yourself, sign up for 1984hosting’s cheapest plan, point the domain to 1984hosting.com, set up an email address in the 1984 control panel, and use these settings in your email client (Claws, Thunderbird, whatever):

IMAP Server: mail.1984.is
IMAP Port: 993
IMAP Auth: SSL/TLS with normal password authentication
SMTP Server: mail.1984.is
SMTP Port: 587
SMTP Auth: STARTTLS with normal password authentication.
Username: full email address (you@yourdomain.whatever)
Password: your -email address- password

The Great MP3 Purge

MP3s suck. They used to be the hotness of the music world, because they were probably the best music compression format available for a while, but they are no longer up to standard. Not only is the compression bad in comparison to other music formats such as OGG Vorbis, but the quality to compression ratio in listener tests is poorer, so they just don’t deliver much for the space saving they offer.

Finally and most insultingly to me, they operate on a patented codec which cannot be decoded without a “licensed” decoder, meaning that if I want to write software that plays MP3s, I have to go beg for a license and fork out huge money; that is a big burden on the open source community’s efforts to create rich media experiences for the world.

You hit this roadblock when you install a fresh Linux distribution on your computer; unless you check a box saying “Install restricted extras” (in so doing you are saying “I have a license or I live somewhere where software patents haven’t insinuated themselves yet”), you can’t play mp3 or h264 media files because both are patented codecs. It’s trivial to ignore and just click the “Install restricted extras” box, but it’s a small burden on the soul that you’re capitulating to a system that, broadly speaking, is used to crush innovation.

So, I elected this year to do two things as New-Year’s-Resolutions: firstly, to purge my biggish music library of MP3s and replace them with better, smaller, freer music files, and secondly to never again buy music from Artists licensed by big media companies who are crushing free speech and thought in Ireland. Sorry Artists; you guys may not be directly to blame, but buying your music supports evil, evil people who are making my life less free.

Both goals are possible in 2013. It’s no longer necessary to have MP3s unless you’re an Apple user (in which case you’ve dug your dirty technophobic grave and can go lie in it), because Android plays FLAC and OGG Vorbis out of the box, as does Linux (although codecs for Windows Media Player and other Windows media software for both should be trivial to install if you’re a Windows user). Also, music distribution is finally moving on from RIAA et al; Bandcamp offers a very slick way to download music in any format of your choosing including FLAC/OGG without paying any money to evil middlemen, and more and more artists are either self-hosting or sharing on bittorrent trackers. For a while, there was a nice service called riaaradar.com for checking artists for affiliation with the RIAA and its global cabal, but that appears to be down.

Of course, just because it’s trivial never to buy another mp3 or RIAA track, it doesn’t follow that it’s easy to backtrack and remove legacy mp3s. It’s an ongoing project. I’m complicating things for myself by refusing to re-purchase albums that I’ve long since lost so I can re-encode them, and I’ve had to dip into second-hand CD sales on Ebay to track down music I value and want to convert. But on the whole, it’s simply time-consuming. I’d talk more about the process, but normal day-to-day culturally important activities are sometimes illegal. I’m sure my good reader knows many channels of distribution through which one might acquire copies of music one has already legally purchased.

To help me focus on replacing artists whose music I have a lot of, I write a quick python script that, given a directory and a filetype, recurses into the subdirectories, counts all the files whose filename contains the search pattern (optionally restricted to file extensions only) and reports back with a list of top-level folders and the number of files it found in each. So, calling the program as “fncensus -x ~/Music “mp3″” will return a list of artists who need replacing. If you’re interested, it’s hosted here on Github.

 Enough Updates

I’ve spammed enough for now; no longer a guilty blogger. More as it arises; I have a number of fun projects I’d like to pursue soon, including building a Data Furnace to supplement our home heating (and mine bitcoins or crunch rainbow tables or something), if I can afford the initial hardware outlay.

Yes, Indiebiotech is Down.

It’s weeks late, but if you’re looking for Indiebiotech.com, or for that matter my old blog domain cunningprojects.com, they are both inactive right now.

Well, not inactive. Rather, fruitlessly pointing at random IP addresses rather than the correct server. This is something that I’m trying to address, and failing to address.

Oh yea, and months without an update.. there’s loads I’m planning to post here STOP I’ve got a lot of news, probably too much to digest in a single post STOP The hardest part is logging in. Expect more spam STOP

Hello again, world!

I’ve migrated my old blog from cunningprojects.com to cathalgarvey.me. It’s not entirely a matter of vanity, although I do rather like having my name on top: It’s part of a broader migration of my online stuff from the US to Iceland.

The reasons for this are chiefly that Iceland is one of the world’s best countries in terms of free speech and free access to information. In fact, trends in Iceland suggest that more free speech may be on the way, not less; that’s bucking a scary global trend towards censorship and surveillance. Given that my new hosting providers are named 1984 Hosting, I’m pretty confident that they value the same things as me in this regard!

I did love my old webhosts at ixwebhosting, and I’d recommend them to anyone who wants a US webhost. Thanks IX!

The Shortlist of Free-Speech Software For Our Newly Censored Ireland

So, our great and glorious Minister Sean Sherlock just signed SOPA into law in Ireland, despite a huge civil outcry. The poorly defined statutory instrument will allow anyone claiming “Copyright Infringement” to seek a court injunction against any website, without having to present evidence and without a consultation with the accused website. The form of the resulting censorship is unclear, but will probably require ISP-level DNS censorship of websites outside Ireland, or direct seizing of those within the Irish jurisdiction.

This is stupid, unfair and myopic (and it won’t work), but it’s not the end for freedom of expression in Ireland. However, the fact that IRMA and others can now arbitrarily demand removal of your blog, youtube video, tweets or server simply by claiming that they suspect copyright infringement (the easiest faked allegation ever devised) means that your freedom of expression will need to be more sophisticated than before.

Thankfully, there are systems enabling free, uncensored speech and content discovery already available at zero cost. While I’ve committed to providing workshops on censorship and surveillance circumvention very soon under the umbrella of Nexus Cork (our local Hackerspace), in the meantime, here’s my quick shortlist:

Web Browsing and Publishing

Tor. The ultimate in current anti-censorship technology, Tor uses an “Onion Routing” system (for which it is named) and layered encryption to route Internet traffic so that it is virtually impossible for even extremely well positioned censors (read: far more powerful than the Irish state or IRMA) to prevent the user from reaching his or her destination online, or to see what that destination is. It is *not* suitable for Bittorrent downloads, but for traversing the Internet freely without censorship or effective surveillance, the Tor Browser is the easiest and most effective tool available.

Tor Hidden Services. A continuation of the above; the Tor network can be used not only to find and view content without censorship or surveillance, but to host uncensorable websites that are all but impossible to locate, provided the sites use secure software in their construction. A server hosting a “Hidden Service” can be reached only through the Tor network (using the Tor Browser, for example), using a unique “.onion” web address that looks like random text. Hosting in this way is not only uncensorable and impossible to locate, it’s free; or as free as your ISP’s up/down bandwidth caps, anyway. Hosting your site somewhere in the cloud is advisable in any case, preferably somewhere where free speech is still considered important.

Communication Privacy

While SOPA Ireland only deals with censorship, it’s inevitable that if Sherlock wants to serve IRMA fully he’ll have to progress to surveillance of daily communications. After all, with the Internet under IRMA’s thumbs, those darned-tootin’ ubiquitous pirates will just resort to far-more-efficient and hard to detect “Hard-drive parties” instead, where friends gather and share gigabytes of data at a time in one another’s homes.

Recall that Email and SMS are both relayed between sender and recipient as plain text normally. People tend to regard email as being like a letter, enveloped and safe from casually prying eyes, but this is not so. Intermediate servers, bored or malicious employees, or overreaching corporations or law enforcement can easily read these communications, pilfering passwords, credit card details, or just private and personal information.

In order to prepare for surveillance either by the government for IRMA, or by IRMA directly (backed by another Sherlock*), encryption of personal communications is a good idea. Thankfully, it’s trivial for SMS messages on Android at least, and relatively easy for Email, provided you’re willing to accept using a client to manage your daily email (you don’t have to sacrifice webmail as a convenience, but you won’t be able to use it to handle encrypted email, because Gmail/Yahoo/MS et al don’t use encryption. How would they read your email if they did, silly?).

PGP (“Pretty Good Privacy”) is the world-class encryption method used to protect email and other critical data. It is a form of “Asymmetric Encryption”, meaning that data is encrypted using one key, and can only be decrypted with another key. Therefore, each user is expected to have a “Key Pair” consisting of a public key, which is shared as widely as possible, and a private key which is kept completely private. Friends/Family/CoWorkers/CoHackers can then email the user privately by using the public key to encrypt the email, so that only the user can decrypt it using the private key. PGP is installed by default on Linux as “GnuPG“, an open-source implementation of PGP from the GNU foundation.

Thunderbird is the premier open-source email client. On its own, it does not provide encryption, but a free plugin called “Enigmail” enables one to easily set up and use PGP encryption for any email account, whether webmail or personally hosted. Enigmail can be installed from within Thunderbird by searching for it in the addons section. Enigmail works by allowing Thunderbird to use GnuPG or PGP, which must be installed on the system already: if you aren’t using a breed of Linux, you’ll need to download and install GnuPG.

APG and K9 Mail – Both Open-Source apps downloadable from the Android Market, APG brings PGP encryption to Android, and K9 Mail natively supports APG encryption and decryption. K9 also happens to be a fantastic mail client, far more granular and customisable than the default mail client or the Gmail app. This can be a problem if you make settings changes you don’t fully understand though, so sticking with default settings might be an idea at first. APG must be installed first, K9 second. If you forget, you can always uninstall and reinstall K9 to get things working well.

Textsecure is an Android SMS application (available in the Android Market) that acts as a drop-in replacement for the default SMS app. In fact, you can even delete the native Android SMS client (mms.apk) using ADB if you’ve got the technical skill, and Textsecure will work fine without it. Textsecure enables local and end-to-end encryption. The former means your SMS history (which can be imported on installation from the old SMS app) is protected by a password and fully encrypted from snooping eyes. The latter means that two users with Textsecure can set up an encrypted session, such that all text sent between them are entirely concealed from the prying eyes of intermediaries, whether network employees, IRMA or the like. The disadvantage is that session setup can be bug-prone and may require several tries/aborts before it works (but it lasts once established), and that letters-per-SMS drops to 60 because of the formatting overheads of sending encrypted text. This is seldom a problem in this age of “Free SMS to all networks” offers, of course.

File and Disk Encryption

If and when Sean and IRMA come calling to peruse your private life in person, or Sean’s future “Stop and Frisk for Data” plans come to fruition, you may want your data to be indecipherable. For Linux breeds like Ubuntu, encryption of your home folder is an option on installation, and means that the parts of the system on which you keep most of your private information are all encrypted securely. It’s not perfect unless the whole disk is encrypted, but home-folder encryption is a great start.

For external drives, Linux supports encryption of disks as an option for ext2/3/4 file systems; when formatting a hard drive under (for example) gParted in Ubuntu Linux, you can choose for the disk to be encrypted and password protected, at the cost of it being incompatible with Windows and probably Mac, which can’t handle ext file systems (although you could put a little universally-recognised partition on the drive containing software that would allow you to use the main partition with the other systems should the need arise..).

More practical for cross-platform interaction between computer users is Truecrypt, the last software to get a mention here. Truecrypt allows you to create encrypted “containers”; essentially virtual drives in the form of a file, which appears to be completely random binary data unless opened correctly in Truecrypt with the correct passphrase. Once opened correctly, Truecrypt containers appear as virtual disks on the computer which can be written to or copied from. Truecrypt supports a dizzying depth of hard encryption and plausible deniability; is it any wonder that it was used by Julian Assange to protect Wikileaks’ data on the move?

So, for mobile drives, format your harddrive using a commonly accepted format like Fat32, and create a giant Truecrypt container on the drive. Remember to include installers for Truecrypt on every platform you’ll need on the drive, so you can open the container when needed.

In Summary

With Local encryption using Truecrypt, a trustworthy computer system such as Linux, a freshly installed, trustworthy custom Android ROM such as cyanogenmod and the software above, nobody will be able to stop you from browsing at will, hosting at will, and communicating at will. Freedom of speech, assembly, expression and belief, restored through open source software.

Share the software and the knowhow. I’ll be hosting workshops soon, when I can spare time to prepare. Share this document; consider it Creative Commons Attribution-only, giving you the right to copy, modify, excerpt or even sell it, provided you give me attribution for my role in writing the original document. Just link back here with my name if you do, thanks.

Go forth. Share!

* Sherlock, n: An act enabling massive disruption of civil rights to satisfy narrow commercial interests.

Bacteriophage T7 Model

As solicited over twitter by TheFrogBlog, I designed a T7 Bacteriophage model for 3D printing via Shapeways. And, here it is:

An SLS-printed model of Bacteriophage T7, the model "Temperate Phage" for E.coli. Shown next to a 10€c piece for scale.

Bacteriophage T7

It’s a little fragile at the tips of the legs, so I might increase the size a little to make it more robust. Also, the neck is a little weak where the head joins the body, so I may have to lower the head a little to strengthen that connection.

Still, turned out really nicely; I’m delighted to have another showy molecular biology toy for my desk!

Bacteriophages like T7 are the viruses of bacteria. They are made almost entirely of protein, and they infect bacteria by attaching to the outside membrane of the cell and injecting the DNA payload in their “head” into the cell. They were used for years to help understand how bacterial genomes worked, by “tricking” the virus into carrying other bits of DNA instead and injecting that DNA into different cells to see what happened. The process was/is called “transduction”, and is recognised as one of many ways that bacteria can collect DNA from unrelated species, perhaps assisting in the spread of antibiotic resistance, or adaptation to new habitats. These days, we have better tools for studying bacteria..but they don’t look as awesome.

Bacteriophages have also been used (and continue to attract attention and investigation) as antibiotic agents; because bacteriophages can undergo evolution to adapt to their hosts, they can in theory provide a resistance-free way of treating bacterial infections. However, they tend to have a really narrow host-range, which limits their usefulness. Still, with synthetic biology, we may be able to engineer more useful “smart viruses” that can infect and kill only the dangerous strains of otherwise harmless bugs, possibly offering us a new way to treat sick tummies that doesn’t wipe out all those important gut bacteria in the process.

You can get one here.

Kindle Touch Hax #1: Personalised USB-connect Screen

Behold, my new Kindle Touch, an extremely kind gift from my family to me:

A picture of a Kindle Touch showing a watermark under the usual "USB Drive Mode" display. The watermark warns that the device is the property of Cathal Garvey and was not sold or given, and asks the reader to return it via contact details given.

A watermark that the average thief probably won't know how to remove. I probably won't get my Kindle back if it's stolen, but at least I'll be inconveniencing the thief..

But what is this? At the bottom of the screen, there’s a message declaring my ownership! That’s not normal for Kindle Touches. It’s a little trick I’ve pulled off thanks to Yifan Lu’s awesome work towards Jailbreaking the Kindle Touch.

Essentially, Lu discovered that the Kindle executes native code embedded in the metadata of mp3 files, and used this fact to install a developer’s key and a basic SSH server on the Kindle Touch. His hack allows you to log into what is basically a small linux device and change the system at will.

If you want a jailbroken Kindle Touch, simply follow Lu’s instructions; download the mp3, and then play it using the mp3 player found under the “Experimental” section of the Kindle Touch menu. Playing the mp3 will install the jailbreak, SSH, and remove the mp3. From there, you have all the power in the world to improve, modify or ruin your Kindle using SSH to login as “root”, the super-user at the core of every Linux distribution.

To merely create an ownership notice of your own, follow the enumerated instructions below on a Jailbroken Kindle Touch. I could make these instructions far smaller by getting the relevant file for you, but then you wouldn’t be learning the how and why of SSH, would you? ;) Perhaps someday I’ll repackage this as a friendly mp3 file or shell script you can execute mindlessly, but for now I have more exploring/modding to do..

  1. Prepare a password for SSH by tapping the search bar on the main screen and typing (without quotes) “;un password PASSWORD“, where “PASSWORD” is the password you want. i.e. if you want your password to be “SunshineBananasWensleydale” then you should type “;un password SunshineBananasWensleydale
  2. Enable usbnetwork on your Kindle by tapping the search bar on the main screen and typing (without the quotes) “;un
  3. Using a linux computer (use an Ubuntu livecd if you use another system), plug the kindle into the USB drive. With usbnetwork enabled, the kindle should appear as an automatic network connection*.
  4. Open up terminal and type “ssh root@192.168.15.244” . When asked if you trust the server/device, type “yes” or whatever it suggests to accept. When prompted for a password, provide the password you set in step 1.
  5. You will be logged in as “root” in an empty folder. For rewrite access, you will need to type “mntroot rw“; do this now, and be careful what you type afterwards or you may brick your device (worst case scenario, but possible).
  6. The USB-connected image is located in /usr/share/blanket/usb/, and it is called “bg_xsmall_usbconnect.png“. The part of the kindle that you can access freely by USB (where you load books/music etc.) is at /mnt/base-us/. So, to get a copy of the file you can work with, type (without quotes): “cp /usr/share/blanket/usb/bg_xsmall_usbconnect.png /mnt/base-us/bg_xsmall_usbconnect.png
  7. This has copied the “USB connected” screen to the folder you see when you mount the kindle for document loading/removal. So, to access this with an image editor, type “exit” to close the SSH session, unplug the kindle, and in the search bar at the main screen type “;un” to disable usb networking.
  8. Now that usb networking is disabled, you can plug the Kindle back into the USB drive again and it should appear as a drive as it normally does. There in the root directory should be the “bg_xsmall_usbconnect.png” image.
  9. Edit this file using an image editor, but bear in mind the following:
    1. Do not change the resolution
    2. Only use black and white
    3. Some text and a battery icon is displayed by the kindle; keep your text at the bottom, and keep it small. You have about an eighth of the screen to work with.
  10. When the image is ready, save it under the same name, and dismount/safely remove and unplug the kindle.
  11. Re-enable usb networking by typing “;un” into the search bar in the main screen, then plug back into the linux PC.
  12. Re-connect via SSH as in step 4, and remount the file system as writable as in step 5.
  13. Back up the original file by typing “mv /usr/share/blanket/usb/bg_xsmall_usbconnect.png /usr/share/blanket/usb/backup_bg_xsmall_usbconnect.png”
  14. Copy over the new file by typing “cp /mnt/base-us/bg_xsmall_usbconnect.png /usr/share/blanket/usb/bg_xsmall_usbconnect.png”
  15. Type “exit” to close the SSH session, unplug, type “;un” in the search bar at the main screen to disable usb networking, and plug back in. When the screen for “USB Drive Mode” appears, your new image should appear!

*Alternative networking route: If you can’t get the USB connection to work, USBnetwork also enables WiFi login for SSH. However, to get the IP address for your Kindle, you’ll need to consult the client list on your home wifi router and compare the MAC addresses of the clients connected to the MAC address of your Kindle, accessible from Menu->Settings->Menu(Again)->Device Info. Then connect to “root@www.xxx.yyy.zzz”, substituting the IP address for wxyz.

Leaving Google Behind: Progress Report

Google, I’m Leaving You.

Somewhere over five years ago, I gratefully accepted an invite to Gmail and rejoiced: it was a wonderful new paradigm in web-based email, and a huge improvement over Yahoo Mail. It’s still one of the best email services online, and still miles ahead of the nearest competition by number of users.

At the time, it was a straightforward social contract; Google would host and provide a great email service, and in exchange, non-human agents (robots!) would scan email in real-time for keywords, and provide ads in real time based on their inferences. This, I thought, and still feel, is pretty fair for such a great free service.

Somewhere along the line, the contract was compromised in innumerable ways. Firstly (but not by importance to me) it seems the “in real time” part is gone. That is, the comfort of knowing (or thinking) that results of algorithmic scanning were not stored or logged, is now gone. It’s generally accepted that Gmail is part of a greater profile-building apparatus built into the google account suite, and as such some content of my private life is entering the public sphere and being sold or revealed to people I don’t know or trust.

More importantly perhaps than Google’s slow abandonment of its “don’t be evil” mantra is the increasing invasiveness of the American Government’s “Be as Evil as Possible” policy. Google provides largely unfettered access to user data and accounts to the various gestapo agencies of the US intelligence and law enforcement apparatus, who form their own profiles on people. There is a mountain of evidence that due process is often ignored and there is more often than not no legally relevant reason behind invasions of this sort; anywhere from casual curiousity to “watch this dissident” reasonings can be applied under the PATRIOT act and its cousins, when the law is invoked at all. Worst of all, Google don’t notify account holders of these invasions even when they are legally capable of doing so.

I don’t know about you, but I am not too happy about having faceless agents from the world’s biggest kidnapping agency reading through my email. It’s not a matter of “I’ve got something to hide”, the most tired straw-man in the privacy-hostile person’s arsenal. If you ask someone whether they’d happily omit the envelope on their snail-mail, even if there’s nothing illegal inside, most people might balk; why let all the guys in between read my soppy I-love-you-mum letter? And yet that’s what we routinely do these days with email and social networks.

Count me out. There’s no reason why I can’t enjoy all the fruits of modern internetting without sacrificing a bit of myself to the police state.

So, I’m making a transition away from Google and toward personal email hosting. It’s going to be an interesting experiment, and I’m not going to dive into the deep end immediately with something so important. The first step is getting all my data from Google so I can safely archive it; that’s several gigabytes of email and attachments, so it’s taking a while. Here’s how it’s going so far.

Leaving Gmail with Archives Intact

So far, getting my email has been the hard part. After the continuation of the infamous “nymwars” debacle on Google+, I decided to ditch that service; at least with “Google Takeout” it was easy to back up all the content I’d put up on that service before hollowing out the profile.

However, it’s hard to be sure that suspending Google+ won’t cripple or ruin the rest of the account; after all, “name violations” on Google+ have lead to people losing access to their entire Gmail account, and the “Delete my profile” apparatus doesn’t make it clear or certain that my general account will be spared.

Unfortunately my Email is sort of a personal archive or cloud-storage thing for me, so backing it up is important but also awkward. I decided to go down a trustworthy hacker-friendly command-line route, because I’m a nerd like that, but I’m starting out with the easiest solution: Thunderbird. Using the Mozilla Foundation’s Open-Source email client, I’m downloading all of my email and using the filtering system in Thunderbird to apply yearly archiving tags to my email. Oddly enough, I’m doing this because the built-in search engine in Gmail seems to be broken (of all companies to botch a search feature..) and won’t let me search/label by date no matter which format I use.

Once I have all of my email reliably labelled by year, I’ll be using “Getmail” to download the email year-by-year. Getmail allows you to save email either as a “maildir” (a set of folders full of individual files for each email) or as a giant file containing everything. I’ll be going with the former. There’s a great writeup on how to use getmail: be sure to read the whole article and the comments if you’re patient enough, because there’s lots of pro-tips and debugging stuff there.

One odd pitfall I hit was in Contact Export/Import: Gmail can export all contacts as a “Comma Separated Values” file, which is great. However, three things happen when you try to import to Thunderbird:

  1. Not all of Thunderbird’s potential fields match the output (Thunderbird has no “Middle Name” field, for example, while Gmail uses it liberally), leaving you with a soup of potential assignments of key data, few of which are perfect.
  2. The inteface to actually match value-to-value is awful; one list can have items shuffled, but because items shove each other down the list as they are moved you can only reasonably do this from the top-down of the other column. As mentioned above, not all potential fields match, and there are oodles of redundant fields, forcing you to “plug” gaps (that is, stupid fields in between fields you actually want to import) with matching fields that you’re not going to use.
  3. When you actually import contacts, all name information is (if matched correctly) neatly stored in each contact, and then ignored when it comes to providing an actual name in the contacts list. Instead, the contacts window just axes off everything after the “@” symbol in the email provided, and uses that as a name. Mind-numbing stupidity.

To remedy this stupidity, I opened the .csv file in LibreOffice and moved around data that couldn’t import correctly (I merged “middle name” into either First or Last name as appropriate, which was labour intensive), deleted all empty columns, moved miscellaneous data into “notes” column, and finally I copied the “First Name” column twice; the two copies were named “Nickname” and “Display Name”, and were imported to Thunderbird as same. Since Thunderbird allows you to display “nickname” and sort by that, I was able to display at least the first names of everyone in the Contacts list. Victory! Remember to save that hacked .csv file so you can import it into other instances of Thunderbird or similar at a later date.

Once I’ve got all my email and all my attachments safely downloaded, I’ll be purging my entire account up until the last few months, and that’ll be “stage 1″ complete in my mind. I’m planning to archive all of the past email data in a Truecrypt file which I can keep safe by redundancy (i.e. copying to CDs etc) without worrying about it falling into snooping hands.

When I get my next Email set up and running, I’ll set up a Gmail redirect and autoreply to inform people of the switch, and begin the migration. People imagine email migration to be extremely difficult, but I’ve done it a few times; in reality, most of the people who actually matter will email you at least once a season, and they’ll quickly change the email they use when they get autoreply’d a few times.

What Then?

Leaving Google might seem a drastic move.. indeed, I’m not actually planning to delete the entire account. After all, the Android Marketplace regrettably requires a google account, and Google Wallet is pretty handy too. For viewing shared documents on Google Docs I’ll need an account too. However, Google will no longer be a central part of my internet experience.

Indeed, I’m generally going to be trying to keep my online behaviour for now on as close to the chest (i.e. Not In America) as I can without making compromises on my mobility and user power. With the amazing software that’s available in the Open Source sphere, I can start hosting a lot of the sort of services I used to rely on Google or similar for, using my own hardware.

Search: To avoid search bubbling and search tracking, I’ll be switching to the far richer and more user-friendly DuckDuckGo.com. More broadly, I’ve lately been thinking that the death of links-pages and webrings was a dangerous dependence-inducing mistake for online culture, but that discussion is for another blog post. There are hints of croudsourced-webcrawling search engines in the works here and there, which would be very interesting if true; yet another potentialy application of idle processor time for net users worldwide would be to help aggregate a map of the internet. More interesting still, perhaps, would be to crowdsource surfing data, anonymised and aggregated from thousands to millions of users, to form a map of the web with keywords and surfing associations intact for indexing. But, that’s not my job or immediate concern as long as I can find stuff with good accuracy, minimal algorithmic interference (“Hey, you’re from Cork and you like Open Source Stuff, why don’t I just omit key results to make you happier?”) and in good time.

Hosting: As much as I love my current web-host (ixwebhosting.com – you’ll like them if you’re in the market for a personal website, I promise!), I am soon going to investigate local alternatives for Domain Name hosting and online storage space for my sites. This isn’t simply because Ix are an American company (although that figures in), it’s also because I want to upgrade to a service that gives me command-line access to a virtual machine, to host services like OwnCloud or Diaspora that need more intensive attention on the setup side of things.

Storage/Documents: I’m planning to get OwnCloud running on my own personal server and host it online through a dedicated domain name or alias of cunningprojects. OwnCloud is slated to include a document editor which might nicely replace Google Docs, already has a built-in music player, and can be synchronised with folders on my computers or Android devices to perfectly mimic the functionality of Dropbox. It’s also got a really pretty web interface, and I’ll be able to give friends and family their own accounts if they want, too. If it’s not enough for Document management, I’ll be waiting eagerly for the

Social: I’ve already moved to Diaspora*, and I invite anyone who’d like to connect with me there to do so. I can’t guarantee a follow-back, but that doesn’t mean we’re not friends; just that we don’t necessarily share online interests! When Diaspora provide functionality for account-migration, I may decide to join a local pod, perhaps one hosted at the local Hackerspace in Cork. Also, I’m staying with Twitter for now. For one thing, their Corporate Culture hasn’t soured yet, and they seem to do the right thing generally; they alert users to government prying (or did once, at any rate), they tread carefully around marketing by labelling it opaquely, etc. Main reason I’m staying with Twitter for now is simply that Twitter is for things I don’t mind shouting aloud for all to hear, so USA prying into my account is unlikely to yield anything that would bother me if revealed. I will be recommending that friends/contacts stop PMing me, however, and use email instead.

Email: The main event, as it were, is Email. Initially I will probably not be switching entirely to local email hosting on my own computer; there’s a minefield that I must become acquainted with when it comes to single-user email hosting because of the complex web of anti-spam out there. Essentially, I’m concerned that without the vouchsafing of Google or a similarly huge organisation, my email may end up filtered by default by most recipients. However, if that could be easily avoided, then I’d love to try hosting my own email server and expanding it into a rich personal service using Open Source webware. With RoundCube, I could have a pretty and reliable webmail interface, and with IMAP support I can continue to use email on my phone with trivial ease. For built-in-chat functionality, you can actually continue to use Google Chat using any chat client, and I’m certain there’s a pretty Open Source webchat client I can use, too.

“Nexus Presents: Happy World: Burma, The Dictatorship of the Absurd”, this Wednesday. Admission Free!

Dear all,
This Wednesday, Nexus Cork (Cork's quite awesome local Makerspace) will be hosting a film screening in the Camden Palace Hotel on Camden Quay.

Entry is free, and you can avail of a copy of the film if you bring a USB-capable Android or Laptop. A lower resolution form of the film is also loaded on the Dead Drop just inside the door to the building.

"Happy World: Burma, the Dictatorship of the Absurd" is a highly acclaimed and engaging documentary of the state of Burma, known to some as Myanmar. A state that has suffered crippling and often bizarre proscriptions and revisions under the rule of a Military Junta, Burma shares the dubious distinction of being the only other holdout state to use the Imperial Measurement system in 2011 with the United States. That's probably not relevant to the film, but I thought I'd share a factoid while the opportunity arose. For more amusing or incensing factoids about Burma, join us this Wednesday, and bring your friends.

Did I mention admission is free, and you can get a copy of this excellent, Creative Commons-licensed film to enjoy forever?

Posted via email from Cathal Garvey

Namecrime Exodus: I Suggest Leaving G+ by September 10th if Namecrime Remains

Dear all:
I'd like to start something called a "Namecrime Exodus"; if by September 10th Google are still forcing people to use real names, I'm leaving Google+ and deleting my account.

I strongly encourage you to post likewise and commit to leaving a defective service that doesn't understand or want to understand the freedoms and cultures of the internet. Google is a company born of and dependent upon the internet and the people who use it. In a dawning era of P2P culture and infrastructure, Google should know that they cannot afford to alienate their customers.

You can find me on Twitter @onetruecathal, so I don't see why I'd bother tolerating another social network if it means violating my principals.

Share the good news: I suggest #NameCrimeExodus as a hashtag. Poke this at people whose opinions count. I suggest +Sergey Brin and +Larry Page for starters.

Posted via email from Cathal Garvey