UPDATE 2/11/2010: Updated the included unswindle to the latest version, fixes errors on unDRMed books. UPDATE 2/27/2010: Added specification that users install the 32-bit version of python 2.6 for Windows, even on 64-bit systems. UPDATE 4/30/2010: Added link to older version of Kindle for PC

People have been asking me if there's a way to convert Kindle books downloaded with Amazon's Kindle For PC application pretty much since the day it was released. Sadly, despite a lot of tinkering on my part, there was no elegant solution^* until now.

A fantastic Kindle For PC conversion tool has been made. And it works really, really well. And it is significantly easier than the old "Kindle-only" method.

Here's how it works:

0) Install Kindle for PC. The latest version of Kindle for PC has broken the scripts, so install this older version, and make sure you disable automatic updates in Kindle for PC's settings.

1) Install 32-bit python 2.6 for Windows from here (even if your system is 64-bit you need the 32-bit version in order for this to work) : http://www.python.org/download/

2) Download unswindle.pyw and mobidedrm.py via this zip file and unzip it somewhere on your PC

3) Open the unzipped folder and double-click unswindle.pyw

Kindle For PC will now open.

4) Select the book you want to convert.

5) When the book loads, simply exit Kindle For PC

A 'Save' dialog will open asking you where you want to save your new decrypted .mobi file. And we're done.

(As it uses mobidedrm, it still won't work on Topaz/.tpz files, and the first one I tried happened to be one. Still, most books will work.)

---

*: I did work out a scheme wherein you record video of your desktop, scroll through the book in Kindle For PC, remove all duplicate frames from the resultant video and then convert to PDF, but it was rather unwieldy.

Yesterday there was much buzz on Twitter regarding "the case of the disappearing George Orwell," in which it was discovered that Amazon had deleted users' purchased copies of 1984 and Animal Farm from their Kindles, refunding them the purchase price rather than continue to let them read it. The idea that's spread through the Internet like wildfire is that this was an action at the behest of this publisher, who decided on a whim that they didn't want to publish it through Amazon any longer, and that Amazon caved and did what they asked. (This particular bit of speculation, which was repeated by even the mighty New York Times, was sourced to 'some guy on an Amazon forum.' Now THAT's journalism.)

In any case, due in large part to the irony present in a media giant secretly deleting the works of George Orwell, this story caught the ire of many people; there's the anti-DRM crowd (in which I'm a member), the anti-Amazon crowd (in which I'm often a member), as well as the pro-Kindle crowd (in which I'm sometimes a member). Several of these crowds began blogging/twittering about it, which caused a feedback loop of other people (who aren't members in any of those groups, but who blindly retweet whatever they see) and it's now being reported pretty much every where.

The problem, though, is that most of what's being spread around the Internet on this subject is misinformation. It's the type of misinformation that, when intentionally distributed by folks like Microsoft as a scare tactic, is referred to as FUD, or 'Fear, Uncertainty and Doubt.' In this particular case, the misinformation in question doesn't appear to have been designed with an intent of malice, but more by of a sense of people not really understanding what's going on.

Anyway: in a nutshell, the following is an attempt to explain what exactly it is that happened. This is not really designed to be in defense of Amazon, but more because I prefer it when the things that people believe are actually true. If they're not, I like to try to rectify it.

Firstly, very few people really have an understanding of how Amazon's Kindle Marketplace works. (Including -- until yesterday, anyway -- me.) Anyone can sell books through Amazon's Kindle Marketplace. If you're buying a book via your Kindle, there's a very real probability that Amazon has nothing to do with the book at all, other than being the means through which it is delivered to you. The people publishing Kindle books include actual publishing companies as well as individual authors, all of whom simply upload their books at which point they're for sale. There have recently been a few cases where people managed to become a Kindle seller, and then uploaded bootlegged copies of the Twilight books priced at $0.99 in hopes of getting rich quick. Obviously the douchebags in question lacked the legal authority to be selling Mrs. Meyer's masterpieces, and Amazon rightfully stepped in and shut them down. I don't think anyone had a problem with this.

Yesterday's George Orwell fiasco, on the other hand, drew the attention and outrage of the masses, and many people (myself included) quite vocally expressed their problems with it.

The thing is, though: as it turns out, the situation is EXACTLY the same as with the bootleg Twilight books.

See, in most (if not all) non-US countries, Orwell's works fall under the public domain, and the publisher in question deals almost exclusively in public domain works. They sell them in other countries, and I suspect that their selling them here just sort of slipped through the cracks. Bottom line: the publisher did not have the legal right to be selling those books in the United States, and when the publisher became aware of this, they removed them from all the different eBook distribution companies through which they publish here in the US, the most famous of which is Amazon. There was no malice, and nobody was trying to cheat anyone; it was just a mistake born from the fluctuating publishing rights in various countries. This was not a result of Amazon "caving" to pressure from a publisher.

The idea that's spread through the Internet like wildfire is that this was an action at the behest of this publisher, who decided on a whim that they didn't want to publish it through Amazon any longer, and that Amazon caved and did what they asked. That's just not what happened, and it's rather unfair to Amazon for people to continue to spread the misinformation that says it is.

Granted, caving to publisher demand is something Amazon has done in the past (see: Kindle 2's text-to-speech function that ultimately got disabled due to whining publishers), and attempting to use patents to bully their competitors out of business is one of their go-to tactics, soanother bit of douchebaggary is an easy thing to believe. Like as we see with companies like Microsoft, there have been many deservingly negative things said about them, all which contribute to making it easier to jump to these sorts of ideas.

In conclusion, regardless of how one feels about DRM or the capability for a centralized bookseller to remove files from your device, I think most of us will agree that most of what's being said on the Internet on this subject is misinformed hogwash. Maybe you can help fix that.

Remember last year when I wrote Amazon a letter suggesting that they put Kindle content on non-Kindle devices? Well, they listened to me:

You're welcome.

Now I just hope that the Android version comes along quickly at the heels of the iPhone version, because I don't use my iPhone. (Want to buy an iPhone?)

Dear Amazon,

The Kindle is nice and all, but it's a lot of money for a dedicated device for reading books when I already have tons of devices capable of reading books. A screen, a wireless connection and a keyboard. That describes a Kindle. That also describes laptops, cellphones, iPhones, PDAs, etc.

If you want to sell lots more ebooks, I suggest you release Kindle software for some or all of those devices. Specifically I would suggest laptops and iPhones. The iPhone is my preferred way to read Kindle books (it is way smaller and lighter than Kindle, and I already have it with me all the time), but it'd be really swell if I could wirelessly purchase the books from you and not have to break the law in order to read them.

Now that Apple has 3rd-party apps for iPhone/iPod Touch, I HIGHLY recommend that you make a Kindle app for them. You'll sell bajillions more books than you already do. BAJILLIONS.

Update: This tutorial is largely superseded by the much nicer and easier 'Kindle For PC' method: see it here.

UPDATE 2: included Preston Lee's online PID generator.

D got a big fat tax refund, so she recently bought one of those new-fangled Amazon Kindle thingies for purchasing and reading books in an electronic form. (She loves it.)

I like to read books in an electronic form on my iPhone, but find that it's pretty hard to come by them legally; there are many different sellers and formats, some of which have certain books but not others. Sometimes they have the book, but not in a format I can do anything with. It's generally easier just to illegally download them from torrent sites.

Amazon has tons and tons of books available for Kindle, and have chosen the standard MobiPocket format as the one their reader uses, meaning it is theoretically trivial to purchase them and convert them to something else. Except that they won't sell them to you unless you have already purchased a $400ish Kindle on which to read them. Meaning you don't need to convert them...

In any case, now that our household has a Kindle, it frees me up to purchase books from Amazon in Kindle format without actually having a Kindle myself. I then just remove the DRM that Amazon puts in the files (to keep people from converting them, natch), and then convert them to HTML or txt to read in Books.app on iPhone. Want to know how to do the same thing?

Howto:

Step 1) Find someone with a Kindle.

Step 2) On their Kindle, go to the Settings menu, and type '411' on the keypad. This will bring up a little information dialog with a bunch of things in it, of which you only need the Serial. It is a 16-character string of letters and numbers. Write it down.

Step 3) Ask the Kindle's owner to buy a book for you. Give them some money so you don't look like a mooch. Once you've given them the money, ask them to log in to their Amazon account and navigate to their 'Kindle Downloads' page from your computer. When they complain, mention that you've already given them money. The Kindle Downloads page will list all the books they've purchased, and yours should be right at the top. Click 'Download to computer' and you'll get a file named 'Title-of-Book.azw'

Step 4) Download MobiDeDRM.zip, which is a small suite of Python scripts that some kind soul wrote and then distributed through links that expire all the time and can be kind of a pain to track down. I've hosted them from my site so that they won't expire. This .zip file contains mobidedrm.py, mobidedrm2.py, kindlepid.py and mobihuff.py.

(These scripts require that you install Python on your system, which is something outside the scope of this howto. I'm on linux, but there's a Python for Windows called "ActiveState Python." Google will help.)

After unzipping the archive, open up a terminal window and pass the Kindle's serial # (which you previously wrote down) to kindlepid.py. Something like this:

python kindlepid.py XXXXXXXXXXXXXXXX

Where all those Xs are replaced with the Kindle serial number. It will return something that looks like this:

Mobipocked PID for Kindle serial# XXXXXXXXXXXXXXXX is Z1QFCDQ*74

Make note of that last 10-character gibberish.

The easiest way to get your PID (or secret code used to unlock the Kindle book) is to use Preston Lee's online Kindle PID generator. Simply follow the instructions there.

Now that you've got your PID, we'll need to remove the DRM from any books purchased for that Kindle.

Step 5) Now it's time to remove the DRM. Do this:

python mobidedrm.py Title-of-Book.azw Title-of-Book.mobi Z1QFCDQ74

where my 10-character gibberish string is replaced with the one you made note of in the last step. This will take about a minute, and when it finishes you'll see:

Decrypting. Please wait... done

Now you will have a decrypted MobiPocket-formatted ebook that you can read in any MobiPocket reader.

If, however, you want to convert it to HTML to read on any device you wish, you'll want to install MobiPerl. (This, of course, will require you to install Perl. MobiPerl's website will walk you through that.)

6) With MobiPerl installed, do this:

mobi2html Title-of-Book.mobi

This will create a directory named 'unpacked' that will contain Title-of-Book.html

Things that can go wrong:

Amazon seems to compress longer books in a slightly different manner than shorter books. If your resulting .mobi file and/or .html file are oddly gibberishy (for example, if the first line starts in the middle of a sentence, and clearly not the beginning of the book), let's go back to step 5.

Step 5b) These 'huffdic-compressed' books require a slightly different script to remove the DRM. Do this:

python mobidedrm2.py Title-of-Book.azw Title-of-Book.mobi Z1QFCDQ74

Note the '2' in 'mobidedrm2.py' in this one and remember to replace my gibberish PID with yours.

6b) The 'huffdic-compressed' files also require a different script to convert them to HTML. Do this:

python mobihuff.py Title-of-Book.mobi Title-of-Book.html

This script will output the .html file in the directory from which you are running it.

All in all, this is as much of a pain as it looks, but the selection and availability of books on Amazon makes it worthwhile to me. They have far more books than The Pirate Bay does, and I feel better paying for them

Good luck.

Update: several readers have alerted me that there's a much newer version of mobiDeDRM available. You can download it here.

Dear Amazon.com,

I think that it's great that you're getting into both the hardware and eBook business, but do you really think it's such a good idea to hinge the success of both on each other?  I mean, I have so many devices now that are capable of A) browsing the net, and B) reading eBooks that it would never even cross my mind to pay you $400 American to get another one so that I could purchase books electronically from you.  (OK, so it crossed my mind, but I immediately said "no.") I'd love to be able to purchase your eBooks and read them on any number of my devices, but you make that impossible without buying a Kindle.

You keep making a big deal about how "Kindle is not a device, it's a SERVICE," which is a great way to look at it -- except for one little detail:  you have to buy the device to access the service, so it's not really fair to say that.  Please consider allowing people to download the Kindle files from amazon.com themselves, rather than requiring that they get sent over the EV-DO network to the Kindle that they probably don't even own.

How about an application people could install on their laptops to access the Kindle network from there?  Or better yet: how about a web 2.0 version of Kindle, allowing people to log in and read their books from any browser-enabled computer?