iCloud Drive and Calibre compatibility

About 7 years ago, I wrote a post about how the ebook library management app Calibre can contribute to problems with Amazon Kindle e-readers: it basically screws their battery life. I detailed a workaround for it, and to this day, I still get visits to that page, so neither Calibre nor Amazon must have gotten around to fixing it.

At one point, I had a large orange chunk on the bar above dedicated to “Documents” that weren’t actually there.

I now have a suspicion that Calibre also causes problems with iCloud Drive, so I’m leaving this here for anyone it might affect. Some scenarios for the search engines below:

  • If you store your Calibre Library folder on iCloud Drive, and have noticed that your remaining space does not reflect the storage you’re using, this is for you.
  • If you have deleted files on iCloud Drive but find that the free space reported by or your device does not immediately update to reflect the deletion, or…
  • If you have removed everything on iCloud Drive but still find space allocated to “Documents” in the “Manage Storage” section of iCloud settings on your iPhone/iPad — in other words, if you expect to have free space, and have done everything including a check of the “Recently Deleted” area and emptied your Recycle Bin, but the amount of free space is still inaccurate, I know that feeling.

This seems to happen to a lot of people, perhaps for other reasons, but it happens. Skip ahead to “Is it you, Calibre?” if you just want my conclusion.

So here’s my experience. A couple of years ago, I had this issue, and had to call Apple Support. It took several calls to resolve, because they wanted me to sign out of iCloud on every device (not an insignificant hassle with multiple devices), and when that established that the issue was on their servers, it had to be escalated to Engineering, and the eventual fix was they wiped everything on my Drive and reset it. I had to backup all my files locally (requires a Mac!) first. I believe I still suffered some data loss.

After that, it was all good, but my confidence in iCloud Drive was shaken, and I didn’t want to use it as storage for anything important. Every year since then, they’ve made enhancements to iCloud Drive, and to the on iOS, which has made me slowly more willing to embrace it again as a cloud file system worthy of My Stuff.

And then this happened again. 13GB of space just wouldn’t come back after I’d deleted files. The files were gone, the space was not reclaimed. After putting it off for two weeks, I got on the phone with Apple again, and two calls later, they managed to “repair” my Drive using some standard tools they have (Engineering was not involved this time). So, a slightly better experience than before.

Is it you, Calibre?

I’ll say upfront that this is a hunch. I don’t have the strength anymore to experiment on my iCloud account and conclusively prove anything.

Both times this happened to my iCloud Drive, I was using it quite “normally”. Nothing fancy, except that my Calibre Library folder was on it, and I knew that the Calibre app was actively updating files on it whenever I added/removed ebooks. This last time, the problem appeared after I’d deleted a ton of files THROUGH Calibre, and as best I can recall, a similar situation took place years ago.

With the Amazon Kindle problem, there’s something about the way that Calibre writes files to the Kindle’s drive that causes it. In other words, Calibre (which is a sluggish cross-platform app that behaves in a very non-Maclike way) may have some non-standard ways of interacting with the OS and filesystem. I think the way that it writes/deletes files isn’t the same as if you manually dragged files around yourself via the Finder. It might be through some low-level UNIX operations, but this is where I’m out of my depth.

So it’s not a stretch to imagine that when you delete ebooks in Calibre, it deletes them from your drive in a way that may cause issues. Deletes them in a way that is invisible to iCloud, so it doesn’t know that the files are gone and it should give you the space back. On a local drive? It works fine, and that’s how it’s used by millions anyway. But on a weird aliased virtual cloud drive that Apple hacked together inside a folder called “Mobile Documents”? Maybe not fine!

Here’s what I’d suggest trying if you have this problem: move your Calibre Library off iCloud Drive. I’ve put mine on Dropbox and it seems fine. Do NOT put it on Google Drive. Call Apple, and have them repair or reset your drive. Some luck is required here, but they’re your only hope. Once you get your missing space back, don’t use Calibre with it again.

I’ll be here with crossed fingers too, waiting to see if this happens again.