DEVONthink: Indexing デボンシンク: インデックスする

DEVONthink is a “personal information manager” (PIM) with a powerful feature called “indexing.” Indexing provides one way to keep data accessible from other apps while still giving access to everything DEVONthink has to offer.

index

Indexing vs. Importing

Indexing works something like this: a mirror image of a file or folder on your drive is created inside your DEVONthink database. Instead of copying files or moving something into the DEVONthink database (called “importing”), the app generates a representation of it inside your database. You can think of it as something like the “alias” feature on the Mac or the “shortcut” feature on Windows. When you open a file in DEVONthink to modify it, the changes are actually made in the file that exists outside of the database. When you move a file from one location to another or fiddle around with folder names, though… things get complicated. More on that at the end of the post.

Suffice to say, managing an indexed file or folder can be tricky. You have to be careful about the various idiosyncrasies and possibly unfamiliar behaviors involved with it. Because of the additional complexity, DEVONthink users have widely divergent opinions on whether you should import or index files. Personally, I’ve found indexing on a single computer to be no hassle at all. Your mileage may vary, of course, but read on to find out if you think it’d be worth a try.

Use Cases

  • The Cloud: Indexing comes in especially handy when using popular cloud services such as Dropbox or SpiderOak. For example, one of my databases is located inside of SpiderOak, and whenever I want to access the contents from a Windows computer (DEVONthink is only available on Mac) I can easily do so, because everything remains in place.
  • Other Apps: Alternatively, I can continue using and enjoying a brilliant app like nvALT while also being able to see and manipulate the same data in DEVONthink. Find and Replace? No problem. DEVONthink cannot search and replace characters in a few thousand notes at once, but BBEdit can, and it’s a simple task to accomplish when you index your stuff. Are you a Scrivener user? Sync your files to an external folder and index that with DEVONthink — I have my folder in Dropbox so that I can sync to the content with Notesy on the iPad as well. Basically, DEVONthink plays well with others, and you don’t have to abandon other apps by placing your data in a proprietary database closed off to everything else (I’m talking about you OneNote).
  • Redundancy: Finally, another benefit provided by indexing is that files seem less likely to suffer from some kind of catastrophic problem — a database can (on rare occasions) become corrupted during regular use, even to the point that it can no longer be opened. With indexed files, the files are outside of the database, safe from harm, and freely accessible.

1. Choose a Folder or File

I prefer SpiderOak to Dropbox for most of my work because it is a more secure service. However, either one is fine for the purposes of indexing. Please remember, though, not to put your DEVONthink database itself inside either of these cloud services.

One caveat: Indexing with SpiderOak works really well when you are using one computer (my use case up until July), but when you add a second computer to the mix and sync with a “local sync store,” things can get quite messy. SpiderOak is a tad bit slow when you have a bunch of files to sync or a large amount of data, and thanks to this lag, you can get older files overwriting newer ones, duplicated groups, and so forth if you are not careful. It’s a mess (easily rectified with backups, but not something you want to waste time doing on a regular basis). While indexing in Dropbox or storing the sync store in it is less secure (encrypted database, but with the password stored in Dropbox), it will probably work more seamlessly for people.

スクリーンショット-2015-05-13-13.52.06

The Japanese folder shown in the screenshot above is the main one I have synced with DEVONthink. The other one is a test folder I made for this post. It contains a rough outline of the folder structure I use for research (I am a historian). It is a tidy little database that wouldn’t work well in the real world, but it at least illustrates what is going on in the screenshot of my actual database at the end of this post.

2. Index the Folder or File

This is the simple part. When you’re done, everything looks almost identical to how it would appear if you had imported the content directly into the database. The only visible difference is the small arrow mark next to each indexed file or folder.

スクリーンショット 2015-05-13 13.52.53

3. Apply an Indexing Script (optional)

One quirk of indexing is that new content created in DEVONthink will not automatically be replicated at the location you are indexing. In order to accomplish this (more or less automatically), I am using a script provided by the DEVONthink well-established authority on indexing, Greg Jones. You can find a copy of it and an explanation in this thread.

スクリーンショット 2015-05-13 13.53.44

4. The Final Product

Here is a screenshot of my actual database. Even if you know Japanese, it probably looks like gibberish, because it is a file structure created for my needs, and there is some stuff going on there that’s unique to DEVONthink. I’ll explain the rationale behind the structure in a future post along with its pros and cons. For now, I want to share how an indexed DEVONthink database looks in the wild.

スクリーンショット 2015-05-12 22.10.03

Final Word: Mirror-ish

Although I used the word “mirror” at the beginning of this post to explain what indexing does, it may not be an appropriate word choice. First, I should say that the DEVONthink database will probably not look *exactly* like what you see in Finder. DEVONthink is much more flexible and capable than Finder, and so it is neither possible nor desirable to try and have an exact replica. Rest assured, all of your data will be safe and sound in the indexed location — it might just be a little out of place.

Second, things are a bit more complicated than the word “mirror” suggests. “Korm,” an experienced DEVONthink user with a far better grasp on the topic than I do, has this to say about it:

Actually, a “mirror image” is not created in the database. Within the .dtMeta files [a set of ten index files named with the suffix “.dtMeta” inside the database package] the database contains a concordance of the words in the indexed file, and certain other metadata is copied, but there is no “image” of the file. When you view an indexed file you are viewing the actual file in situ in the filesystem. DEVONthink uses available Quick Look [an Apple OSX preview feature] plugins to display the content — if there is no plugin there is no display. (This is the same result as selecting a file in Finder and pressing Space to see the preview.)

When you change an indexed file it is not the case that changes are made in the database and “also in the file existing outside the app”. There is only one file, and only one change — to the file in the filesystem outside the database. So, there is no “mirror” except in a metaphorical sense.

indexing

Indexing and aliasing are different technologies. With aliasing, OS X tracks where the original is located. If you move the original, OS X has no trouble finding it. Indexed files, however, are not tracked. DEVONthink stores the path of that file at the time of indexing but has no means (currently) to track it or to change the address.

It’s a lot to digest, but knowing these additional details might help you avoid some pitfalls in the future. Many thanks to Korm for providing the explanation and the diagram. As you can probably already tell from this and other information provided by users on the DEVONthink forums, there is a vibrant, friendly, and helpful user community for DEVONthink that can help you whenever you have any questions about putting it to work for your use case.

Related Links

  • Macademise
  • Tom’s Guide is full of outdated, misleading, and mistaken information. However, lots of ground gets covered and useful information gets provided in order to clear things up.
  • Jason Heppler explains how to use DECONthink in graduate school.
  • Me in an earlier post on getting started with DEVONthink.