Evernote: Getting the Most Out of Handwriting Recognition

Evernote has collaborated with Moleskine to offer “smart” notebooks that are designed to provide the best character recognition (see this blog post for details). How do the Evernote branded Moleskines work in the real world? How can you get the most out of Evernote’s handwriting recognition? I’ve created a test note full of images to find the answers — as you read along, give it a spin and see what you find.

当

1. Any Kind of Paper Will Work

I was surprised to discover that regular unlined paper performed slightly better overall than the Evernote branded Moleskine paper (for example, in a search for the word “that“). Both the regular unlined paper and Moleskine paper searches worked better than lined paper when using the Evernote Document Camera, but using the Scanner Pro app with black and white settings lightened the lines enough so that the results for the lined paper were the same as those for unlined paper. Evernote has promoted the Scanner Pro app in its “Trunk,” so perhaps its compatibility is to be expected.

The image below shows the highlighted search results for “that” on lined paper. Two of the three hits for the word were from regular paper.

that

Of course, the Moleskine notebooks come with three months of Evernote Premium (worth about USD $15) and they are very high quality, so I think the question isn’t whether Evernote Moleskine notebooks will improve your handwriting recognition (regular paper actually works a little better), but rather: how much are the beautiful notebooks and the Premium subscription worth to you? Personally, I prefer Moleskine notebooks to regular paper, and I think if you figure in the savings from the Premium subscription, they actually are not unreasonably priced. I have not tested the lined Moleskine paper, but presumably Evernote optimized it for their character recognition.

The image below shows the Evernote notebook I used for this test. Admittedly, the “Sketchbook” is probably a little less common than some of the newer versions, but I like the fact that it is easily used to write Japanese horizontally and vertically.
evernote moleskine

2. Use the Evernote Document Camera

The Black and White images from Scanner Pro worked just as well as the Document Camera images from the Evernote app (for example, ““), but because the Evernote Document Camera does everything auto-magically and sticks it directly into your note, it is the best choice. If you are using regular lined paper, though, you’ll want to use the black and white settings on Scanner Pro instead to get rid of the lines. The grayscale images from the Scanner Pro app did not do as well, possibly because of extra “noise” from the images. This is an interesting contrast to scans of published materials, which seem to do alright with grayscale (see my post on getting the most out of Evernote’s character recognition).

The image below shows the highlighted search results for “之” on regular unlined paper with vertical Japanese text using the Document Camera. Scanner Pro had the same results. Interestingly, the Moleskine paper did not produce any hits for the character in the vertical text.
之

3. Print Clearly and Avoid Cursive (unless it is Japanese)

In English, the printed lower-case sentences produced the best results followed by the upper-case and then the cursive. However, results differ according to the words contained in the snippet text (for example, the word “have” did surprisingly well). In Japanese, both my printed and cursive sentences were recognized about the same. This is good news for Japanese Evernoters!

Below is an image showing a search for the word “have,” which is one of the few words found in the cursive passage. Admittedly, my handwriting is atrocious, and this isn’t a very fair test of Evernote’s abilities. Even I can’t read it sometimes!
have

4. Numbers Don’t Do As Well As Letters

Evernote seems to have a difficult time with numbers. The best number in my tests was “100” — I was surprised to see that it did not show up in the search results for the Document Camera image of the Moleskine page, but did appear for the Scanner Pro image of the page.

The image below shows the hits for “100” in the Scanner Pro image of the Evernote Moleskine page. Only one of the three numbers on the page was recognized.
100

5. Mixing Languages on the Same Page is OK

When more than one language is written on the page, the search recognition sometimes doesn’t do as well for English (for example, the word “pieces” doesn’t show up on the first Moleskine page of mixed languages, but does on the second one that only has English). However, I’ve had inconsistent results. Sometimes, a word is recognized on the mixed Moleskine image but not on the English-only one. Of course, this assumes that you have told Evernote to look for more than one language. If you haven’t done this and you happen to write a word in another language, Evernote won’t be able to read it, but it will not affect how it interprets the rest of the text.

The image below shows a search for “pieces.” For some reason, Evernote only finds it in the cropped image of the Moleskine page. On the other hand, “made” is recognized in the full image of the Moleskine page, but not on the cropped version. I cannot explain this inconsistency.
that

6. Write Japanese Left to Right

Vertical text will almost never be recognized. Evernote only found a few characters in my writing samples (for example, ““). I hope that Evernote will improve this capability in the future, because I know many people write vertically and probably don’t realize that doing so will make it much more difficult for Evernote to recognize their handwriting.

This image shows the results of a search for the character “進.” As you can see, some of the vertical text was recognized. One of the characters wasn’t, even though it looks almost identical to one next to it that was.
進

My Test

Materials

  • Uniball pen 0.7 “Gel Grip,” Black
  • Moleskine notebook, Evernote Edition, “Sketchbook
  • Index cards (lined on the front, unlined on the back)
  • iPad 3 using the “Document Camera” feature in the Evernote app
  • iPad 3 using the “Scanner Pro” app with black and white settings (no adjustment)
  • iPad 3 using the “Scanner Pro” app with grayscale settings (no adjustment)

Writing Samples

I used a snippet of text from a sixteenth-century Japanese source (see details below), and my English translation of it. I wrote the Japanese vertically and horizontally in cursive and printed. I wrote the English translation printed regularly, printed in all-caps (how I usually take notes), and in cursive. In total, there were 14 sample sentences.

The Japanese text came from Yusuhara monjo 由原文書 [Documents of Yusuhara] (undated)/6/26 Tawara Chikasuke kiganmon 田原親資祈願文 [Prayer by Tawara Chikasuke], Zōho teisei hennen Ōtomo shiryō 増補訂正編年大友史料 [Expanded and revised, historical documents of the Ōtomo arranged in chronological order], Vol. 17, edited by Takita Manabu 田北学 and Takeuchi Rizō 竹内理三 (Volumes 24–35) (Ōita: Takita Manabu (24–35 published by Takita Yuki 田北ユキ), 1962–1979), 157, doc. 323.

The Japanese text reads: 「為当病祈念鳥目百疋令進献之候間御懇丹所仰候恐々謹言」. I have only written 「為当病祈念鳥目百疋令進献之候間」 in my tests. I have translated it as: “In order that prayers be made for my illness, 100 copper pieces have been offered, so…”

passage

Recognition Settings

I had my recognition settings for both English and Japanese in all of the tests, and then I reset them for just English, and later for just Japanese. After resetting, I exported the attachments and dragged them into a new note that should have been indexed according to the new settings. However, all of the test results turned out the same. At some point, I plan to test these in another account that is only set to English or Japanese.

Limitations

My tests only involved a small sample of text written by me, and I am sure tests using more writing samples from me and other people would produce more accurate results.

For this reason, I hope that people reading this blog will post their own experiences, and hopefully a link to their shared notes so that we can get a better sense of the strengths and weaknesses of character recognition in Evernote.

Also, it should be mentioned that all of these tests were performed on October 25 and 26, 2013. Evernote is probably continually improving its character recognition capabilities, so the results may change over time. I will update this blog every few months to see if anything has improved in my writing samples.

Finally, I am limited by the poor camera on the iPad 3. I would not be surprised if photos taken using better cameras change the results, and when I have some time I will try to add those results to this post. The reason I am focusing on the iPad is because that is where I do a lot of my work and that is the device I usually have at hand to take photos.

A Few Test Results

These are a few of the numbers I am seeing. Although I think it is more interesting to see where the results are appearing and where they are not, making a table for all of that data would take a little more time than I have to spend on the project.

  • 為: 14 matches / 40 instances
  • 当: 18 matches / 40 instances
  • 病: 15 matches / 40 instances
  • 祈: 6 matches / 40 instances
  • 念: 16 matches / 40 instances
  • 鳥: 18 matches / 40 instances
  • In: 16 matches (in the handwritten notes) / 36 instances
  • order: 1 match / 36 instances
  • that: 3 matches / 36 instances
  • prayers: 4 matches / 36 instances
  • “be”: 8 matches / 36 instances
  • made: 8 matches (in the handwritten notes) / 36 instances

Find Out More

There might be something I can do to modify my handwriting to achieve better results in Evernote. I am thinking about what I can do based on blog posts such as this one that details how Evernote performs its image recognition. If you have any ideas, let me know!