Last month I was whining about “handwriting OCR”, a technology that has mysteriously and simultaneously existed in a quantum superposition between |in use every day at the USPS⟩ and |doesn’t actually work⟩ since at least 1998. (See Why Does Handwriting OCR Suck in 2023?)
GPT-4 with Vision (GPT-4V) is now available to ChatGPT Plus users (somehow Microsoft didn’t release this one first). I figured I’d take a break from my ranting and put it to the test:
GPT-4V transcribes my notebook into Markdown.
It didn’t nail the indentation of the bullet points, but I think this is getting pretty close to useful. If anyone can think of a way to fine-tune GPT-4V (I think we may need to wait for the API) - please let me know. Maybe I’ll finally be able to digitize my handwritten notebooks this decade.
Shout out to “All About AI” for showing the example that inspired me to revisit this. At 00:32 - he draws an outline of a program in his notebook and asks GPT4 to write the corresponding code. Full video below: