Adobe’s Dually Noted adds AR annotations to physical books and documents

Although digital reading software and hardware have come a long way over the past decade, there’s something to be said for the classic way of adding annotations to physical books and documents — educators and collaborators still enjoy writing notes in the margins, along with corrections that overlap the original text. Today, Adobe is announcing a potential solution called Dually Noted, leveraging augmented reality and cloud technologies to create virtual annotations that can be viewed on physically printed materials.

Combining computer vision with cloud access to a digital version of a printed document or textbook, Dually Noted lets a teacher add notes to the digital content using Adobe Document Cloud, while students can see notes on their physical materials using AR. At first, students might have to hold a phone or tablet up to the real-world document to see the notes, but once AR glasses become more common, the notes would blend into the document more naturally.

In a live demonstration, Adobe’s Curtis Wigington showed how a teacher’s notes can pop up on the screens of student devices, and how students could add their own responses and questions by simply tapping on recognized text or images and then speaking to the device. Students probably wouldn’t want to read entire books this way, he said, but they could use Dually Noted to check in with the teacher for key study points or questions.

Above: A teacher and student interact with a cloud-based digital discussion shared via typing and/or voice.

Image Credit: Adobe

Educational institutions wouldn’t be the only beneficiaries. Adobe expects the AR annotation technology could be used to make medical and legal documents more understandable, as well as bills and other types of paperwork. Virtual notes could be limited to personal one-on-one collaborations or shared more broadly, depending on what the commenter chooses.

Adobe’s hope is to make the physical-digital collaboration process simple and widely available. As a plug-in or component of an app such as Acrobat Reader, Dually Noted’s AI and computer vision enable the software to recognize the physical source material without forcing the user to hunt for the digital document. Then augmented reality composites the digital annotations on the real-world book or document, constantly moving to preserve the location selected digitally.

Dually Noted is an Adobe Sneak built upon research conducted by summer intern Jing Qian, which is to say that it’s under development but not yet guaranteed to arrive in a specific Adobe software release. For that reason, the actual release date for Dually Noted is ambiguous, but the underlying technology looks like it’s pretty close to ready for prime time, with only UI polishing standing out as necessary.