Join gaming leaders, alongside GamesBeat and Facebook Gaming, for their 2nd Annual GamesBeat & Facebook Gaming Summit | GamesBeat: Into the Metaverse 2 this upcoming January 25-27, 2022. Learn more about the event. 

If your dreams occasionally involve spreadsheets that extend endlessly in all directions, good news: Researchers have developed a virtual reality spreadsheet interface that could expand Google Sheets and Microsoft Excel files from flat screens into 3D spaces. Rather than being nightmare-inducing, it could actually make spreadsheet apps more usable than before.

While traditional spreadsheets have been limited by the boundaries of 2D windows and displays, the research team envisions VR opening up adjacent 2D workspaces for related content, then using 3D for everything from floating menus to cell selection and repositioning. In one example, a VR headset mirrors the view of one spreadsheet page displayed on a physical tablet, while two virtual sheets sit to the left and right, permitting drag-and-drop access to their cells, as an overview hovers above.

Alternatively, tablet-surrounding areas could display useful reference materials, expanded views of formulas, or the full collection of a spreadsheet’s pages displayed as floating previews. Another possibility is a single spreadsheet page that stretches far further than the 30-degree diagonal field of view occupied by a typical tablet on a desk, utilizing more of the ~110-degree fields of view supported by VR headsets.

Fans of the film Minority Report and portrayals of similarly holographic future 3D interfaces will appreciate the team’s use of a floating pie menu — complete with a drop shadow on the spreadsheet — for selecting features and functions, as well as spherical rather than flat buttons and other visual elements that appear to leap off the flat pages. The 3D spreadsheet workspace could also be extended with floating desktop objects, such as a virtual trash can, to make disposing of unwanted content more intuitive.


The 2nd Annual GamesBeat and Facebook Gaming Summit and GamesBeat: Into the Metaverse 2

January 25 – 27, 2022

Learn More

Interestingly, the project’s research team includes members of the Mixed Reality Lab at Germany’s Coburg University, as well as a professor from the University of Cambridge and two principal researchers from Microsoft — mixed reality expert Eyal Ofek and UX engineer Michel Pahud. But their work isn’t limited to potential Microsoft applications: Dr. Jens Grubert, one of the paper’s authors, tells VentureBeat that the cross-organizational team includes “long time collaborators” and actually used Google Sheets rather than Excel for the backend.

In addition to an HTC Vive Pro VR headset and Microsoft Surface Pro 4 tablet, the researchers employed a spatially tracked stylus, enabling precision direct input for spreadsheet interactions while adding the freedom of in-air movement within a 3D space. Unsurprisingly, the VR spreadsheets can be used on existing commodity PC hardware, and the virtual UI was created with the Unity engine. Sheets pages are rendered within Chromium browser windows to match the resolution and size of the Surface Pro 4’s screen.

Full project details are available in the “Pen-based Interaction with Spreadsheets in Mobile Virtual Reality” research paper. If you’re interested in deeper dives, you can see a video of the project here, as well as a broader exploration of the team’s VR-tablet research here, ahead of their presentation at the IEEE’s International Symposium on Mixed and Augmented Reality, being held online from November 9 to 13.


VentureBeat's mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact. Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:
  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more
Become a member