A short ‘how-to’ on making macromolecular structures viewable in the ‘Augment’ augmented reality app.
Allister Crow recently posted a brilliant twitter post of a movie showing a crystal structure in augmented reality: https://twitter.com/Allister_Crow/status/933000138552901632 and he posted a simple HowTo.
To the viewer, they just scan the front page of our paper using the Augment app, and the structure appears strapped on top. Several people have asked how this is done – this short guide aims to show how, I've updated it with a few more options. I’d love to see some other structures uploaded by other labs! If you need further help, feel free to contact (email@example.com)
I've taken the original instructions and expanded them to include a few extra options including how to add colours based on a comment by @tomkazimiers
Download a PDB file from the protein databank, or make your own using X-ray crystallography, NMR or CryoEM. Or you can simply use a molecule file file format such as .sdf, mol2 etc.
You now need to create vrml file
A number of applications can be used to create the files required for 3D display
- Jmol application (not applet) export as VRML (.wrl format) or Wavefront (.obj format)
- PYMOL export as VRML2 (.wrl format).
- VMD export as VRML (.wrl format) or .stl format.
- MOE using io_vrml.svl script export as VRML (.wrl format)
- UCSF Chimera can export crystal structures as STL and VRML files
For Pymol display as a cartoon object then
Go to File-> Save Image as->VRML2, call it: “mystructurecartoon.wrl”.
You then need to convert the .wrl file to .stl format, (note I think the .stl format does not support colours.)
Convert the vrml format file to ‘stl’ format using meshconv. Use the command: meshconv –c stl mystructurecartoon.wrl. It may take some time, but meshconv will produce “mystructurecartoon.stl”.
or use vrml-stl, Use the command ./vrml-stl infile.wrl outfile.stl
Or if you prefer not to use the command line you can download MeshLab which also has the option of exporting to other file formats. One disadvantage of using the .stl format is that it does not support colours. If you use MeshLab you can export to .obj format which does support colours. If you import the .wrl mesh into MeshLab you can export as a Wavefront .obj format, however after considerable searching it seems you have to transfer the colour from vertex to face before exporting. There are many other options that can be used to enhance the display but I thought i'd keep it simple.
This produces two files my structure.obj and my structure.obj.mtl which is the material file that defines the surface, you need to compress these two files into a single Archive.zip file for uploading to the Augment site.
There are also online sites you can upload your file to e.g. https://www.makexyz.com/convert/wrl-to-stl to do the file conversion, but I heard issues reported for these.
Set up a free account with ‘Augment’ (http://www.augment.com/)
Login and upload “mystructurecartoon.stl” to Augment (or, even better, rename it something more sensible and upload that). Or upload the zip file containing the .obj and .mtl files.
Assuming the model is fine, you now need a ‘tracker’. Allister used the front page of his paper, but you can use anything that is appropriately marked so that the app can uniquely recognise it and keep track of its orientation/position via the camera. In the case of the paper, open the pdf in ‘Preview’. Click the first page and go to File->Export. Call it ‘Mytracker.png’ – change the format to ‘PNG’, set the resolution to 200 dpi and hit Save.
Upload ‘Mytracker.png’ on your Augment account. It may take a while.
Click the edit button on the tracker icon, and you can tell it to open the 3d model that you already uploaded by selecting from the menu.
You are ready to go – open Augment on your phone/tablet, scan the physical print-out of your paper first page, and the model should appear, glued to the top of it. You can edit orientation in the app and save them.
I used a simple 2D image of the file shown below. If you print out this image, you should be able to scan it with Augment and see a protein with the domains and ligand all independently coloured.
Update, I've just tried scanning the image on the screen and that seems to also work!
You should see something like the video below.
This might be an interesting addition to text books or lecture notes.
Another workflow has been published "Visualizing 3D molecular structures using an augmented reality app" https://chemrxiv.org/articles/Visualizing3DMolecularStructuresUsinganAugmentedRealityApp/10031888.
We present a simple procedure to make an augmented reality app to visualize any 3Dchemical model. The molecular structure may be based on data from crystallographic data or from computer modelling. This guide is made in such a way, that no programming skills are needed and the procedure uses free software and is a way to visualize 3D structures that are normally difficult to comprehend in the 2D space of paper.
Last updated 26 Octoberr 2019