Sunday, November 20, 2016

Photogrammetry, a Don't Do This guide to 3D modelling

Photogrammetry is a technique that stitches together pictures into a 3D surface, which is quick and useful for paleontologists, and also really impressive to their parents (OMG IS THAT 3D? YOU WORK IN THE FUTURE AND THE PAST ALL AT ONCE YOU MAKE US SO PROUD).

One of the big advantages of photogrammetry over 3D scanners and microscribes is speed and information quantity: it takes Meaghan only 15-20 minutes to photograph a specimen at a museum, which makes her visits much shorter and saves her money. This has sped up her collection time so that she now has a dataset of hundreds of oreodonts and modern animals with which to do her dissertation - too much data, if you asked the rapidly-panicking grad student who saw all this work looming on the horizon last year.

One thing that Meaghan is chronically bad at is following any sort of verbal or written directions (worst navigator of all time), preferring instead to stumble blindly through making hundreds of her own mistakes. So since Meaghan is now an expert in "how to mess up Photogrammetry" we thought it was time to lay some of that knowledge onto you, dear readers.

Have you ever struggled to use the program Agisoft Photoscan to make 3D models of skulls? If so, you're part of a very niche group of nerds we here at the blog would like to address in a short segment we call "Dear Photoscan Princess," which is code for "Dear blogger who spent too much time in the basement at University of Oregon screaming at her computer," and also code for "Dear Meaghan." To save us time and effort, let's just stick with the last variant and answer a few questions that Meaghan made up for herself as a writing tool so she could write a blog post on how to suck less at using this program.

Dear Photoscan Princess, what is your ideal set up for taking photos?
Part 1: Get a Table
I've heard some people have a lot of luck with turn-tables, but I haven't tried those. Usually I go low-tech and use items already at the museum. In that case it's important to get a small table that you can walk all the way or at least most of the way around. Otherwise you're going to be laying across that table to get pictures from behind, and while it is funny to play Austin Powers the fossils don't really respond to your "ooh baby yeah" comments and you look like a serious dumbass if anyone walks in. Trust me. Also, cropping yourself out of your own photos is weird.

Part 2: Remove Distractions
The ideal room for photogrammetry is boring and small, with plain walls that aren't the same color as whatever you're photographing. Most of Meaghan's images have plenty of stuff in the background: other skulls, overhead lights, ladders, her arms, or labmate Win McLaughlin, for example. The trouble is that if these things are replicated in many many photos, the program will helpfully model them for you. This is a waste of energy and time, and will also remove resolution from the part you're trying to look at.

The program, wasting its time.

In particular areas of reflection on the floor or bright overhead lights will be problematic because while the computer WANTS to reconstruct them, it does not understand the concept. This leads to a couple of problems - in some cases, glare on your specimen will cause the model to just... not be there. In other cases, blending of glare with background lights will occur and you'll get fun floaty cloud structures. There are ways to edit these out, but if you can avoid photographing lights that contact the outline of the specimen, you should do that.

Part 3: Table Cloth
That said, you do want something interesting for the specimen to sit on. Black cloths are very common, and these can be nice for publishable photos, but in general a textured background gives the program something else to use as additional reference points. Don't use something shiny (most of my Field Museum camels have a shiny binder reflected up onto their jaws, which looks ridiculous). Also, use scale bars - but ideally, use ones that aren't shiny! Make sure to get some of those in focus too, or your scale bars won't be particularly useful

...nice and fuzzy scalebar, great.

Dear Photoscan Princess, what sort of camera should I have? How many pictures should I take?
Meaghan initially thought it would be a really great idea to go into photogrammetry with a really nice museum-quality camera. It wasn't. This camera took forever to focus, and had to be perfectly still to maintain any depth of field, which wasn't good when Meaghan found herself lunging behind specimens into awkward positions to get the right angle on the specimen.

It's better to instead use a digital camera that takes decent photos but has a good automatic setting, particularly one that has an internal stabilizer that compensates for the shakiness of your caffeinated, jet-lagged-and-fumbling fingers. You want to take lots of photos, and you want to take them quickly: 50-60 photos per side is Meaghan's shot-for standard, so she can delete blurry ones and have good coverage overall. Take them from as many angles as you can creatively discover in the moment. Try to get a few good shots of the scale bar, too, so that doesn't turn out blurry. For a mental picture of your coverage, Meaghan has helpfully provided an image:

Dear Photoscan Princess, what's your Princess-Approved Editing Process like?
Painful. Slow. Caffinated.

Dear Photoscan Princess, my specimens are all pixellated! What happened?
You forgot to change your bounding box. Meaghan didn't discover this trick until she'd already made 50% of her models - don't fall prey to this same trap. Change the red bounding box so it is a tighter fit around the specimen, and brings the specimen into better focus.

Notice the thin red box. It's too far away from the specimen, bring it in closer for a better resolution model.

Dear Photoscan Princess, my specimens are really slow to load, I can't really do anything with them. WTF?
That's code for "my specimens are toooo big." You had a lot of pictures and chose a high model resolution, didn't you? Try decimating your mesh to something a little more reasonable, then build your texture again. You'll find this version a lot easier to work with.

Dear Photoscan Princess, I can't zoom in! How am I supposed to see features when my specimens blink out?
I don't know why this happens, but I DO know how to fix it! Right click on the specimen, and click "center view." This will automatically zoom in a bit, as well as centering whatever you looked at. Go ahead and zoom in more. If it blinks out again, right click and center again. It's a weird bug with a weird fix. I dunno, I don't computers, and maybe they've fixed it by now.

Dear Photoscan Princess, I seem to have developed ghostly jaws! What happened?
I have a lot of ghost-jaws: that is, specimens where the jaw fell over slightly, I didn't notice, and so technically the photos incorporate two different positions of the same specimen and the computer created a siamese-twin effect. Most of mine came from poor propping, but if you bump something that can happen too. Find the offending photos if you can.

GOOD LUCK PHOTOGRAMMETERS! May all your specimens turn out complete, and none of your scale bars be blurry.