AMI: Toronto, 2012: Day 1

Today is Wednesday.  The first technical day of the AMI conference.  This is the day that the workshops are held, which are generally awesome, but also cost extra money because of their inherent awesomeness. We weren't able to attend any workshops this conference, but we did arrive!

Leslie and I flew from SFO at 8 am this morning and arrived in Toronto around 4pm.  My phone has been off for a while, so I feel a tad cut adrift from time.  We ended up taking a shuttle to the hotel and encountering tremendous traffic.  Seriously.  It was intense.

We got here just in time to change and head down to the salon opening.  We met some fantastic people, and I didn't remember to take any photos.  The projects, as always, are thrilling and inspiring.  And we got to meet up with John and Matt and Patrick and Melody and Christine and Betty and... yeah, it was a great time.

So, have I been able to gush enough yet?  I am very, very happy to be able to come this year, and I can't wait for the rest of the program to start tomorrow.

View from our hotel room

AMI: Toronto, 2012

Well, Leslie White and I are headed to the AMI: Illuminate conference in Toronto tomorrow morning!  I'm super excited about the trip.  And to finally be working for someone (namely, Stanford) that will support extra learning.  This looks to be an awesome conference, that's for sure.  

Things I am excited about in the program (besides everything): Digital textbooks, molecular imaging, online learning, poster sessions, technique showcase, interactivity in healthcare, 'imaginative realism,' designing innovation, the Vesalius Trust Auction (I have two 'eyes' in that!), Medical gaming, Stereoscopic 3D production, painterly photoshop, Unity!!, education through simulations... the list just goes on and on and on...  Such an amazing conference.  How could you not want to go?  And that doesn't even touch on the benefit of just keeping in contact with some of my favorite people.


We also decided to submit the beta version of our app to the Salon, so here's hoping.  It wasn't (and still isn't...) finished yet, but it was a good working version. 

All in all, I think it's going to be a blast.  I'm taking a notebook and an open mind.  I can't wait to learn.

Paper Mache Eye: Take 2, part 3

So!  I am 99% done with this project. And I left it at work, so I can't get that 1% done tonight, but it just involves cutting a slightly larger window for the light switch - not a big deal.  I haven't updated about it in a while... so, here we go!

I've had a ton of fun with this.  A complete learning experience, and never quite what I expected.  I made a few mistakes, and if I had all the time in the world, I would make one more of these.  I might still, if I want one for myself.  But, this one turned out pretty well even if I feel it's a bit cheesy, all told.

Electricity!

But, I'm jumping the gun a bit.  So, I left off with the eye being mostly finished, at least in my mind at that point.  The paper mache had dried, and the iris had been... semi-painted. 

From the beginning, I had this idea of adding a cornea to the eye as well.  I felt like the white tissue paper was translucent enough to work.  And with a hole cut in the iris for the pupil, I could get a balloon in far enough to make a small armature for the cornea.

 So, I taped down the balloon to flatten it out some, and started to add a very, very thin layer of paper mache.  I put a release on the balloon and only wanted to use one layer of the tissue paper.  I wasn't even sure if it would stand up on it's own after removing the balloon.  I let it dry for two days, just to make sure.

And, it worked!  Sorta.  I wasn't very happy with it, though.  The paper wasn't nearly as translucent as expected.  Very milky white.  I'm still not sure why it didn't dry quite as see-through as it seemed to on the eye itself.  It may have to do with the color under it when it was on the retina though.  But, I left it for now.

Late one night, I decided to modify the lighting rig.  I had two door lights, the ones with the magnetic opener.  So, I ripped one apart.  Just completely demolished the casing in the hopes of lessening the space it takes up.  And, low and behold, it worked! 

This was a pretty big gamble, and the magnetic on/off switch did die (and I have not a single clue why.  there's nothing that actually broke.  So why would the magnetic on/off stop working outside the plastic?)  However, this time, the reward was worth the risk.  Although I did lose the diffuser effect.

All lit up, with the (mostly opaque) cornea.

I decided that it was way past time to get the backing painted.  I sketched out where I wanted the otic cup and fovea, and the vessels.  I did make the arteries match up to the thread arteries on the eye itself.  I'm inordinately proud of that.

I added a basic orange, and then a few layers to help lay things out.  And I decided to use the fovea as a type of light source.

Laying in artery placement

Adding shadows to the vessels

So I started to block in where I wanted the arteries to be.  And I found out that trying to paint tiny arteries on a 4 x 4" square with a set of brushes meant for 30 x 40" canvases at the least... is difficult. 

Very difficult.

Shadows.

More shadows.

I stared to add in the shadows that would be cast onto the retina wall.  And I started layering in the shadow more loosely, trying to get a more natural feel.  I kept the style pretty rough, though.

More shadows, still.

Highlights!

I made the shadows deeper, and started to use different colors.  I also defined the otic cup a bit more.

Then I added highlights along the vessels themselves.

 

Then I got a little crazy with the shadows and color layering.  The entire piece had felt a little disconnected, and I couldn't figure out why.  So I went more dramatic.  I layered in a few layers of shadow over the whole thing, and then I added a red and orange layer for good measure.  It's not perfect, but it's a sight better than it was.

But then I got a little paint happy.

I decided that since the background and the retina of the eye had taken color so well, I was going to add some to the white and just a hit of blue on the cornea.

It looked horrible.

Luckily, it peeled off.  Sorta.  the entire cornea was ripped off just like a bit of paint.  Goodbye, cornea.  Not sure I'm going to miss you.

With a combination of scraping, knives, sandpaper of various grits, and judicious use of finger nails, I was able to get the majority of the white paint off.  Nasty paint.

And I took a huge leap of faith and cut a viewing window into the eye.  I ended up liking the backdrop just a tad too much. 

And, no way to go back now!

Cutting a piece did help me to avoid the area where the white paint wouldn't come off.  This cut section had a short stardom role as test painting for about a hour or so, then was put to pasture as just a useful piece of funk.

I don't know if I'm crazy or just persistent (or both, quite possibly), but I tried paining the sclera once again.  I did test it on the eye scrap, and it turned out alright. 

I wanted the texture between the retina and sclera to match as well - at the moment, both sides are painted over with a black wash and then blotted dry to preserve some of the translucency.

I also added some different colors to the iris.  Made it a light blue with more striations and mixed in just a hint of green.  Mostly blue, as you can see.  Some iridescence as well.

Let there be light!

And that brings us back here.  Once I retrieve my eye from Stanford, I'll post some 'finished' pictures.  And consider not doing a blow by blow again... that makes for some long posts.

Research: project scheduling

I don't like to schedule things.  As in, I don't feel that I estimate dates well.  Telling someone that I will finish that in 1 hour or 1 week feels... strange to me.  And yet, very necessary in my day-to-day world.  I am much better at estimating dates than I used to be, now that I have seen how long project take in the 'real world' with my work environment taken into account. 

However, I often just give a general guideline - I will be done with the entire thing in 2 weeks.  Breaking a project down into it's component parts is a level of difficulty that I dislike even further.  What do you mean I need to know how much time it takes to segment vs. retopo vs. texture vs. animate vs....  Give me two weeks and I'll give you the finished product.

But!  That's not life.

And breaking my research project into manageable chunks will work out.  I do like the feeling that something has been finished.  Even if it's something like finishing the first coat of paint or one bone segmented from a skull.  Little blue check marks next to an item that I needed to complete gives me a warm and fuzzy feeling.

So, despite the fact that I wanted to write this:

Even this has more detail than usual...

 I gritted my teeth, put on my thinking cap, and wrote this:

What?  That looks... almost official.  Like a project schedule, even.

So, I have made my project schedule drawn up and submitted it to my committee and even blogged about it.  I think it is high time to paint.

Self portrait sketch

In addition to other things, I am working on a challenge from CG Hub - 'skinless portraits.'  Very fun.  So, here is my quick color study of myself.  And I do mean quick, as in a sketch. 

Webcam shot - I'm tired.

So, this will be the basis of a 'skinned' portrait.  Fun!  I love CG Hub - they have great ideas.

Amira: lasso tool

Okay, so we 'left off' with just a base segmentation started, using the threshold tool.

Regardless of the nature of the end product (at least most of the time), the bone marrow area will need to be filled in.  I typically like to do this before I do anything like segmenting out separate structures.

This can be done using a combination of the other tools available.  I find the lasso tool to be a great place to start.

Lasso tool:

It actually works a lot like Photoshop.

Like the other tools, the lasso tool has options that affect how it works.  I often use it in the default mode (as seen here), but some of the bells and whistles can be interesting.



The 2D Freehand shape, the one I use the most, allows you to draw any shape on the individual CT slice and the lasso tool will select whatever is inside of that ring.  Fairly self explanatory.  If you select the "Auto trace" check box, the tool starts to act more like the magnetic lass tool in Photoshop.

The trace function can snap to the edges of a selection when selected.  This is also where the 'trace edges' check box comes into play.  When checked, it snaps the freehand lasso to the nearest edge, for the most part.  If unchecked, it tends to lasso around the nearest area of contrast.

 

This can be a useful tool for filling in the interior of the bones.  However, I feel like I can get a similar effect while using less time in just 'freehand' mode.  Because there is often a thicker area of bone around the interior, it can be easy to do a quick circle around that.  Of course, I am using a wacom tablet instead of a mouse, so the motion feels very natural to me.

The next option over is the 'ellipse' setting.  This draws a plain circle, or ellipse (the oval shape and size are controlled by you).  I remember using this when I first started out, but I very rarely use it now.  I prefer the control of the freehand.  I suppose it can be a useful tool to interpolate between for selecting a swath of voxels.

And the final option in the 2D portion - the rectangle setting.  Yeah, I never used this. Because I work with very organic shapes, I just have never used it.  Like the ellipse tool, I suppose it would be useful for eliminating voxels along the edge of scans that get selected by accident?  I still use the free hand lasso for that though.  But, it's an option.

A pretty cool part of the lasso tool, and one that I often (regrettably) forget about, is the 3D lasso mode.  It has two options - inside and outside.  With the lasso tool selected, enter into 3D preview mode.  The view in "3D" checkbox needs to be selected next to the material, and a point cloud will be generated based on that material.  With the lasso tool activated, you can draw in 3D space and Amira will interpolate that selection in the Z plane.

You can see here that if you aren't lined up correctly, you may end up killing some of you model inadvertently.  So you'll need to keep an eye out for that.  But, it's a very fast and easy way to deselect those pesky voxels at the edge of a scan.  Anything you select in this mode can be added or subtracted from a material.






Great, so that's how the lasso tool works.  Now, what do I use it for?

Personally, I use it to fill in the gaps, like I said when I first started this post (hours ago, so I had to check again...)  If you fill in one slice, and then choose one many slices down the road, you can fill in large gaps.  It's similar to putting a lid on both sides of a tube.  The tube is now water tight and the selection will stay within those lids.

Hold 'shift' to select multiple areas

Here I've used the Pick & Move tool to selected all of the areas between two slices that I filled in.  With the 'all slices' box checked, Amira will automatically fill in just that area, if it is water tight.  If there is any leak to the outside volume, the selection will leak as well.  These worked pretty well.

Something to keep in mind can be how thick the outer bone is - in things like long bones where that thickness is larger, this process can save huge amounts of time.  In skulls?  Not quite as useful.


In any case, that is the lasso tool in a nutshell.

Paper Mache Clay: Eye 2

I let the eye dry for a couple of days, and I've put the first coat of paint on it.  It's supposed to be a 'dark' piece- so I'm thinking of a sculpture look-alike rather than realistic painting.  I hope that I can pull it off.  And quickly!

Ready to get my painting groove on!

First layer - Burnt Umber, Raw Sienna, and Indian Red.  Mostly burnt umber, as you can see.  This is just to give my dark under tones.







But! Would I really be an eye without eye lashes? No!  So, three gauges of wire stuck in and superglued to make the eye lashes.

A thicker wire with the other two gauges for the upper lashes.  Thinning out as it goes towards the corner of the eye.

And just the two thinner wires for the lower lashes.  Again, thinning as it goes towards the corner.

Painted the wire to match the base color. 

I thought about adding glasses as well, but I made the eye too big to do that.  Whoops.  And I made this one directly on the illustration board, so no take backs!

I've started to add in the green under tone as well.  After this dries completely, I'll be able to start doing washes with other colors to give it a richer look.

Amira: threshold tool

I was asked an interesting question today - how do I like Amira as a segmentation program?  Short answer - I do like it.  I used Mimics in Grad school, but I've used Amira exclusively since then.  So, I decided to do a post on the tools I use in Amira 5.3.3 to segment. This is not meant to be a 'how to use everything in Amira' post - just a quick look at what tools I use to do what when I segment. (Just for the record, the user manual is over 500 pages long.  No joke.)  This post may end up being a series of posts...

But before I just jump into the tools, let's touch on Amira in general first.  Amira has two sections that I use the most often.  It opens up into the Object Pool, where you can load your data set and label file.

Mode menu in the upper left of Amira

These are kept separately by Amira, but a data set cannot be viewed in the Segmentation Editor unless it has a label file associated with it.



Switching to the Segmentation Tool view mode, you can see where your scan is loaded in the Image Data area.  You can have multiple label files associated with one scan, and those will be located in the Label Data area.  You can switch between your label files at any time using the drop down list.  If you can't see the scan itself, it's likely you don't have a label file associated with it yet.  Just click the 'new' button to the right of the Label Data area to create one.

Within each label file are 'materials' used to actually segment the scan.


I think of like a series of tracings - you have your original drawing that you are breaking up into layers using tracing paper.  All of the tracings together make up the 'label' with each sheet being an individual material.  The main thing to keep in mind is that every pixel of the scan can only be claimed by one material at a time.  Materials do not overlap.

Your initial label file comes with two materials - the Exterior and the Inside.  Exterior isn't actually a material per say - rather, it's where all of the pixels that aren't in a specific material are kept.

If you were to view the exterior in 3D, it would just be a block.

Here you can see that the 'exterior' material has all of the pixels not currently assigned to the 'inside' material.  It shows as just a block.  It's the only material that I allow to stay as a default assignment.

I did a very fast thresh-holding to show how pixels assigned to an different material will show in 3D space.  Material names and colors can be changed in the material list by double clicking on them.  They can be turned on and off in both the 3D and 2D view ports.  The outline in the 2D view port corresponds to the color of the 3D object and is defined by the color block next to the material's name.

Tool Icons

These are the tools available for basic segmenting.  From left to right: Pick & Move, Brush, Lasso, Magic Wand, Threshold, Blow Tool.  My most used tool?  The brush, hands down.  But I have used all of them, so we'll take a look at examples of when I would use which tool.  I will be going in order used during segmentation though, not actual tool order.

Threshold: Unless I am starting on a data set that has already had some segmentation done, this is my first tool of choice.

The threshold tool allows you to select all the pixels within a certain grey-scale range.  As you can imagine, this is a fast and dirty way of grabbing as much of a certain object at once.  This works best on high contrast material, such as bone.  (It works even better if all you have in a scan is bone, as is the case with this canine skull I am working with right now.  Otherwise the editing grows exponentially.)

Most tools have more options available to change.  Those are located below the tool menu itself.  So the threshold tools allows for both single slice and all slice selection (if you uncheck 'all slices' it will only look at the current slice) and you can threshold within a certain material only.  After setting the threshold, one must select that setting to be able to add it to a material.

You can set what range of pixels will be selected by the threshold tool directly above the tool menu.  Without getting too complicated (Hounsfield units, etc) - there is a small histogram that shows the range of values within the scan: white being on the thousands on the right and representing bone or metal, black going down to -1000 on the left and representing air (0 is water).  Humans can't actually see that large of a variation in grey levels, hence the window width and window level values.

However, to simplify - threshold can select a range of these values.  You can dynamically see what is selected in blue as you adjust the slider.

Once you are satisfied with the selection, you must actually 'select' it to be able to add it to the material.  Once selected, it turns red (and mixes with the blue of the threshold tool to turn a purplish color) and the selection tools turn red on the left.  "Add selected voxels" or the "+" button will begin the segmentation process.








Now, why would you threshold on a single slice only?  Or even within a material?

Both options are useful when trying to separate a single material into more complex selections.  Say you are trying to remove teeth from the whole skull material.  Or bone from a more universal body scan.  Again, this only really works in a whole scale way on areas of higher contrast, such as bone and teeth.

You can see that the black areas are not selected - the insides of the bone will need to be filled in using different tools.  But this gives us a great selection to start from.

And I will look at the other tools another day.