Research: project scheduling

I don't like to schedule things.  As in, I don't feel that I estimate dates well.  Telling someone that I will finish that in 1 hour or 1 week feels... strange to me.  And yet, very necessary in my day-to-day world.  I am much better at estimating dates than I used to be, now that I have seen how long project take in the 'real world' with my work environment taken into account. 

However, I often just give a general guideline - I will be done with the entire thing in 2 weeks.  Breaking a project down into it's component parts is a level of difficulty that I dislike even further.  What do you mean I need to know how much time it takes to segment vs. retopo vs. texture vs. animate vs....  Give me two weeks and I'll give you the finished product.

But!  That's not life.

And breaking my research project into manageable chunks will work out.  I do like the feeling that something has been finished.  Even if it's something like finishing the first coat of paint or one bone segmented from a skull.  Little blue check marks next to an item that I needed to complete gives me a warm and fuzzy feeling.

So, despite the fact that I wanted to write this:

Even this has more detail than usual...

 I gritted my teeth, put on my thinking cap, and wrote this:

What?  That looks... almost official.  Like a project schedule, even.

So, I have made my project schedule drawn up and submitted it to my committee and even blogged about it.  I think it is high time to paint.

Amira: lasso tool

Okay, so we 'left off' with just a base segmentation started, using the threshold tool.

Regardless of the nature of the end product (at least most of the time), the bone marrow area will need to be filled in.  I typically like to do this before I do anything like segmenting out separate structures.

This can be done using a combination of the other tools available.  I find the lasso tool to be a great place to start.

Lasso tool:

It actually works a lot like Photoshop.

Like the other tools, the lasso tool has options that affect how it works.  I often use it in the default mode (as seen here), but some of the bells and whistles can be interesting.



The 2D Freehand shape, the one I use the most, allows you to draw any shape on the individual CT slice and the lasso tool will select whatever is inside of that ring.  Fairly self explanatory.  If you select the "Auto trace" check box, the tool starts to act more like the magnetic lass tool in Photoshop.

The trace function can snap to the edges of a selection when selected.  This is also where the 'trace edges' check box comes into play.  When checked, it snaps the freehand lasso to the nearest edge, for the most part.  If unchecked, it tends to lasso around the nearest area of contrast.

 

This can be a useful tool for filling in the interior of the bones.  However, I feel like I can get a similar effect while using less time in just 'freehand' mode.  Because there is often a thicker area of bone around the interior, it can be easy to do a quick circle around that.  Of course, I am using a wacom tablet instead of a mouse, so the motion feels very natural to me.

The next option over is the 'ellipse' setting.  This draws a plain circle, or ellipse (the oval shape and size are controlled by you).  I remember using this when I first started out, but I very rarely use it now.  I prefer the control of the freehand.  I suppose it can be a useful tool to interpolate between for selecting a swath of voxels.

And the final option in the 2D portion - the rectangle setting.  Yeah, I never used this. Because I work with very organic shapes, I just have never used it.  Like the ellipse tool, I suppose it would be useful for eliminating voxels along the edge of scans that get selected by accident?  I still use the free hand lasso for that though.  But, it's an option.

A pretty cool part of the lasso tool, and one that I often (regrettably) forget about, is the 3D lasso mode.  It has two options - inside and outside.  With the lasso tool selected, enter into 3D preview mode.  The view in "3D" checkbox needs to be selected next to the material, and a point cloud will be generated based on that material.  With the lasso tool activated, you can draw in 3D space and Amira will interpolate that selection in the Z plane.

You can see here that if you aren't lined up correctly, you may end up killing some of you model inadvertently.  So you'll need to keep an eye out for that.  But, it's a very fast and easy way to deselect those pesky voxels at the edge of a scan.  Anything you select in this mode can be added or subtracted from a material.






Great, so that's how the lasso tool works.  Now, what do I use it for?

Personally, I use it to fill in the gaps, like I said when I first started this post (hours ago, so I had to check again...)  If you fill in one slice, and then choose one many slices down the road, you can fill in large gaps.  It's similar to putting a lid on both sides of a tube.  The tube is now water tight and the selection will stay within those lids.

Hold 'shift' to select multiple areas

Here I've used the Pick & Move tool to selected all of the areas between two slices that I filled in.  With the 'all slices' box checked, Amira will automatically fill in just that area, if it is water tight.  If there is any leak to the outside volume, the selection will leak as well.  These worked pretty well.

Something to keep in mind can be how thick the outer bone is - in things like long bones where that thickness is larger, this process can save huge amounts of time.  In skulls?  Not quite as useful.


In any case, that is the lasso tool in a nutshell.

Amira: threshold tool

I was asked an interesting question today - how do I like Amira as a segmentation program?  Short answer - I do like it.  I used Mimics in Grad school, but I've used Amira exclusively since then.  So, I decided to do a post on the tools I use in Amira 5.3.3 to segment. This is not meant to be a 'how to use everything in Amira' post - just a quick look at what tools I use to do what when I segment. (Just for the record, the user manual is over 500 pages long.  No joke.)  This post may end up being a series of posts...

But before I just jump into the tools, let's touch on Amira in general first.  Amira has two sections that I use the most often.  It opens up into the Object Pool, where you can load your data set and label file.

Mode menu in the upper left of Amira

These are kept separately by Amira, but a data set cannot be viewed in the Segmentation Editor unless it has a label file associated with it.



Switching to the Segmentation Tool view mode, you can see where your scan is loaded in the Image Data area.  You can have multiple label files associated with one scan, and those will be located in the Label Data area.  You can switch between your label files at any time using the drop down list.  If you can't see the scan itself, it's likely you don't have a label file associated with it yet.  Just click the 'new' button to the right of the Label Data area to create one.

Within each label file are 'materials' used to actually segment the scan.


I think of like a series of tracings - you have your original drawing that you are breaking up into layers using tracing paper.  All of the tracings together make up the 'label' with each sheet being an individual material.  The main thing to keep in mind is that every pixel of the scan can only be claimed by one material at a time.  Materials do not overlap.

Your initial label file comes with two materials - the Exterior and the Inside.  Exterior isn't actually a material per say - rather, it's where all of the pixels that aren't in a specific material are kept.

If you were to view the exterior in 3D, it would just be a block.

Here you can see that the 'exterior' material has all of the pixels not currently assigned to the 'inside' material.  It shows as just a block.  It's the only material that I allow to stay as a default assignment.

I did a very fast thresh-holding to show how pixels assigned to an different material will show in 3D space.  Material names and colors can be changed in the material list by double clicking on them.  They can be turned on and off in both the 3D and 2D view ports.  The outline in the 2D view port corresponds to the color of the 3D object and is defined by the color block next to the material's name.

Tool Icons

These are the tools available for basic segmenting.  From left to right: Pick & Move, Brush, Lasso, Magic Wand, Threshold, Blow Tool.  My most used tool?  The brush, hands down.  But I have used all of them, so we'll take a look at examples of when I would use which tool.  I will be going in order used during segmentation though, not actual tool order.

Threshold: Unless I am starting on a data set that has already had some segmentation done, this is my first tool of choice.

The threshold tool allows you to select all the pixels within a certain grey-scale range.  As you can imagine, this is a fast and dirty way of grabbing as much of a certain object at once.  This works best on high contrast material, such as bone.  (It works even better if all you have in a scan is bone, as is the case with this canine skull I am working with right now.  Otherwise the editing grows exponentially.)

Most tools have more options available to change.  Those are located below the tool menu itself.  So the threshold tools allows for both single slice and all slice selection (if you uncheck 'all slices' it will only look at the current slice) and you can threshold within a certain material only.  After setting the threshold, one must select that setting to be able to add it to a material.

You can set what range of pixels will be selected by the threshold tool directly above the tool menu.  Without getting too complicated (Hounsfield units, etc) - there is a small histogram that shows the range of values within the scan: white being on the thousands on the right and representing bone or metal, black going down to -1000 on the left and representing air (0 is water).  Humans can't actually see that large of a variation in grey levels, hence the window width and window level values.

However, to simplify - threshold can select a range of these values.  You can dynamically see what is selected in blue as you adjust the slider.

Once you are satisfied with the selection, you must actually 'select' it to be able to add it to the material.  Once selected, it turns red (and mixes with the blue of the threshold tool to turn a purplish color) and the selection tools turn red on the left.  "Add selected voxels" or the "+" button will begin the segmentation process.








Now, why would you threshold on a single slice only?  Or even within a material?

Both options are useful when trying to separate a single material into more complex selections.  Say you are trying to remove teeth from the whole skull material.  Or bone from a more universal body scan.  Again, this only really works in a whole scale way on areas of higher contrast, such as bone and teeth.

You can see that the black areas are not selected - the insides of the bone will need to be filled in using different tools.  But this gives us a great selection to start from.

And I will look at the other tools another day.  

Segmenting the Canine skull, part 2

I feel like I got a goodly amount of work done on the skull today.  Even if I'm still floundering a bit on where suture lines are. 

Canine skull, end of day 2

I have around 32 different photo references I've pulled from the net, including one where the different bones of the skull have been painted and labeled. 

A very useful powerpoint from the Calgary Association of Veterinary Students.  This is a screenshot from a slide.  They also have the same skull with dorsal, ventral, and rostral views.  I refer to this a lot.



Of course, every skull has some amount of variation.  And I'm not even sure of the breed of the dog scanned.  However, it is a long snout breed, at least.  Those seem to be more common on the internet as well.  I have the feeling that the dog I have is an older example, as sutures tend to become less distinct as individuals age.

I have succeeded in pulling out the mandible, mandibular teeth, nasal bone, incisive bone, zygomatic bone, maxillary teeth and most of the maxillary bone so far. Truthfully, I've been having a hard time even getting a comprehensive list of all of the bones that make up the skull.  It's somewhere between 22 and 50...  Ah, google.

So, I have done three things to rectify my lack of knowledge.  First, a library!  I found a library on Stanford campus that has numerous canine vet books that I should be able to check out.  Second, Amazon! I also found a fairly inexpensive book (creatively called Dog Anatomy, by Peter Goody) on amazon that I decided to buy... I must admit that I turned to Amazon before the library, so technically these are in the wrong order.  However, the prices on Amazon made me sit back and realize that there are much better ways I would like to spend my money, bibliophile tendencies aside.  Which leads me to my third choice, eBay!  Yes, I am now the proud owner of a dog skeleton.  This makes me happier than I probably should be.

My text reference should arrive on Tuesday.  And in the mean time, I've been segmenting. 

Putting the roots on the mandibular teeth.

The newly added mandible and mandibular teeth.

Trying to figure out the nasal and incisive bones.  I can't tell you how tricky those were.

Adding the maxillary bone.  My skull is starting to look colorful!

 

This is often how I work - multiple views, with the 3D preview to the side.  Because I am having such a difficult time spotting the suture lines, I am relying upon a combination of CT and 3D to figure out where different bones are.  This is why I feel an actual skull is so necessary, at least for me.  Then I will be able to trace the sutures on the physical specimen.

I'm off to the library tomorrow, to further my knowledge of canine anatomy.  I do feel a bit nervous, having started this process without hearing from the actual official people... And truthfully I got a little segmentation happy.  I meant to write my proposal this weekend, not segment canine skulls.  So, realistically I should take a step back and figure that out.  Ick.

Segmenting the Canine Skull

Despite not actually heard back from anyone official on the project idea, I decided to do some segmenting.  Yes, this is what I do for fun on a Saturday night.

The bane of my segmenting existence - ethmoid type bones, and turbinates

I am not looking forward to the canine nasal cavity.  There are a lot of delicate structures in there, that are very, very hard to segment out in Amira.

Canine skull, from an awesome person on deviant art

I have started on other, potentially easily to identify bones on the skull.  However, I've already run into issues there.  The scan I am using, while beautiful, doesn't have the clearest suture lines.  I'm switching back and forth between the three views (sagittal, coronal, and axial) and often viewing the scan side by side with the projected 3D model.  This allows me to see my changes in real time, so I can compare with reference texts and photos.

Amira 3D point cloud

Trying to identify the jugal bone

And the bones of the canine skull are different than those of a human.  So, while the experience I gained segmenting out many, many skulls at my previous job is helpful here, I am struggling somewhat with where suture lines are supposed to be.  I would love to get my hands on an actual skull.  Which is an idea to run by my vet expert....

EDIT: 4 hours later, I'm starting to feel some concern.  I chose what I thought would be 'easy' bones to start off with, but... it has been around 3-4 hours of work and I'm just now finishing them up.  And I'm fairly certain that the lacrimal bone is in there somewhere.  I'm hoping that I just misjudged what would be less difficult, and not that the rest of the skull will be even more complex.  I definitely need a physical canine skull.  Photo references are all well and good, but an actual skull would be much more useful.

Zygomatic bones in red.  (It would be helpful if all of my references agreed on the bone names.) 

 

Graduation?

One of the things that I regret the most about my time at UIC: BVIS - not finishing.  I went through the (awesome) program, but I never technically finished.  I left for the great California wilds with the research project undone, and it has truthfully been a source of shame for me.

One of those times when you speak up proudly, but mumble the last bit: "Yes, I went to UIC, *but... mumblemumbleresearchprojectunfinshedmumble*."

So!  I am going to finish.  And to aid in that very doable endeavor, I'm going to blog about it as well.  I know it's going to make for scintillating reading.

To start off, I emailed John, the director of the program at UIC now, to find out if I could still graduate at all.  I figured the worst he could say was 'no.'  But, I recieved both a positive answer and the timeline I needed to work in.  That resolved, I decided to scrap my old research on the "Virtual Heart."

I didn't finish the first time, and I had the feeling that if I tried to continue down that road... it was unlikely to happen this time either.  I need a project that I can really get behind.  One that I am likely to enjoy working on most of the time, to get me through that whole 'have to write a ginormous paper' part.

Not to mention, I would truly have to do the literature over again, as most of the articles I had read at the time are already obsolete.  There have been great strides in the virtual world - personally, I've already learned so much about it!  So, not such a great 'research project' now.

One of the things that John asked me was what would I be researching.  This, surprisingly enough, took me aback.  What would I be researching?  Well.... hmph.  What would I be researching?  What would be novel about my project?  What qualified my idea as research potential?

I had this vague idea that I wanted to do something with canines.  And I wanted to concentrate on the skull.  At least most likely.  I had this nebulous concept of 2D to 3D to real time... but then, what would I be researching?  I needed a second opinion.  And I needed to find out what the situation in the veterinarian world actually was.

This was starting to sound like the beginning of a literature review...

I started to research, and it looked very likely that a new type of education was very much needed in the vet world.  But I really wanted to hear from someone close to a 'source.'  So, I was able to get in touch with a Dr. Spriet, from UC Davis, after a bit of perseverance.  I learned that just because people can get very busy, that doesn't mean they are personally ignoring you.  Don't get discouraged.  We had a great talk about the clinical relevance of different projects, the state of veterinary education now, etc.  And I have a much firmer idea of what type of project I want to do now.

So, now I 'just' need to get cracking on a better proposal and then start my literature review.  Have I mentioned that I dislike writing papers? 

The short proposal of a proposal:

Veterinarian programs have been seeing a significant decrease in the number of hours devoted to animal cadaver dissections, turning instead to DICOM slices, and potentially incorporating pro-sections.  This further complicates the learning of complex areas of anatomy, such as the cranial nerves.  Understanding the cranial nerves and their location is especially important when diagnosing node disease in canines, and has been identified as a key difficult study point.  Currently the anatomy of the cranial nerves in canines is taught using a combination of CT and MR slices with basic annotations, power points, journal papers, and anatomy text books.  Cranial nerves are difficult to identify on CT due to contrast resolution, but the bone window for the skull foramina can be located.  Comparing that with the MR slices allows cranial nerves to be located in the most basic sense of the word.  And with actual dissection of the animals becoming much more rare due to time and money constraints, students find the spatial awareness of where cranial nerves are to be hard to understand.

I propose to use a combination of high resolution CT scans and MRI data to build an interactive 3D model of the bones of a canine skull and it's cranial nerves.  The individual bones would be identifiable, as well as the major landmarks and foramina.  The cranial nerves would be added in using Maya, and also be interactive and identifiable.  The structures can be labeled and annotated to increase student knowledge.  Pertinent labels will be identified using the expert help of Dr. Spriet, from UC Davis.  A slider with the CT and/or MRI slice data (superimposed over the 3D model for comparison) will be included to facilitate learning the spacial cues in both 3D and 2D.

Cranial nerves have not been well identified for veterinary students, especially not in an interactive 3D environment.  Combining the DICOM data with the 3D models is a very interesting area of study, and not well researched at this point.  Pairing the two would be a great step forward in the 3D anatomy world as a whole.  While a few people have been able to achieve the combination of a volume rendering of a CT dataset and a 3D surface model, I have not seen slice data combined with a 3D surface model.  However, this is the more likely way a vet will be viewing the data in the practice.  So, combining these learning tools would be of a great benefit to veterinary students.  The process of this development would be CT/MR segmentation (Amira) to 3D (Maya/3D Coat) to Realtime Interactive app (Unity3D).

Obviously, I still have areas to research for my literature review.  I need to better understand canine cranial nerve anatomy, document 3D as a learning tool, and start to research how to begin to combine DICOM with 3D surface models.

Dr. Spriet was also interested in comparing equine cranial nerves with canine, but I was worried that taking it to that level may be overreaching on this project.  I think I could do the segmentation, but I have the feeling that researching and coding the combination of CT and surface models will be an interesting undertaking.  I may be completely wrong, but that would be my initial assessment.