Conceptually, it isn’t too bad: given an observed blurry image y, find the corresponding unobserved (latent) sharp image x.  The complexity stems from the noise in the problem (and there’s always noise), the challenge in deciding when an image is “sharp,” and the overall vagueness of the problem.

The problem is usually modeled using an equation like

y = x ⊗ h + n

where y is the observed blurry image, x is the unobserved sharp image (the recovery of which is our goal), n is noise, ⊗ is the convolution operator, and h describes the blur (commonly known as the Point Spread Function, or PSF).  Since we’re solving for x, and we know only y, the problem is clearly ill-posed: we have three unknowns and only one equation.  What to do?

The problem can be helped somewhat by imposing constraints on the variables.  Here’s an example.  For simplicity, assume that all of the variables are scalars and * is the multiplication operator.  If we are told that

41 = x * h + n

and nothing more, then there are many valid solutions.  For example, a valid solution is x=7, h=3, and n=20.  Another equally valid solution could be x=41, h=1, and n=0.  However, if we are told instead that

41 = x * h + n

s.t.  x is even, 2 < h < 5, and n ∈ {0,1}

then we can clearly see that the only valid solution satisfying the constraints is x=10, h=4, and n=1.

Similarly, the challenge in solving the deblurring problem is imposing the proper constraints on the variables.  In the image deblurring problem, the variables are typically represented as matrices, which may each have millions of elements, which means that there are, essentially, millions of unknowns.  A brute-force, exhaustive search of the solution space — trying every possible combination of values — is completely infeasible.  We need some constraints.

Fortunately, we do know a few things about the sharp image, PSF, and noise that can get us pointed in the right direction.  First, the image values are bounded.  On computers, in files like JPEGs and PNGs, every pixel, for every channel (red, green, and blue), has an integer value between 0 and 255, inclusive; that is, there are 8 bits per pixel (2^8 = 256), and 3 channels, leading to 24 bits of data per pixel, or 24-bit color.  That’s one constraint: all of the pixels in our blurry and sharp images need to have values between 0 and 255, inclusive.

A second constraint, this time on the PSF, is that the PSF is energy-conserving.  In other words, when the PSF blurs the image, it doesn’t add or remove any energy to the pixel, so instead of a single bright pixel, you might instead get several dim pixels that sum to the same value as the bright pixel.  This tells us two things: first, the range of element values in the PSF is between 0 and 1, inclusive; and second, the sum of all of the elements in the PSF must equal exactly 1.

Constraints on the noise term can be a bit tricky, and tend to be chosen for simplicity.  Fortunately, the model tends not to be overly sensitive to the accuracy of the noise (noisy noise!), so a Gaussian or Poisson distribution is often chosen, with 0 mean and a standard deviation chosen based on empirical observation of known images.  The noise term has a lot to absorb: optical imperfections, inconsistencies across the frame, sensor noise, quantization noise, compression noise, and more.

From there, however, the problem gets really tricky.  How do we know when we’ve found the sharp image?  How do we solve a system with so many variables?  The plot thickens…

All of modern civilization is built upon the effort and productivity of generations past.  So it goes in the software world.  For my quasi-stealth-mode computational photography project, I’m making use of some great tools:

• Linux — Operating system.  Free.  Fast.  Familiar.
• Apache — Web server.  Can be overkill, but it does a good job integrating with…
• Ruby on Rails — Framework.  Ruby is a wonderful, modern language.  Rails is getting increasingly mature.   Also using a bunch of packages and related tools, like Mongrel, which make life easier.
• Aptana/RadRails — Integrated Development Environment (IDE).  Specifically, RadRails.  Development for Rails in Aptana is made easier by the underlying Eclipse engine, the cooperation with the Ruby debugger, and the decent integration with…
• Subversion — Version control.  Because I had it installed, Rails plays nicely with it, and I haven’t had a chance to get familiar with Git.
• Octave — Numerical computation.  Very similar to Matlab in its vector-based programming language, but it has some nice features that make it particularly well-suited for my needs.  Also, it’s free, whereas acquiring Matlab would force me to mortgage my firstborn child.
• FogBugz — Bug tracking.  And scheduling.  I thought I should give it a spin, seeing as how several of my friends from Rose and Stanford work at Fog Creek.
• Amazon Flexible Payments Service — Payments.  A business is about money, and money requires being paid.  Amazon FPS makes accepting credit cards easier than, say, getting a merchant account and putting together a PCI-compliant server.   That, and Amazon doesn’t have the stigma of PayPal.

It hasn’t all been a smooth ride.  Aptana crashes at least twice a day.  I’m still in the process of learning Ruby (which is one of my motivations for using it).  I’ve had to patch bugs in Octave and Rails.  That said, it’s easier than doing everything from scratch in, say, C++.

At first, I felt a bit guilty for standing on the shoulders of others.  Hell, the entire motivation behind the project is based on decades of academic research.  But like the architects who use the steel alloys of others to build their skyscrapers, and the auto manufacturers who use minerals mined by others in the distant corners of the globe, I too will take pieces from others and combine them to make 2+2=5.  Perhaps one day, somebody will incorporate my work into their own, thus continuing the great cycle of humanity.