Conceptually, it isn’t too bad: given an observed blurry image y, find the corresponding unobserved (latent) sharp image x.  The complexity stems from the noise in the problem (and there’s always noise), the challenge in deciding when an image is “sharp,” and the overall vagueness of the problem.

The problem is usually modeled using an equation like

y = x ⊗ h + n

where y is the observed blurry image, x is the unobserved sharp image (the recovery of which is our goal), n is noise, ⊗ is the convolution operator, and h describes the blur (commonly known as the Point Spread Function, or PSF).  Since we’re solving for x, and we know only y, the problem is clearly ill-posed: we have three unknowns and only one equation.  What to do?

The problem can be helped somewhat by imposing constraints on the variables.  Here’s an example.  For simplicity, assume that all of the variables are scalars and * is the multiplication operator.  If we are told that

41 = x * h + n

and nothing more, then there are many valid solutions.  For example, a valid solution is x=7, h=3, and n=20.  Another equally valid solution could be x=41, h=1, and n=0.  However, if we are told instead that

41 = x * h + n

s.t.  x is even, 2 < h < 5, and n ∈ {0,1}

then we can clearly see that the only valid solution satisfying the constraints is x=10, h=4, and n=1.

Similarly, the challenge in solving the deblurring problem is imposing the proper constraints on the variables.  In the image deblurring problem, the variables are typically represented as matrices, which may each have millions of elements, which means that there are, essentially, millions of unknowns.  A brute-force, exhaustive search of the solution space — trying every possible combination of values — is completely infeasible.  We need some constraints.

Fortunately, we do know a few things about the sharp image, PSF, and noise that can get us pointed in the right direction.  First, the image values are bounded.  On computers, in files like JPEGs and PNGs, every pixel, for every channel (red, green, and blue), has an integer value between 0 and 255, inclusive; that is, there are 8 bits per pixel (2^8 = 256), and 3 channels, leading to 24 bits of data per pixel, or 24-bit color.  That’s one constraint: all of the pixels in our blurry and sharp images need to have values between 0 and 255, inclusive.

A second constraint, this time on the PSF, is that the PSF is energy-conserving.  In other words, when the PSF blurs the image, it doesn’t add or remove any energy to the pixel, so instead of a single bright pixel, you might instead get several dim pixels that sum to the same value as the bright pixel.  This tells us two things: first, the range of element values in the PSF is between 0 and 1, inclusive; and second, the sum of all of the elements in the PSF must equal exactly 1.

Constraints on the noise term can be a bit tricky, and tend to be chosen for simplicity.  Fortunately, the model tends not to be overly sensitive to the accuracy of the noise (noisy noise!), so a Gaussian or Poisson distribution is often chosen, with 0 mean and a standard deviation chosen based on empirical observation of known images.  The noise term has a lot to absorb: optical imperfections, inconsistencies across the frame, sensor noise, quantization noise, compression noise, and more.

From there, however, the problem gets really tricky.  How do we know when we’ve found the sharp image?  How do we solve a system with so many variables?  The plot thickens…