On Sun, 2015-01-25 at 02:57 +0100, Krzysztof Kosiński wrote:
Hello
For one of the courses I am doing at the university, I want to create a pixel art cleaner which would complement the recent GSoC pixel art vectorizer. It is a well known problem that most pixel art images available on the Internet are not appropriate inputs for the vectorizer: they have multiplied pixels, additional grids, various artifacts, defacements with watermarks, etc. Here are a few examples:
- Multiplied pixels only:
http://research.microsoft.com/en-us/um/people/kopf/pixelart/supplementary/re...
- Border, grid, JPEG artifacts:
http://img.wonderhowto.com/img/56/01/63456484792752/0/make-pixel-art-minecra...
- Grid, imperfect alignment:
http://fc04.deviantart.net/fs70/i/2013/258/6/b/link_pixel_art_grid_by_matbox...
- Grid, imperfect alignment, defacement:
http://1.bp.blogspot.com/-jOu8vhgnSFQ/U7sQV0HSRwI/AAAAAAAACEs/2lGHanqeF90/s1...
- Grid, nonuniform pixel colors:
http://fc06.deviantart.net/fs71/f/2011/227/6/4/link_pixel_art_by_malandrus-d...
- Scan of a hand-filled grid paper:
http://th04.deviantart.net/fs71/PRE/f/2013/207/5/b/goku_pixel_art_by_hideman...
- Photographs:
http://fc03.deviantart.net/fs71/i/2013/336/1/6/glumanda_pixel_art_by_beffra-... http://farm8.staticflickr.com/7258/7434425816_9b810a3b6d_z.jpg
I'm looking for a few ideas on how to design an algorithm that would automatically convert these images to a clean input to the pixel art vectorizer. Obviously any successful algorithm will exploit three basic facts about pixel art:
- The pixels are laid out in a rectangular grid.
- Color within a pixel is uniform.
- The color palette is limited.
My first guess is to use a GSL and find the parameters of the pixel grid by using a generic optimizer. The cost function could include the variance of color within a pixel, area of the pixels (larger pixels should be preferred), and a few other parameters. After identifying the parameters of the pixel grid, the colors of the pixels would be found by averaging and then clustering (to eliminate JPEG artifacts and defacements).
Initially I considered using FFT methods, but this would never work for images such as 5, 6 or 7, and ideally I want to cover them as well. Another possibility is to use the Hough transform to find sharp lines in the image and then look for regularities in the lines.
Does anyone have more ideas?
No. 7 needs an extra step of transforming but a FFT should work directly on the others. I imagine there is photography software that can do the transforming of 7 automatically.
I am not sure why you think FFT won't work on 5. You have the background grid which should provide a large amplitude for the grid frequency. Even without the background, the shading within the pixel is like a sawtooth function where the grid frequency should still be the largest component.
FFT should also work on 6. The roughness along the edges of the cells shouldn't over-weigh the dominant frequency.
To remove defacement, histogram the color of the pixels within the cell and take the color bin with the largest component. You can then compare that value with the value for neighboring cells or other cells in the drawing as an additional check.
This looks like a fun project! Keep us posted.
Tav