Always looking for some ways to learn the computer to ‘see’, I came accross two very interesting algorithms: ‘Real-time Contrast Preserving Decolorization’ by Cewu Lu, Li Xu and Jiaya Jia. And Potrace by Peter Selinger
Decolorization – the process to transform a color image to a grayscale one – is a basic tool in digital printing, stylized black-and-white photography, and in many single channel image processing applications. RTCPD is aiming at maximally preserving the original color contrast alleviate a strict order constraint for color mapping based on human vision system.
The second algorithm Potrace(TM) is a tool for tracing a bitmap, which means, transforming a bitmap into a smooth, scalable image.
Playing around with both algorithms, I came up with the idea to use them for a project I had in mind for some time: Sketching pictures on my wall using a Pico projector.
First I had to implement both algorithms in Xojo. The result of RTCPR can be found in the attached source code in the RTCPRGB2Gray() method:
Private Sub RTCPRGB2Gray(pic as Picture) dim imcols as Integer = pic.Width dim imrows as Integer = pic.Height dim sRGB as RGBSurface = pic.RGBSurface dim Mat(-1,-1) as ABDoubleVector redim Mat(imrows-1,imcols-1) dim c as Color for y as Integer = 0 to imrows - 1 for x as Integer = 0 to imcols - 1 c = sRGB.Pixel(x,y) mat(y,x) = new ABDoubleVector(c.Blue, c.Green, c.Red) ' switch to BRG next next dim s as Double = 64.0 / sqrt(imcols*imrows) dim cols as Integer = (s * imcols) + 0.5 dim rows as Integer = (s * imrows) + 0.5 dim sigma as Double = 0.05 dim W() as ABDoubleVector = wei() dim P() as ABDoubleVector dim det() as Double dim pos0() as ABDoublePoint dim pos1() as ABDoublePoint for i as Integer = 0 to cols - 1 for j as Integer = 0 to rows - 1 dim x as Integer = (i+0.5) * imcols/cols dim y as Integer = (j+0.5) * imrows/rows pos0.Append new ABDoublePoint(x,y) pos1.Append new ABDoublePoint(x,y) next next pos1.Shuffle for i as Integer = 0 to pos0.Ubound dim c0 as ABDoubleVector = mat(pos0(i).y, pos0(i).x) dim c1 as ABDoubleVector = mat(pos1(i).y, pos1(i).x) add c0,c1, P, det next cols = cols / 2 rows = rows / 2 for i as Integer = 0 to cols - 2 for j as integer = 0 to rows - 1 dim x0 as integer = (i + 0.5) * imcols / cols dim x1 as integer = (i + 1.5) * imcols / cols dim y as integer = (j + 0.5) * imrows / rows dim c0 as ABDoubleVector = mat(y, x0) dim c1 as ABDoubleVector = mat(y, x1) add(c0, c1, P, det) next next for i as integer = 0 to cols - 1 for j as Integer = 0 to rows - 2 dim x as integer = (i + 0.5) * imcols / cols dim y0 as integer = (j + 0.5) * imrows / rows dim y1 as integer = (j + 1.5) * imrows / rows dim c0 as ABDoubleVector = mat(y0, x) dim c1 as ABDoubleVector = mat(y1, x) add(c0, c1, P, det) next next dim maxES as Double = -99999999999 dim bw as Integer for i as integer = 0 to w.Ubound dim es as Double for j as integer = 0 to p.Ubound dim l as Double = P(j).x * W(i).x + P(j).y * W(i).y + P(j).z * W(i).z dim detM as Double = det(j) dim a as double = (L + detM) / sigma dim b as double = (L - detM) / sigma es = es + log(exp(-a * a) + exp(-b * b)) next es = es / (p.Ubound+1) if eS > maxEs then maxEs = Es bw = i end if next dim result(-1,-1) as Double redim result(imrows-1,imcols-1) for y as Integer = 0 to imrows - 1 for x as Integer = 0 to imcols - 1 result(y,x) = mat(y,x).z*W(bw).x + mat(y,x).y*w(bw).y+0 result(y,x) = mat(y,x).x*W(bw).z + result(y,x)*1+0 sRGB.Pixel(x,y) = rgb(result(y,x),result(y,x),result(y,x)) next next bw = 0 End Sub
Potrace is a complete framework to trace an image and converting it to Xojo did take some time. While writing it, I also made some changes to it to optimize the result. One of it was doing my own thresholding using the surrounding pixels to smoothen the edge detection.
dim v, v1, v2, v3, v4, v5, v6, v7, v8 as Integer dim ex,ey, weight, weighttotal, total as Double 'SISThreshold2 alain for y = 1 to H - 2 for x = 1 to W - 2 v = sDebugPicRGB.Pixel(x, y).red v1 = sDebugPicRGB.Pixel(x+1, y).red v2 = sDebugPicRGB.Pixel(x-1, y).red v3 = sDebugPicRGB.Pixel(x, y+1).red v4 = sDebugPicRGB.Pixel(x, y-1).red v5 = sDebugPicRGB.Pixel(x+1, y-1).red v6 = sDebugPicRGB.Pixel(x-1, y-1).red v7 = sDebugPicRGB.Pixel(x+1, y+1).red v8 = sDebugPicRGB.Pixel(x-1, y+1).red ex = (v1+v2+v5+v6)/4 ey = (v3+v4+v7+v8)/4 if ex > ey then weight = ex else weight = ey end if weighttotal = weighttotal + weight total = total + weight * (v) next next dim Threshold as Integer = 128 if weighttotal <> 0 then Threshold = total/weighttotal end if dim ret as Boolean = (sDebugPicRGB.Pixel(0, 0).Red >= Threshold) for y = 0 to H for x = 0 to W c = sDebugPicRGB.Pixel(x, y) if c.red >= Threshold then Matrix(x+1, y + 1) =true else Matrix(x+1, y + 1) = false end if next next for x = 0 to MatrixW Matrix(x,0) = true Matrix(x, MatrixH) = true next for y = 0 to MatrixH Matrix(0,y) = true Matrix(MatrixW, y) = true next
This gave me all the tools to start tracing an image. But for my project I wanted to have a sort of ‘sketchy’ feeling. For this I needed a couple of things because I wanted the drawing and painting to be done gradually. I had to build it with delays, repetition, randomization and color fading. I wanted to start from a picture and create something like this:
DrawImage() does all that. It first draws the beziers with some little random mistakes in grey. A gradual painting is then used to fill in the polygons. Finaly it repeats the process with the correct lines in white.
You may notice that a lot of code is redundant, but this is intended because I needed to have enough delays to mimic a more natural feeling.
Here is a demo video showing the result of this project on my wall. It takes about ten seconds before it starts drawing as I had to grab my camera :-). Five different images are drawn.
The project with full source code can be downloaded here