sketcher: An R package for photo to sketch effects

Hiroyuki Tsuda

By using the sketcher package, you can convert a photo into a line drawing image.
Drawing style (lineweight and inclusion/exclusion of shadow/shading) can be controlled.
Some examples (source photos and generated sketches) are shown below.

Paper

Details of this package are described in the article below:

Tsuda, H. (2020). sketcher: An R package for converting a photo into a sketch style image. PsyArXiv Preprints.
https://doi.org/10.31234/osf.io/svmw5

Dependencies

Mac OS X users need to install XQuartz (https://www.xquartz.org/).

Installation

The package is available on CRAN. It can be installed with:

install.packages("sketcher")

Then, attach the package.

library(sketcher)

(Optional) Installation via GitHub

To install the development (latest) version of the sketcher package via GitHub, you can use the devtools package.

NOTE:
- On Windows, Rtools needs to be installed to install a package from GitHub.
- On Mac, XCode may be needed to be installed to install a package from GitHub.

devtools::install_github("tsuda16k/sketcher")

Example image

The sketcher package has a built-in image, which is useful when you want to try sketch effects right away. The image is named face.

Internally, this is just a numeric array of size 600 x 460 x 3 [y-coordinate, x-coordinate, color channel]. Each element of the array represents a pixel value, which can range between 0 and 1.

dim(face)
# [1] 600 460   3

To plot an image, use the plot() function.

plot(face)

(In examples below, I actually used this image, which has higher resolution than the image provided by the package. Sketching results will be similar, but in general aesthetically more pleasing results can be obtained with images of higher resolution, at the cost of processing time.)

Load an image

To load your own image, use the im_load() function.

im = im_load("path/to/your/image.jpg")
plot(im)

The jpg, png, and bmp formats are supported.

You can load an image from the web (URL). For example,

im = im_load("https://htsuda.net/pages/sketcher/sketcher_face.jpg")

will load a high resolution version of the face image noted above (compare dim(face) and dim(im)).

Apply the sketch effect

The built-in face image is used in the following examples.
For consistency purposes, the face image is labeled as im.

im = face

Use the sketch() function to apply the sketch effect.

im2 = sketch(im) # may take some seconds
plot(im2)

Basically, that’s it.

The sketch() function has several parameters to control the style of sketch.

Arguments of the sketch() function

A table of arguments of the sketch() function:

Argument	Meaning	Value	Default
im	An input image	image
style	Sketch style	1 or 2	1
lineweight	Strength of lines	a numeric, >= 0.3	1
smooth	Smoothness of texture	an integer, >= 0	1
gain	Gain parameter	a numeric betw 0 and 1	0.02
contrast	Contrast parameter	a numeric, >= 0	20 (for style1) or 4 (for style2)
shadow	Shadow threshold	a numeric betw 0 and 1	0.0
max.size	Max resolution of output	an integer, > 0	2048

The default is sketch(im, style = 1, lineweight = 1, smooth = ceiling(lineweight), gain = .02, contrast = NULL, shadow = 0, max.size = 2048).

- im: an image, obtained by using the im_load() function.
- style: while style 1 focuses on edges, style 2 also retains shading.
- lineweight: as the name suggests. set a numeric value equal to or larger than 0.3.
- smooth: noise/blob smoother. set an integer value equal to or larger than 0.
- gain: this parameter may be useful for noise reduction in the dim region of a photograph.
- contrast: contrast of the sketch image is adjusted by this parameter.
- shadow: if given a value larger than 0 (e.g., 0.3), shadows are added to sketch.
- max.size: the size (image resolution) of output sketch is constrained by this parameter. if the input image has a very high resolution, such as 20000 x 10000 pixels, sketch processing will take a long time. In such cases, the algorithm first downscales the input image to 2048 x 1024 pixels, in this case, and then apply the sketch effect.

The effects of these parameters on sketch appearances are described in detail below.

Saving the image

Use the im_save() function to save an image.

im = face
im2 = sketch(im, style = 1, lineweight = 1, shadow = 0.3)

# myimage.png is saved in the current working directory
im_save(im2, name = "myimage", path = getwd())

# newimg.jpg is saved to a specified directory
im_save(im2, name = "newimg", path = "set/your/path", format = "jpg", quality = .95)

By default, an image is saved in the png format. When using format = "jpg", you can set the quality of jpg compression (default is 0.95).

The effects of sketch parameters

- style and lineweight

The most important parameters of the sketch() function would be style and lineweight.

im1 = sketch(im, style = 1, lineweight = 3) # style 1, with lineweight of 3
im2 = sketch(im, style = 2, lineweight = 3) # style 2, with lineweight of 3

While style 1 is good at extracting fine edges, style 2 retains shading, as shown in the figure below. Note that the shading gets more salient when lineweight is given larger values.

- smooth

The smooth parameter controls the degree of smoothness of image texture. By increasing the smooth value, fine-scale edges, noises, and blobs are eliminated (see the figure below).

In most cases, aesthetically pleasing results will be obtained when smooth parameter is equal to or larger than lineweight. If smooth parameter is not given in the sketch() function, it is assigned with the same value as the lineweight parameter (actually, ceiling(lineweight) is assigned to the smooth parameter because it must be an integer value).

- gain

A constant value (gain) is added to an input image before the extraction of edges to produce a sketch. A sketch can be very noisy at dark/dim regions of the input image. In such cases, increasing the gain parameter, such as gain = 0.2 or gain = 0.3, may reduce the noise. In most cases, however, you don’t have to care about this parameter.

- contrast

By increasing the contrast parameter, a sketch is darkened. When a sketch appears whitish, you may need to increase the contrast value.

im1 = sketch(im, contrast = 10)
im2 = sketch(im, contrast = 30)

- shadow

Shadow can be added to a sketch by using the shadow parameter. By default, the sketch function does not include shadow (shadow = 0). In many cases, however, adding shadow will be needed to produce a reasonable result (described later).

im1 = sketch(im, shadow = 0.3) # add shadow to a sketch

Tips for successful sketching

For some images, good results may be obtained with the default parameters of the sketch function. However, in many cases, the default sketch will produce an unsatisfactory result. Here I show some cases where the default sketch fails, and explain how to fix it.

Case 1

Outline is missing and texture is lacking in the default sketch. By using style 2 and adding shadow, sketch(im, style = 2, shadow = 0.4), the problems are largely solved. In addition, by setting the smooth parameter to 0, sketch(im, style = 2, lineweight = 1, smooth = 0, shadow = 0.4), the sketch succeeded in representing the fine texture of the bird body (click/tap and enlarge the image below to see the effects).

Case 2

Due to the lack of edges in the dark region of the face, the default sketch, sketch(im), produced a weird result. This can be fixed by adding shadow, such as sketch(im, shadow = 0.4). Finding the right value of the shadow parameter usually requires trial and error, but usually a value between 0.3 and 0.6 will produce a reasonable result.

Case 3

The sketch algorithm detects edges in an image. If objects have unclear edges/outlines, sketching will fail. This typically happens when a photo contains 1) out of focus regions or 2) objects with unclear outlines such as furry animals.

A cat image below is such an example. The background of the photo and the lower body of the cat is out of focus, and the outline of the cat is blurry. The default sketch, sketch(im), was unsuccessful in representing the cat body. Zero smoothing, sketch(im, smooth = 0), enabled capturing the hairs/textures of the cat to some extent, at the cost of increased noise in background, and still the body and legs of the cat is not represented well. Unfortunately, there is no simple solution to the difficulty with blurred/defocused images.