Smart Photo Cropping

Using composition rules to automatically crop human-annotated photographs

Project Members

  • Dustin Escoffery, Graduate
  • Wai Son Wong, Undergraduate
  • Scott Orzech, Undergraduate
  • Josh Lehman, Undergraduate
  • Joe Chen, Undergraduate

Objective

We want to design a system that can help people become better photographers. The ideal application would be a "smart camera" that provides artistic suggestions directly in the viewfinder. The problem is, it is unrealistic to teach a computer to be an expert photographer. Fortunately, many photography techniques are simplified into easy-to-apply rules.

Balance Rule-of-thirds Spacing

We consider three rules for measuring composition. Balance is defined by the proximity of the center of mass to the middle of the image. Rule-of-thirds represents distance to any thirds-line. Spacing relates to the distance between objects and the borders of the frame.

Our hypothesis is that basic composition rules offer an improvement to the average person's photos. To test this hypothesis, we construct an image cropping experiment that approximates the choices made in photography.

Procedure

  1. Collect 25 panoramic images
  2. Develop a tool for manually cropping/annotating images
  3. Collect 15 crops and 4 labels per panorama [375 / 100 total]
  4. Develop an algorithm to automatically crop a labeled image
  5. Generate 15 crops per label, per panorama [1500 total]
Collecting Human Data

We developed a program called CropTool (download) to facilitate image annotation. Written in C++ using the Qt toolkit, this application allows users to draw bounding boxes around salient elements of an image. The data is exported to a custom label format representing a vector of rectangles.

Collecting Computer Data

We developed a program called SmartCrop to automatically crop a labeled image. Written in C# using the .NET framework, this application uses a Monte Carlo algorithm to simulate potential crops. The candidates are scored and pruned based on a composition heuristic developed in concurrent research (see Panorama project).

Results

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Human Crops (top) Annotated Image (middle) Smart Crops (bottom)

Links

UC Santa Cruz, Computer Graphics, Fall 2010