CMPS290B - Computational photography and vision on mobile devices

Instructor:    Prof. James Davis
Location:      JBE 372 
Time:            MW 3:30-5pm 


Getting Started with Symbian C++

Installation procedure (2007-10-24)


Direct to eBay (Josh McCoy and Gillian Smith)

Virtual Touch-Pad Project (Orazio Gallo and Sonia Arteaga)

The Transient Rendering (James Skorupski and Adam Smith) - [VIDEO]

Recording a Game of Go (Steven Scher and Ryan Crabb)

Aerial Photography Using a Nokia N95 (Dave Ilstrup and Mariano I. Lizarraga)

Image Recognition on Mobile Phones (Serdar Sali)

PhotoExplorer (Feng Tang)

Depth Sensing using Time Of Flight Principle (Neeraj Kumar)

Product Info via Cellphone (Shane Brennan)


Digital cameras are producing a dramatic revolution in the way people
think about image formation. Digital cameras are no longer (or soon won't
be) merely replacements for film cameras. Instead they are sensors with
pixels that record *millions* of samples of information per frame. By
_computing_ on this information, we can build new scientific sensors and
new tools for artists to better control their image than an standard
camera would provide. Mobile devices are also producing a revolution in
_where_ we do our computing and the tasks that we can do. Traditional
interaction techniques such as keyboards and mice map poorly to phones,
however the cameras embedded in the phones provide a possible new method
of interaction.

The course readings will be focused in the areas of computer graphics,
machine vision, and image processing. The discussion, projects, and ideas
will be somewhat broader since I think this area is ripe for revolutionary
new applications. Indeed, our goal in this course is to read papers,
brainstorm, and figure out the next kill app!

There is a "hidden" agenda to this class which is to both teach and give
hands on experience in the process of doing good research. We'll be
reading a lot of recent research papers. Students taking the course for
credit will need to complete a project and write a "conference" style
paper describing their work. We'll be reviewing each others papers and
holding a mini-conference in class. Papers _will_ be submitted to a real
conference. Past experience suggests that you should expect a 50% accept
rate. In the past, grad students in their first few years really like this
part of the course, but some more senior students think its obvious how to
do good research and therefore unneeded.

Nokia Research Labs has agreed to provide some 'mobile devices', some
support using the SDK for programming them, some research mentoring, and
some hiring of interns later on. I anticipate that many students will like
to make use of these devices and opportunities, but it will not be
strictly required.



  • For every class:
        Read paper.
        Prepare follow on ideas about paper .
  • For every week:
        Find an (important) reference for your project / per person.
  • Presentation once during the quarter:
        Represent a paper as if it was your own work
        (We will use recorded video of the actual author presenting)
        Answer questions about the work and any background methods
        Give a suggestion for some sort of future work that would make a good research paper
  • Project Schedule

Final Documentation

    Final presentation

How to review a paper











    (project ideas)





    Seam carving for content-aware image resizing
    [more discussion of projects]




    [James travelling]





    Guest Andrew Adams gives talk on stabalization on cel phone





    More project planning





    Discussion: what goes into related work
    Scene completion using millions of photographs

Team proposals / Work Timeline






Photo tourism: exploring photo collections in 3D



Multiscale shape and detail enhancement from multi-light image collections

[Schedule team meetings]





Team Meetings:
3:00PM - 4:00PM: Gillian Smith, Josh McCoy

4:30: Serdar Sali





    Whats in an intro 

Multi-aperture photography??????


Image and depth from a conventional camera with a coded aperture????????

(Discussion Leader: Orazio Gallo)

related work (at least 10 papers)





Team Meetings:

4:00 PM - 4:30 PM: Adam Smith, James Skorupski

4:30 PM - 5:00 PM: Dave Ilstrup, Mariano Lizarraga

5:00 PM - 5:30 PM: Orazio and Sonia





No class - James Davis in Taipei





No class - James Davis in Taipei


"related work" is reviewed/fixed by classmates
intro - contribution statement




  Image-based tree modeling - Adam Smith

Capturing and animating occluded cloth

(Discussion Leader: James Skorupski)






Team Meetings:

3:30 PM – 4:00 PM: Dave Ilstrup, Mariano Lizarraga

4:30 PM – 5:00 PM: Serdar Sali





    Discussion: what goes in supporting figures 


VideoTrace: rapid interactive scene modelling from video (Discussion Leader: Gillian Smith)

supporting figures planned
"intro" is reviewed/fixed by classmates
related work is updated




    Discussion: what goes in a presentation


Veiling glare in high dynamic range imaging(Discussion Leader: David Ilstrup)





(team meetings)






 "supporting figs" reviewed/fixed by classmates




    Feedback on related work in general








Team Meetings:
3:30 PM - 4:00 PM: Gillian Smith, Josh McCoy
4:00 PM - 4:30 PM: Adam Smith, James Skorupski
4:30 PM - 5:00 PM: Orazio and Sonia





    Feedback on intro in general


Factored time-lapse video

(Discussion Leader: Sonia Arteaga)

full paper






Soft scissors: an interactive tool for realtime high quality matting

(Discussion Leader: Feng Tang)



Image deblurring with blurred/noisy image pairs

(Discussion Leader: Josh McCoy)





(team meetings)






"full paper" reviewed/fixed by classmates
Video draft






Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs



Computational time-lapse video





(team meetings)





    Your Project Presentations
    Evals for end of quarter

Video revised/Presentations




    Your Project Presentations

The Transient Photometric Response Function [Adam and James]

PhotoExplorer [Tang]




    Your Project Presentations

Virtual Touch-Pad [Orazio and Sonia(12/07/07)]



UAV Image Registration [Mariano and Dave]
Direct to eBay [Gillian and Josh]
Depth Camera [Neeraj]

Image Recognition on Mobile Phones [Serdar]









    Finals Slot (???)

Show your work presentations
Revised paper (~4 page)




Team proposal

Find teammates. Decide on a project. Create a web page (not wiki) linked from the class web page and _in_ the class directory. The web page should include sections for Abstract and References. The abstract should be a paragraph description of the project which includes a contribution. Write as if the project is completed and you are describing your work. References should be annotated. Its sufficient to include one sentence on what the paper is about, and one sentence on why its related to your project. Also annotate who added each reference.


Every paper we read

You should think about ideas about how to extend each paper we read, and what might make a good follow-on research paper. Don't just copy the future work section, think of how it might be combined with your own work, or the paper we read last week, or something else. We'll discuss these ideas in class. Turn in a description of your idea written up in 1/2 page or less on a physical sheet of paper in class.



You will write your paper in stages: related work, intro, figure and results, everything else. You will get your classmates to help revise your work before I see it. After we have the most important section revised we will put the whole thing together. Its important to learn how to write by reading other peoples work. I believe you will find it much easier to help your classmates see their mistakes than it is for you to see your own.


When you work on most graphics related topics its critical to make a video. Its impossible to evaluate your work without it. The video gets submitted with a paper. At a bare minimum the video shows the results of your work. However its good practice to make the video tell the whole paper story. If someone only watched your video they would know why your problem is important, what your key contribution is, a brief overview of how you did it, and of course your impressive results. Think of the video as an advertisement to read your paper. 

Make a video that is 2-5 minutes in length. Use any video editing program you like. Encode the video as MP4 at 640x480. Link to your web page.



You will give a group presentation in class twice. Once as a draft, and again at the final time. The presentation should have the same organization as your paper: motivation, contribution, relation to previous work, what-you-actually-did, results, conclusion. In a real conference presentation, previous work is often dramatically condensed, in our class we’re going to leave it in because its critical to properly explaining what your contribution is.

In addition to content. Presentation style is important. Use pictures, or videos. Look at the audience. Etc.

Make a presentation. Link it to your project page. Deliver it in class.