CMPS290B - Computational photography and vision on mobile devices

Instructor:    Prof. James Davis
Location:      JBE 372 
Time:            MW 3:30-5pm 
 

 

Getting Started with Symbian C++

Installation procedure (2007-10-24)

Projects

Direct to eBay (Josh McCoy and Gillian Smith)

Virtual Touch-Pad Project (Orazio Gallo and Sonia Arteaga)

The Transient Rendering (James Skorupski and Adam Smith) - [VIDEO]

Recording a Game of Go (Steven Scher and Ryan Crabb)

Aerial Photography Using a Nokia N95 (Dave Ilstrup and Mariano I. Lizarraga)

Image Recognition on Mobile Phones (Serdar Sali)

PhotoExplorer (Feng Tang)

Depth Sensing using Time Of Flight Principle (Neeraj Kumar)

Product Info via Cellphone (Shane Brennan)

 

Digital cameras are producing a dramatic revolution in the way people
think about image formation. Digital cameras are no longer (or soon won't
be) merely replacements for film cameras. Instead they are sensors with
pixels that record *millions* of samples of information per frame. By
_computing_ on this information, we can build new scientific sensors and
new tools for artists to better control their image than an standard
camera would provide. Mobile devices are also producing a revolution in
_where_ we do our computing and the tasks that we can do. Traditional
interaction techniques such as keyboards and mice map poorly to phones,
however the cameras embedded in the phones provide a possible new method
of interaction.

The course readings will be focused in the areas of computer graphics,
machine vision, and image processing. The discussion, projects, and ideas
will be somewhat broader since I think this area is ripe for revolutionary
new applications. Indeed, our goal in this course is to read papers,
brainstorm, and figure out the next kill app!

There is a "hidden" agenda to this class which is to both teach and give
hands on experience in the process of doing good research. We'll be
reading a lot of recent research papers. Students taking the course for
credit will need to complete a project and write a "conference" style
paper describing their work. We'll be reviewing each others papers and
holding a mini-conference in class. Papers _will_ be submitted to a real
conference. Past experience suggests that you should expect a 50% accept
rate. In the past, grad students in their first few years really like this
part of the course, but some more senior students think its obvious how to
do good research and therefore unneeded.

Nokia Research Labs has agreed to provide some 'mobile devices', some
support using the SDK for programming them, some research mentoring, and
some hiring of interns later on. I anticipate that many students will like
to make use of these devices and opportunities, but it will not be
strictly required.

 

Requirements

  • For every class:
        Read paper.
        Prepare follow on ideas about paper .
  • For every week:
        Find an (important) reference for your project / per person.
     
  • Presentation once during the quarter:
        Represent a paper as if it was your own work
        (We will use recorded video of the actual author presenting)
        Answer questions about the work and any background methods
        Give a suggestion for some sort of future work that would make a good research paper
     
  • Project Schedule
       

Final Documentation

    Webpage
    Paper
    Video
    Final presentation
 

How to review a paper

 

 

Schedule

Month

Day

   

Due

 

October 

1

    Introduction 
    (project ideas)

 

 

 

3

    Seam carving for content-aware image resizing
    [more discussion of projects]

 

 

 

    [James travelling]


 

 

 

10

    Guest Andrew Adams gives talk on stabalization on cel phone

 

 

 

12

    More project planning

 

 

 

15

    Discussion: what goes into related work
    Scene completion using millions of photographs

Team proposals / Work Timeline

 

 

17

  

 

Photo tourism: exploring photo collections in 3D

 

 

Multiscale shape and detail enhancement from multi-light image collections


[Schedule team meetings]

 

 

 

19

Team Meetings:
3:00PM - 4:00PM: Gillian Smith, Josh McCoy

4:30: Serdar Sali

 

 

 

22

    Whats in an intro 

Multi-aperture photography??????

 

Image and depth from a conventional camera with a coded aperture????????

(Discussion Leader: Orazio Gallo)

related work (at least 10 papers)

 

 

24

   

Team Meetings:

4:00 PM - 4:30 PM: Adam Smith, James Skorupski

4:30 PM - 5:00 PM: Dave Ilstrup, Mariano Lizarraga

5:00 PM - 5:30 PM: Orazio and Sonia

 

 

 

26

No class - James Davis in Taipei

 

 

 

29

No class - James Davis in Taipei

 

"related work" is reviewed/fixed by classmates
intro - contribution statement

 

 

31

  Image-based tree modeling - Adam Smith

Capturing and animating occluded cloth

(Discussion Leader: James Skorupski)

 

 

 

November

2

Team Meetings:

3:30 PM – 4:00 PM: Dave Ilstrup, Mariano Lizarraga

4:30 PM – 5:00 PM: Serdar Sali

 

 

 

5

    Discussion: what goes in supporting figures 

 

VideoTrace: rapid interactive scene modelling from video (Discussion Leader: Gillian Smith)


supporting figures planned
"intro" is reviewed/fixed by classmates
related work is updated

 

 

7
 

    Discussion: what goes in a presentation
    

 

Veiling glare in high dynamic range imaging(Discussion Leader: David Ilstrup)

 

 

 

9

(team meetings)

 

 

 

12

    Holiday

 "supporting figs" reviewed/fixed by classmates

 

 

14

    Feedback on related work in general
    

 

??????

 

 

 

 

16

Team Meetings:
3:30 PM - 4:00 PM: Gillian Smith, Josh McCoy
4:00 PM - 4:30 PM: Adam Smith, James Skorupski
4:30 PM - 5:00 PM: Orazio and Sonia

 

 

 

19

    Feedback on intro in general
    

 

Factored time-lapse video

(Discussion Leader: Sonia Arteaga)

full paper

 

 

21

   

 

Soft scissors: an interactive tool for realtime high quality matting

(Discussion Leader: Feng Tang)

 

 

Image deblurring with blurred/noisy image pairs

(Discussion Leader: Josh McCoy)

 

 

 

 

(team meetings)

 

 

 

26

    

"full paper" reviewed/fixed by classmates
Video draft

 

 

28

 

 

Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs

 

   

Computational time-lapse video

 

 

 

30

(team meetings)

 

 

December

3

    Your Project Presentations
    Evals for end of quarter

Video revised/Presentations

 

 

5

    Your Project Presentations

The Transient Photometric Response Function [Adam and James]

PhotoExplorer [Tang]

 

 

7

    Your Project Presentations

Virtual Touch-Pad [Orazio and Sonia(12/07/07)]

 

 

UAV Image Registration [Mariano and Dave]
Direct to eBay [Gillian and Josh]
Depth Camera [Neeraj]

Image Recognition on Mobile Phones [Serdar]

 

 

 

    

 

 

 

 

    Finals Slot (???)

Show your work presentations
Revised paper (~4 page)

 

 

 


Team proposal

Find teammates. Decide on a project. Create a web page (not wiki) linked from the class web page and _in_ the class directory. The web page should include sections for Abstract and References. The abstract should be a paragraph description of the project which includes a contribution. Write as if the project is completed and you are describing your work. References should be annotated. Its sufficient to include one sentence on what the paper is about, and one sentence on why its related to your project. Also annotate who added each reference.
 

 


Every paper we read

You should think about ideas about how to extend each paper we read, and what might make a good follow-on research paper. Don't just copy the future work section, think of how it might be combined with your own work, or the paper we read last week, or something else. We'll discuss these ideas in class. Turn in a description of your idea written up in 1/2 page or less on a physical sheet of paper in class.

 


Paper

You will write your paper in stages: related work, intro, figure and results, everything else. You will get your classmates to help revise your work before I see it. After we have the most important section revised we will put the whole thing together. Its important to learn how to write by reading other peoples work. I believe you will find it much easier to help your classmates see their mistakes than it is for you to see your own.


Video

When you work on most graphics related topics its critical to make a video. Its impossible to evaluate your work without it. The video gets submitted with a paper. At a bare minimum the video shows the results of your work. However its good practice to make the video tell the whole paper story. If someone only watched your video they would know why your problem is important, what your key contribution is, a brief overview of how you did it, and of course your impressive results. Think of the video as an advertisement to read your paper. 

Make a video that is 2-5 minutes in length. Use any video editing program you like. Encode the video as MP4 at 640x480. Link to your web page.
 

 


Presentation

You will give a group presentation in class twice. Once as a draft, and again at the final time. The presentation should have the same organization as your paper: motivation, contribution, relation to previous work, what-you-actually-did, results, conclusion. In a real conference presentation, previous work is often dramatically condensed, in our class we’re going to leave it in because its critical to properly explaining what your contribution is.

In addition to content. Presentation style is important. Use pictures, or videos. Look at the audience. Etc.

Make a presentation. Link it to your project page. Deliver it in class.