handheld augmented reality

Augmented Reality Anywhere and Anytime   

Projects

   Social Augmented Reality

   Information Presentation

   Real-Time Self Localization

   Structural Modelling

   AR Graphics

   AR Navigation

   Augmented Reality Games

   Past Projects

 

Technology

   Hybrid SLAM

   Panorama SLAM

   Planar SLAM

   Model-Based Tracking

   Marker-Based Tracking

 

Software Libraries

   Studierstube ES

   Studierstube Tracker

   Muddleware

   ARToolkitPlus

 

More

   Videos

   Media/Press

   Team

   Publications

   Collaborations

   Student projects

   FAQ

   Site map

 

The Handheld Augmented Reality
Project is supported by the
following institutions:

Qualcomm

 

Christian Doppler Forschungsgesellschaft

 

Graz University of Technology

 

 


 

Come to ISMAR 2011

ISMAR2011

 

Follow us on twitter

Student Projects

This page lists all available student projects from team members of the Handheld Augmented Reality group. Feel free to contact us for more information!

 

Real-Time HDR Images on Smartphones

Better camera control helps in first place helps to capture nicer images. Moreover, it can help in improving the image acquisition process, to allow for better algorithm performance in subsequent processing steps. For example, by actively controlling the exposure change between consecutive frames in a video stream, the quality of a single captured image can be improved considerably, reducing the number of under- or overexposed areas. On the one hand, this makes images looking more pleasing to the user, on the other hand it helps in further use in Computer Vision based processing.

The goal of the project is to implement a method for automatic generation of HDR images in a live video stream on a mobile phone. By using special hardware features, HDR images shall be generated in real-time and used to improve the performance of Tracking algorithms in the area of Augmented Reality. Likewise, the task is to create high-quality panoramas. The development should be done on a normal desktop PC while the Smartphone the methods shall be deployed on is a Nokia N900.

Required Skills: Graphics and vision basics, C/C++ programming know-how in Linux, interest in computer vision, computer graphics and augmented reality

Further information: PDF, Online (ICG)

 

Hardware Accelerated Naive Keypoint Descriptors and Matching

Natural Feature Tracking (NFT) is an important technique for Augmented Reality applications. NFT can roughly be divided into feature detection and tracking. The most costly part hereby being the detection and classification of key points. Though, the actual extraction is mostly computationally inexpensive, key point descriptors still need large amounts of cycles and mostly are using implementations that are heavy in conditional dependencies. However, key-point descriptors can also be seen as a specialized implementation of signal processing and therefore could be offloaded into an FPGA, DSP or the GPU common in most hardware including mobile devices. In a first step this project investigates nave descriptors such as, but not restricted to OSID [1] in order to evaluate the feasibility of this approach. Secondly, an approach of key point matching is needed to avoid the read-back of candidates. The objective is to understand and evaluate run-time behavior and robustness of the key-point descriptors and matching with the focus on a target implementation on a mobile or embedded platform.

Required Skills: Graphics and vision basics, C/C++ programming know-how, know-how in either GPGPU, DSP or FPGA programming, interest in computer vision, computer graphics and augmented reality

Further information: PDF, Online (ICG)

 

Depth Estimation with Mobile Camera Stereo-Setups

The use of a stereo camera setup allows the estimation of depth and distance of an object from a pair (or set) of images. Using fast video adapters the calculations necessary can be done in real-time. These desktop video adapters can be considered at the high end of the entire range of available graphics processors concerning their performance and energy consumption. GPUs used in mobile devices are much less performant, but considerably more energy-aware. These GPUs include a simpilar feature set and can be deployed in a wide range of mobile devices. In the near future, these chips cannot be replaced by considerably more powerful versions, but should still be capable to solve the task of depth estimation if a careful algorithm design is chosen. A special use case of these methods is given by the deployment of stereo camera setups in future modern Smartphones.

Required Skills: Graphics and vision basics, C/C++ programming know-how, interest in computer vision, computer graphics and augmented reality

Further information: PDF, Online (ICG)

 

Active-Search Strategies in HD Video Streams

Current Tracking methods consist of very computationally expensive parts. To allow these algorithms to run in real-time, it is necessary to employ different methods for reducing the amount of computational load in various areas. Active Serach is a method that can be used to actively search feature or image correspondences across consecutive frames in videos. This is possible due to a temporal and spatial dependency of the individual images. The strategies used can also be deployed to special hardware, like DSPs or GPUs, leveraging the special suitability of the individual hardware type for certain computational tasks.

The goal of this project is to investigate and compare methods of Active Search for local-feature based tracking methods, both using GPUs and multi-core CPUs. While existing algorithms can and shall be adapted to higher image resolutions, the methods shall also be modified to be able to run with high-performance video feeds (e.g. cameras delivering 100fps or more). The special suitability of certain computational steps should be evaluated for GPUs, especially for the use of these methods on Smartphone hardware. The ultimate goald is to radically improve the performance of active search algorithms by using special hardware deployed especially in the mobile devices sector.

Required Skills: Graphics and vision basics, C/C++ programming know-how, interest in computer vision, computer graphics and augmented reality

Further information: PDF, Online (ICG)

 

Simultaneous Model-Based Tracking and SLAM

In this project, the advantages of a combined model-based and SLAM tracker should be investigated. Model-based trackers such as StbNFT deliver 6DOF poses relative to scene targets which are learnt offline. SLAM systems such as PTAM attempt to cope with unknown scenes, creating 3D maps online and track from them in parallel. In the combination, known objects within the scene could be detected and registered in a common coordinate system (e.g. the SLAM map). This allows for new possibilities in the design of Augmented Reality apps. The first milestone of the project will be to integrate e.g. the aforementioned systems and thus create an environment for further experiments.

Required Skills: Graphics and vision basics, multi-view geometry, software design (C++, OO programming), interest in 3D computer vision and augmented reality

Further information: PDF, Online (ICG)

 

Tracking Algorithms in Heterogeous Multi-Processor Systems

Many tracking algorithms consist of a detection and a tracking part. The further is often computationally expensive to estimate the camera position relative to the actual target. The subsequent tracking part can take advantage of the initial estimate, and further adjust and calculate pose updates from frame to frame. Consequently, this can be done much more efficiently.

In this project existing algorithms should be extended and adapted to better fit heterogeneous multiprocessor systems. For example, any sort of set-top boxes, smart cameras or modern smartphones running Android or the iPhone can be listed here. On the one hand, the task is to use the GPU for desired calculations to speed up the entire algorithm, on the other hand algorithms can be made more efficient by parallelising them onto multi-core CPUs. Additionally or alternatively, DSPs can be used to exploit their high suitability for certain signal processing tasks. The development can be either done in a conventional desktop environment (high resolution, large computational resources), or directly in the smartphone area (low resolution, low computational power).

Required Skills: Graphics and vision basics, C++ skills, interest in smartphone programming, interest in computer vision and augmented reality

Further information: PDF, Online (ICG)

 

Interaction for AR Authoring

The goal of this project is to explore new ways of interaction which can be used for Augmented Reality Authoring. The idea is to create new interaction metaphors which differ from desktop based interaction methods. One way could be to use the sensors build into an mobile phone. This project is within the Christian Doppler Lab for Handheld Augmented Reality. For more information contact Tobias Langlotz (langlotz(at)icg.tugraz.at)

Required Skills: Knowledge of C++ or object oriented programming, Interest to work on new interaction techniques for mobile phones

 

Interaction for AR Authoring

The goal of this project is to explore new ways of interaction which can be used for Augmented Reality Authoring. The idea is to create new interaction metaphors which differ from desktop based interaction methods. One way could be to use the sensors build into an mobile phone. This project is within the Christian Doppler Lab for Handheld Augmented Reality. For more information contact Tobias Langlotz (langlotz(at)icg.tugraz.at)

Required Skills: Knowledge of C++ or object oriented programming, Interest to work on new interaction techniques for mobile phones

 

Development of methods to improve the visual coherence in AR on hand held devices

A convincing seamless integration of virtual content and real-world content is still a challenging task – especially on hand held devices. The aim of this project is to implement different methods to improve the visual coherence. Development platform: Nvidia Tegra DevKit ( mobile GeForce), OpenGL ES 2.0, OpenGLSLES

 

Computer Vision based localization for mobile phones

This project targets the problem of localization of cellphones. Instead of using GPS or WIFI Triangulation for the localization of mobile phones, a vision based approach should be developed. Therefore a picture is send from the cellphone to a server. The result of the vision based analysis on the server should be transmitted to the cellphone as coarse estimation of position and direction.

 

Computer Vision on programmable GPUs for mobile phones

Programmable GPUs will be soon available on end-user mobile phones. These GPUs will allow to run Computer Vision algorithms at a speed and precision previously unreachable on a mobile device. The goal of this project is to implement and evaluate the feasibility of state-of-the art Computer Vision algorithms for real-time pose tracking (e.g., SIFT, Ferns, ...) and object recognition on a mobile phone. As a development platform we provide the student with access to one of the most recent (and few) development boards for mobile phones of the future.

 

Gesture Detection

6DOF pose tracking can be used to give computers commands via gestures (specific movements). In this project a gesture toolkit shall be developed on top of an existing tracking library. The gesture toolkit receives 6DOF tracking data as input and outputs recognized gestures such as waving, drawing shapes (e.g. a circle).

 

Graphical Editor for Studierstube ES Scene-Graphs

Our mobile phone AR solution "Studierstube ES" uses a proprietary scene-graph that is optimized for rendering on devices running OpenGL ES or Direct3D Mobile. A natural problem with proprietary solutions such as this is the need for an authoring toolkit. In this project a graphical editor shall be developed that shows the scene-graph structure in a tree-view and based on reflection automatically creates generates dialog boxes for editing properties of nodes.

 

Optimizing tracking targets to improve the quality of natural feature tracking

In our StudierstubeES software framework we can track targets directly from natural features (e.g., posters or game boards), without the requirement for any artificial props. Unfortunately, not every tracking target performs equally well: uneven distribution and low distinctiveness of features are just two of the issues that can bring to a quick drop in tracking quality. One tracking target can often be "improved" : the goal of this project is to develop a tool that analyzes and predicts which parts of the tracking target are more prone to bad tracking.

 

Panoramic previews for Augmented Reality browsers

Within our projects on Augmented Reality browsers, we are using panoramic previews of the location of all labels in the surroundings. Such previews show a full 360 panorama with all labels, and the current orientation of the device's camera.

The goals of this project are:

  • Visualization of the panoramic preview. Several techniques exist in Information Visualization, such as compressed views, fish-eye lenses and perspective walls. One or more of these techniques can be implemented and tested.
  • Touch-based interaction with the panoramic preview. Touch interaction could be used e.g. for browsing the labels, or for scrolling through the preview.

You will be provided with an already-working basic system and you will be asked to extend it.

 

Poor man's SLAM on a mobile phone

SLAM is a Computer Vision method used for online creation of a 3D map of the environment for real-time tracking purposes. The goal of this project is to simplify the SLAM approach by reducing its complexity to a simpler planar case (e.g., a map, a movie poster, ...). This stripped-down SLAM version would then require a smaller computational effort, and become applicable in real-time mobile phone applications.

 

Scripting Support for Studierstube ES

Studierstube ES is one of the most advanced AR frameworks for mobile phones. The goal of this project is to add scripting support for this AR framework to improve the rapid prototyping abilities of StbES. As Studierstube ES was designed from ground up to make best use of the handheld platforms the scripting language should follow this design rules. LUA seems to be a promising candidate to achieve this goals. Therefore ways of automatic wrapping the existing API should be identified like the use of SWIG.

 

Text Tracking

It has recently been shows that regular text can be effectively recognized and tracked from low resolution images. In this project a software shall be developed to train text passages for tracking (extracting tracking information and storing them in a database) plus a client software to track from previously trained paragraphs of text. Finally recognized paragraphs shall be used for 6DOF pose estimation.

 

 

 

copyright (c) 2014 Graz University of Technology