Stylus-based Hand-written Formulae Recognition

612cc2ee 3234 4af5 b404 c373c0a7683f.jpg 500 500

Organization: Advanced Interactive Technologies

Involved Host(s): Admin AIT

Abstract: The overall goal of this project is to explore novel ways to automatically recognise and digitalise hand-written formulae.

Description: The overall goal of this project is to explore novel ways to automatically recognise and digitalise hand-written formulae. The project will focus on formulae written using tablet-style stylus. During the thesis the student will build a system which combines advanced machine learning techniques for continuous character/shape analysis based on strokes, with novel stylus-based user interfaces for automatic digitalisation of formulae. Ideally, the final system will also be able to solve digitalised formulae (using existing solvers), and optionally it will suggest to the user ways to reduce complicated expressions. Depending on the thesis outcome, the student may be offered the possibility to enter the CRHOME [1] competition for online handwritten mathematical expressions recognition. [1] CROHME: Competition on Recognition of Online Handwritten Mathematical Expressions - [2] Hands-On Math: A page-based multi-touch and pen desktop for technical work and problem solving - Robert Zeleznik, Andrew Bragdon, Hsu-Sheng Ko, Ferdi Adeputra, [3] SetPad: a sketch-based tool for exploring discrete math set problems - Travis Cossairt, Joseph LaViola [4] Structure Editing of Handwritten Mathematics: Improving the Computer Support for the Calculational Method - Alexandra Mendes, Roland Backhouse, Joao Ferreira

Goal: This project has three main objectives: 1. Develop and experiment a variety of stroke-based character recognition algorithms, which will have to work at interactive rates and with continuous pen-based input. If required, the student is expected to extend existing algorithms or formulate novel solutions. 2. Design and implement an interactive user interface which allows user to quickly sketch, digitalise and solve handwritten mathematical expressions. The interface should work in real- time, and there should be no noticeable delay by the user. 3. Explore the notion of “interactive formulae reduction”: while recognising the expression, the system suggests to the user possible ways to reduce complicate formulae. This last aspect is optional

Contact Details: The ideal candidate will have a background in Computer Vision and/or Machine Learning. Solid programming skills, especially on vision and ML related topics, and an interest in hands-on development and experimentation is also a requirement. Supervisors: Dr. Fabrizio Pece and Prof. Dr. Otmar Hilliges

Labels: Master Thesis CLS Student Project (MPG ETH CLS)
Topics: Information, Computing and Communication Sciences

© 2017, Copyright Max-Planck-Gesellschaft - Imprint