berlin robotics education

Aktuelle Forschung in KI & Robotik, TU Berlin

BSc seminar taught in German, 3 credits
Lecturer: Prof. Toussaint
Institution: Technische Universität Berlin

Learning outcomes

• Understanding of methods currently used in AI and Robotics research
• Motivation to learn relevant AI methods and underlying theory (esp. the relevant mathematical basics)

Content

• Topics and methods of recent research published at AI & Robotics conferences (esp. RSS, CORL) will be discussed.
• Students present publications (in the style of a reading club), explain their structure, methods used, and identify the relation to previous work.

Algorithmen und Datenstrukturen, TU Berlin

BSc lecture taught in German, 6 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
Students gain fundamental knowledge on imperative and object-oriented programming, the ability to formulate a specification and convert it into an implementation. Additionally students will become familiar with essential data structures and algorithms and they will be able to select and implement suitable data structures.

Content

• Runtime estimations and proofs of correctness
• Introduction to Java
• Advanced data structures (e.g.: realization of sets)
• Graph representation
• Flow problems (modeling, max-flow, min-cut)
• Algorithms for optimization problems (e.g.: Branch & Bound, Backtracking)
• Scheduling

Bio-inspired Computer Vision, TU Berlin

MSc course taught in English, 9 credits
Lecturer: Prof. Gallego
Institution: Technische Universität Berlin

Learning outcomes
Students have an in-depth understanding of selected early vision models of human perception and their implementation in hardware (neuromorphic cameras). They are able to present and convey the acquired knowledge and skills eloquently.

Content
The project topic is subject to change, and will be made available before the start of each semester. Please see: http://www.psyco.tu-berlin.de/teaching.html

Einführung in die Künstliche Intelligenz, TU Berlin

BSc lecture taught in German, 6 credits
Lecturer: Prof. Toussaint
Institution: Technische Universität Berlin

Learning outcomes
Students will gain an integrative understanding of the research field of Artificial Intelligence that integrates data-based AI (especially machine learning) and model-based AI (especially planning & reasoning) in an equal fashion. Students will understand AI from the perspective of decision theory, machine learning, optimization, and classical problem solving. Students will be able to independently implement basic algorithms from these areas from scratch and understand them in detail. Furthermore they will know which AI problem formulation and algorithms are adequate for a given application problem.

Content
Problem definition and algorithmic approaches in the fields of

• Decision theory (incl. reinforcement learning, bandits, control theory).
• Machine learning
• Optimization
• Inference, Classical Planning, & Problem Solving

Moreover, basic and recurring algorithmic principles such as:

• Dynamic Programing
• Optimization- vs. sample-based methods
• Decision trees

Event-based Robot Vision, TU Berlin

MSc course taught in English, 6 credits
Lecturer: Prof. Gallego
Institution: Technische Universität Berlin

Learning Outcomes
Participants will learn basic concepts, theoretical foundations and relevant algorithms developed in the field of event-based (i.e., neuromorphic) vision. Upon completing the module, participants will have an overview of the field, spanning from the principle of operation of event-based sensors (e.g., event-based cameras), their advantages and disadvantages, to the methods used to process their output for a target application. Participants will also be aware of the differences with standard (frame-based) computer vision, in terms of methods, performance criteria and applications.

Content
This course is the first of its kind, worldwide. To the best of the instructor’s knowledge, no similar course has been offered anywhere due to the novelty of the topics covered, which have appeared in research conferences and journals over the last ten years.

The topics covered include the following:

• Bio-inspired principle of operation of event-based (i.e., neuromorphic) sensors.
• Event-based feature detection and tracking.
• Event-based motion estimation: optical flow estimation, 3D reconstruction, camera localization and ego-motion estimation, simultaneous localization and mapping (E-SLAM).
• Stereo depth estimation in dynamic scenes.
• Image intensity reconstruction from events.
• Event-based pattern recognition, classification and machine learning.
• Event-based signal processing and filtering.
• Event-based sensor fusion.
• Event-based control.
• Event-based (i.e., spike-based) hardware.
• Novel applications in event-based vision.

Event-based Robot Vision Project, TU Berlin

MSc course taught in English, 9 credits
Lecturer: Prof. Gallego
Institution: Technische Universität Berlin

Learning Outcomes
Participants of this project course will gain practical experience in applying techniques from event-based and computer vision to solve problems in robot perception (motion estimation, recognition, etc.). Participants will work individually or in a small team collaboratively, and will acquire knowledge about the state-of-the-art on event-based vision related to the chosen problem.

Content
Event-based vision is an emerging technology that promises to offer advantages to overcome some of the limitations of traditional, frame-based cameras and visual processing pipelines (from sensors to output, actionable information), such as latency, dynamic range, bandwidth and power consumption. To unlock the advantages of event-based cameras, new algorithms are needed to process their unconventional output (a stream of asynchronous pixel-wise intensity changes, as opposed to the familiar video images of standard cameras). This project is related to the investigation and development of tailored algorithms and methods to tackle specific problems in event-based vision (motion estimation, segmentation, object detection and recognition, etc.). At the beginning of the module, students receive or select project topics from a list of possible ones, as well as some introductory material related to the chosen problem. After setting the project teams and topics, the suitable tools to carry out the project are discussed and set up. The students prepare a project plan, specify the data on which they will be working on and the steps that are anticipated for a successful completion of the project. During the remaining weeks the students develop their projects and discuss the progress with the instructor, to guide future action items. At the end of the project, the students present their findings to other students in the module, with an oral presentation. They summarize not only the technical outcome of the project but also the difficulties and lessons learned during the project.

The general topics include but are not limited to:

• Algorithms: visual odometry, SLAM, 3D reconstruction, optical flow estimation, image intensity reconstruction, recognition, stereo depth reconstruction, feature/object detection, tracking, calibration, sensor fusion (video synthesis, visual-inertial odometry, etc.).
• Event camera datasets and/or simulators. – Event-based signal processing, representation, control, bandwidth control.
• Event-based active vision, event-based sensorimotor integration.
• Applications in: robotics (navigation, manipulation, drones…), automotive, IoT, AR/VR, space science, inspection, surveillance, crowd counting, physics, biology. – Model-based, embedded, or learning approaches.
• Novel hardware (cameras, neuromorphic processors, etc.) and/or software platforms.
• New trends and challenges in event-based and/or biologically-inspired vision (SNNs, etc.).
• Event-based vision for computational photography. A longer list of related topics is available in the table of content of this repository: https://github.com/uzh-rpg/event-based_vision_resources/

Learning and Intelligent Systems: Project, TU Berlin

MSc project taught in English, 9 credits
Lecturer: Prof. Toussaint
Institution: Technische Universität Berlin

Learning outcomes
After attending the module, students have extensive experience with working in a research lab on specific topics of current research in Learning & Intelligent Systems. They have in-depth knowledge and practical understanding of the specific topic they work on during the project. They are capable of conceptualizing a research project, formulating a realistic workplan, and managing the research work autonomously.

Content
Changing topics from Robotics, AI & Machine Learning. Project topics are tightly integrated with the department’s research activities.

Motion Planning, TU Berlin

MSc lecture taught in English, 6 credits
Lecturer: Prof. Hönig
Institution: Technische Universität Berlin

Learning outcomes
Motion planning is a fundamental building block for autonomous systems, with applications in robotics, industrial automation, and autonomous driving. After completion of the course, students will have a detailed understanding of:

• Formalization of geometric, kinodynamic, and optimal motion planning;
• Sampling-based approaches: Rapidly-exploring random trees (RRT) and probabilistic roadmaps (PRM);
• Search-based approaches: State-lattice based A*;
• Optimization-based approaches: Sequential convex programming;
• The theoretical properties relevant to these algorithms (completeness, optimality, and complexity). Students will be able to:
• Decide (theoretically and empirically) which algorithm(s) to use for a given problem;
• Implement (basic versions) of the algorithms themselves;
• Use current academic and industrial tools such as the Open Motion Planning Library (OMPL).

Content
This course is jointly developed and held by Dr. Andreas Orthey (Realtime Robotics) and Dr. Wolfgang Hönig (TU Berlin). It provides a unified perspective on motion planning and includes topics from different research and industry communities. The goal is not only to learn the foundations and theory of currently used approaches, but also to be able to pick and compare the different methods for specific motion planning needs.

Part 1: Foundations

• Introduction, Motivation, and Problem Formulation
• Configuration space, Transformations, Angular representations, Metrics •
• Efficient collision checking

Part 2: Search-Based

• A* and relevant variants with their theoretical properties
• Motion primitives, state-lattice-based planning

Part 3: Sampling-Based

• Sampling theory (dispersion, discrepancy) • Tree-based planner: RRT, EST
• Roadmap-based planner: PRM
• Asymptotically-optimal sampling planner: RRT*; PRM*
• Open Motion Planning Library (OMPL)

Part 4: Optimization-Based

• Overview of continuous constrained optimization formulations
• Mathematical encoding of motion planning problems

Part 5: Current and Advanced Topics

• Realtime motion planning
•  Hybrid search-, sampling-, or optimization-based motion planning
• Machine learning-based motion planning
• Multi-robot motion planning: dRRT, M*

Optimization Algorithms, TU Berlin

MSc lecture taught in English, 6 credits
Lecturer: Prof. Toussaint
Institution: Technische Universität Berlin

Learning outcomes

The students will be able to develop and apply optimization algorithms. They can formulate real-world problems appropriately as mathematical programs. They have a detailed understanding of the different categories of optimization problems, and methods to approach them. They have a basic understanding of the theory behind and properties of optimization algorithms. They have an overview of and experience with existing optimization software and are able to apply them to solve optimization problems.

Content

The course is on continuous optimization problems, with focus on non-linear mathematical programming (constrained optimization).

Part 1 introduces efficient downhill algorithms in the unconstrained case:

• gradient descent, backtracking, Wolfe conditions, convergence properties
• covariant gradients, Newton, quasi-Newton methods, BFG

Part 2 will introduce efficient algorithms for constrained optimization:

• Basics on KKT
• Log-barriers, Augmented Lagrangian, primal-dual Newton
• Phase I optimization

Part 3 will dive into large-scale, sparse, and structured solvers, dealing with non-convexity, and applications

• Existing libraries, CERES, structured NLPs, solving constraint graphs
• Optimization plus sampling to handle non-convexity
• Branch-and-bound-type methods
• Applications in AI, Robotics, & ML

Robotics, TU Berlin

MSc lecture taught in English, 6 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
After completing the module, the students will have knowledge of the problems and practical solutions to controlling multi-joint robot systems. They will also have acquired methods to abstract and simplify complex, non-linear problems in the realm of action, perception, and representation, which are the basis for cognitive and intelligent robots.

Content
Concepts, algorithms and application-specific aspects of Robotics:

• Kinematics, dynamics, position control, trajectory generation, controller tuning, collision avoidance, visual servoing, probabilistic robotics, Simultaneous Localization and Mapping (SLAM)
• Practical implementation on a real time control system

Robotics: Advanced, TU Berlin

MSc lecture taught in English, 6 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
Students gain a deep understanding of problems and methods of robotics in the context of embodied intelligence. They are able to critically consider and apply established and novel concepts and methods.

Content

• Methods, algorithms and limitations of state of the art robot grasping, collision avoidance,
• Force control,
• Integrated path planning and control,
• Visual perception,
• Representation of space,
• Machine learning

Robotics: Current Topics, TU Berlin

MSc seminar taught in English, 3 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
After attending the module, students have in depth knowledge of specific areas of Robotics under active research. Students are capable of doing literature research and review literature critically. They are able to write papers in a journal or conference format, and can communicate complex matters in oral presentations.

Content
Seminar on changing topics from Robotics and related fields of research, e.g. kinematics, dynamics, control, localization, planning, representation learning, heuristics, collision avoidance, computer vision, machine learning, probabilistic robotics.

Robotics: Fundamentals, TU Berlin

BSc lecture taught in German, 6 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
In this module, students acquire knowledge of fundamental paradigms of autonomous, mobile robot systems for performing complex tasks in unknown environments. The module teaches basic knowledge of algorithms for capturing and processing sensor data, planning and generating robot motions, representing the environment, and dealing with uncertainty in the environment. Experience in software development is gained as well as the development of embedded software systems interacting with the real world through sensors and actuators.

Content

• Introduction to the components of mobile robots (sensors, actuators, real-time control, kinematics, representation of the environment, path planning in 2D, localization in 2D, dealing with uncertainty and incomplete knowledge, probabilistic models and recursive estimation).
• The lecture content is complemented by group exercises with real robots (iCreate Platform with ROS and Python). Students will develop the necessary components to move a robot in a controlled manner in space. This includes programming motion primitives, localization algorithms, path planners, and map generation methods. The exercises are finalized with a robot tournament

Robotics: Project, TU Berlin

MSc project taught in English, 9 credits
Lecturer: Prof. Brock
Institution: Technische Universität Berlin

Learning outcomes
After attending the module, students possess in-depth knowledge of specific topics in ongoing, state-of-the-art robotics research. Students gain both theoretical knowledge and practical experience, and, where appropriate, conduct real-robot experiments to develop and validate their research ideas. They know how to organize, track and adapt a project to resource constraints. They know how to work independently towards a project objective while balancing scientific and practical objectives. Students possess skills in project planning, giving project presentations, discussing project ideas in a team, writing scientific reports, and documenting source code for use by others.

Content
Varying topics from robotics and related fields of research, e.g., machine learning, perception, motion planning, manipulation, control, and hardware design. Project topics are tightly coupled with the department’s ongoing research activities. Therefore, we encourage prospective students to visit the department’s website to learn about possible project topics: https://www.robotics.tu-berlin.de/menue/research/