EP30.FSF1120 — Nonlinear Optimization

This course provides a rigorous introduction to theory and algorithms for solving Nonlinear Optimization (Nonlinear Programming) problems. The course starts with fundamental mathematical concepts, emphasizing Convex Analysis as a cornerstone for modern optimization. Key topics include necessary and sufficient Optimality Conditions (First-order, Second-order, Karush-Kuhn-Tucker - KKT), with an emphasis on the use of subgradients and duality for analyzing optimality. The course then provides an in-depth study of numerical algorithms for Unconstrained Optimization (Gradient Descent, Accelerated First-order methods, Newton's, Quasi-Newton methods) and Constrained Optimization (Penalty, Barrier, and Interior Point Methods, as well as modern methods for Convex Constrained Optimization such as Proximal and ADMM algorithms). The course aims to develop the analytical skills required to formulate, analyze, and solve complex optimization models encountered in engineering, economics, data science, and machine learning.

Requirements for Final Examination

Students are allowed to take the final/terminal exams (50%) if their attendance score (10%) is 5 or higher (on a 10-point scale).

See Appendix 2, Rubric 1 for attendance evaluation.

Requirements for Class Attendance

Students are responsible for attending all classes. In case of absence due to force majeure, students must provide sufficient and reasonable proof. For each absence, 1 point will be deducted from the attendance score. Students who miss more than 3 classes, with or without valid reasons, will be considered as failing to complete the course and must re-register.

Students will be awarded points for constructive comments. These points will be added to attendance and group assignment scores.

Groups that do not submit their group work will receive a score of 0. Late submissions will incur a deduction for each day they are late.

Requirements for In-class Behaviour

The course is conducted on the principle of mutual respect between learners and lecturers. Any behavior that disrupts the teaching and learning process is strictly prohibited.

Students must arrive on time for class. Those who are more than 10 minutes late will not be allowed to attend. Noise and other disturbances that affect the learning process are not permitted.

Laptops and tablets may only be used for taking notes and performing calculations related to lectures and exercises. Use for any other purpose is strictly prohibited.

The content of the course consists of 4 chapters:

• Chapter 1: Mathematical Foundations and Convexity
• Chapter 2: Unconstrained Optimization
• Chapter 3: Constrained Optimization Theory
• Chapter 4: Advanced Topics and Applications