Professor Ashok Cutkosky at Boston University is teaching the renowned course EC525: Optimization for Machine Learning (Fall 2023), which delves into the cutting-edge algorithms behind the optimization techniques that power modern deep learning models. I believe this course is especially valuable for anyone looking to build a deeper understanding of the theoretical underpinnings of deep learning, particularly when it comes to deriving complex formulas and mastering the sophisticated mathematical expressions often used in academic research.
One thing I really appreciate about Professor Cutkosky’s approach is his awareness of how challenging proof-writing can be for many students. To address this, he has provided a series of exercises designed specifically to help students hone their proof-writing skills. These exercises are a great way to solidify the foundational concepts that drive the optimization techniques discussed in the course.
Additionally, in these exercises, Professor Cutkosky provides several helpful suggestions and guidelines to assist with proof-writing:
- Stick to the basics: Focus on the definitions and axioms provided in the problem. Avoid the temptation to bring in more advanced concepts like vector space bases or singular value decompositions.
- Don’t assume things are obvious: Many statements might seem self-evident, but crafting a rigorous proof often requires more careful thought.
- Use the axioms methodically: When stuck, work through the list of axioms one by one. It’s easy to overlook less commonly used ones that might be the key to the solution.
- Leverage contrapositive reasoning: Remember that the contrapositive of a statement, $A \implies B$, is logically equivalent to $\neg B \implies \neg A$. This technique can often simplify the proof process.
These strategies are crucial for tackling the challenges of abstract proof-writing.
To follow Professor Cutkosky’s course and share my own solutions and reflections on these exercises, I’ve launched this blog series titled A Commentary of the Pre-Practice for Boston University’s EC525 Course. It is expected to include 8 posts: this preface, 6 posts with detailed commentary on each of the 6 problems, and a concluding post. I will update this series periodically, and you can click here to bookmark it or access it directly via the hyperlink at the end of the page.
Notably, based on their types, I will assign appropriate titles to the blogs for each of the 6 problems to make them easier to reference.
Preface: Motivation and Overview (You are currently browsing this post)