Discrete Optimization

[First crate] minilp – a linear programming solver in pure Rust

Hi everyone! I'd like to share my first substantial Rust project – minilp, a linear programming solver.

Linear programming is a mathematical optimization technique for finding the optimum of a linear objective function with linear equality and inequality constraints which is useful in a lot of contexts.

I began writing code that became this crate when I was taking the discrete optimization MOOC. The course is awesome BTW. The lecturer is very entertaining and enthusiastic and the lectures are a joy to watch. There are no stupid quizzes, just a set of discrete optimization problems and you must provide sufficiently good solutions to them to pass. You can implement your solvers in any language and of course I was using Rust. When the need for a linear programming solver arose, I decided to implement my own. After I had a simple textbook implementation of the simplex algorithm I realized that it was hopelessly inadequate and that we were supposed to use existing LP solvers anyway. Turns out there was a lot of progress since the algorithm was invented in 1940s... But at that point sunk cost fallacy kicked in and I decided to improve my solver to the point of being usable. When I was done and passed the course, I thought that I might as well open-source it. Turns out, polishing a crate for open-sourcing is not trivial either, but anyway here it is!

The code mainly involves a lot of fiddling with arrays, so it is probably not the problem where Rust the language shines most (although these array accesses are bound-checked!). Still, writing code in a low-level language with a modern feel is a joy.

I haven't included performance comparisons because there is still a lot of room for improvement 😅. Although my solver outperforms established solvers like CLP and Glop on some problems, it is more of a fluke and for now they offer more robust performance.

I've also included a nice demo: a solver for the traveling salesman problem that uses my crate. It is intentionally simplistic, but is able to find optimal solutions to problems of size up to 150 points quite quickly.

That's it! I'd love to hear any feedback and hope that someone will find my crate useful!

Are scheduling algorithms really machine learning? This sounds much more like discrete optimization.

I do agree this space of algorithms is much more valuable to start with and provides a strong foundation to understand machine learning more deeply. But this is not machine learning.

Anybody who tries to solve a scheduling problem with machine learning is probably in academia and not the real world.

https://www.coursera.org/learn/discrete-optimization

Machine learning primarily excels at two things: categorization and regression. Complicated and specialized search algorithms are an entirely different animal but can lend ideas to machine learning via loss function optimization.

There are 3 good courses on Discrete Optimization. Especially this one: https://www.coursera.org/learn/discrete-optimization . One the hardest and most satisfying course I have ever done.

Also depends what your interests are. There are good courses on cryptography, algorithms, electronics (analog and digital), Machine Learning, Graphics. Target courses from top universities like Stanford, Yale, MIT, etc.

Thanks a lot!

I did a couple other articles on optimization stuff, because I took the coursera discrete optimization course a while ago (which I highly recommend!).

Ahhh dang you're right! I always forget to. I'll go back and edit it, thanks!

I’d recommend the discrete optimization course on coursera from university of Melbourne for specifically those types of problems. Pretty algorithm heavy but had great homework exercises associated with good lecture material for a number of classic problems including tsp, facility location, knapsack, etc. There are “official” rules to the homework submissions, but you can also use the homeworks to get first had experience with different solvers, frameworks, or hand made algorithms. All of the homework was in python. I used it to get familiar with pyomo and google’s or-tools and it was perfect. It sounds like it could be just what your looking for.

https://www.coursera.org/learn/discrete-optimization

Mostly https://www.coursera.org/learn/discrete-optimization besides that mostly trial and error. Thanks for your read

There's also a nice video on coursera, so it might not be easy to view it idk how that works (I can see it, but I took that course so it might be different).

https://www.coursera.org/learn/discrete-optimization

This course has a lot of stuff you might be interested in. It uses different optimization methods for problems like knapsack, traveling salesman, graph coloring, warehouse location, vehicle routing.

Coursera has a course called discrete optimization on the topic. The course started last week, so now is still a good time to join if you want to know more!

Coursera has some. I've never taken them but I've had pretty good luck with Coursera in general.

https://www.coursera.org/learn/discrete-optimization

Course recommendation for pursuing a career in AI. Looking for feedback and advice.

Hello, I'm currently following a machine learning course in order to learn more about AI and was wondering which other course(s) I should add to this in order to further my knowledge of AI. Machine learning course: https://www.coursera.org/learn/machine-learning I'd prefer if the course was on Coursera but I'm open to other options. Here are other courses I thought i should add to my list: Neural networks: https://www.coursera.org/learn/neural-networks# Discrete optimization: https://www.coursera.org/learn/discrete-optimization Data mining: https://www.coursera.org/specializations/data-mining Thanks for your help. Any other feedback is appreciated.

From a financial standpoint, it probably isn't worth paying for a couple of extra courses and delaying getting a job for another semester. Maybe you'll use some of these topics in your career, maybe not, but by that logic you should take every single course offered just to be safe. IMO it's more important to show you're capable of learning (which the degree indicates) than it is that you're already familiar with specific concepts.

Personally, I'd advise to hurry up and get the diploma which will open up doors and make your resume better, start earning a salary and climbing up the career ladder earlier. If there are any topics that interest you, you can try auditing the courses later or find other MOOCs that teach these topics. For instance, here are MOOCs I quickly found for [HDDA],(https://online-learning.harvard.edu/course/data-analysis-life-sciences-4-high-dimensional-data-analysis?delta=1) optimization, and reinforcement learning.

Yes, you can do just about everything with only python.

Most of the topics fall under the umbrella of operations research. I don't know if your are interested out of just pure curiosity,but these topics have many applications in industry if you are looking for career direction, particularly in scheduling, logistics, data science, and general optimization. I could perhaps give more relevant advice if I knew your interests and background better.

Heuristic-based local search is a technique in discrete optimization. Here's a course I took that helped explain discrete optimization, including a section on local search techniques. https://www.coursera.org/learn/discrete-optimization. The assignments aren't very clearly structured imo but lectures are good.

Neural Networks are a way to approximate the relationship between inputs and output as a nonlinear function. Here is an introductory course I took a while back on machine learning that I would highly recommend: https://www.coursera.org/learn/machine-learning. And a follow up course: https://www.coursera.org/learn/neural-networks-deep-learning

https://www.datacamp.com has many good courses for building models in python, particularly relating to data science.

Both of those websites charge about $30/month for unlimited courses. A couple other buzzwords that might be worth looking into are Monte Carlo Simulations, newton's method, gradient descent, game theory. Here is a lighter documentary on alphaGO, a program developed by google that beat the world champion in Go: https://www.youtube.com/watch?v=9gzMQOa5MD4. And if you are more interested in the AI side of things I would highly recommend this playlist on reinforcement learning by deepmind: https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ.