I wrote a optimization code which is linked with EPANET2 for a real large network. I want to speed up the run time of code by implementation multi-core processors and parallel computing but I think the EPANET2 hasn’t this ability.
Does anyone have experiences about applying EPANET2 in a parallel computing program?
Hi @alireza2032. I have experimented some with parallelization, so I’m happy to share some practical advice. Best to start with profiling the code running various network sizes. It’s not always/ever intuitive where time will be spent until you’ve done this step.
I’ve seen that the matrix solver is extremely efficient even for pretty large networks. What seems to be more expensive computationally is determining pipe losses - and lucky for us, that is quite amenable to parallel execution, since each pipe computation is independent of the others.
If you are opening many
inp files, I’ve seen for large networks that
mindegree takes substantial time to reorder the matrix - there are alternative sparse matrix reordering algorithms that take advantage of parallelism - “ParMETIS” looks promising for that reason.
If you are looking for courser-level parallelism, then you could run multiple different simulations at the same time, one on each CPU core available. Whether you can do this with existing epanet code probably depends on how it’s compiled, and what system it’s running on - for linux shared libraries there is an issue with epanet’s global variables being shared across client instances, which is not good. I have a dev branch in my GitHub repository with some wrapper structs to get around that problem if you want to take a look.
additionally, if you are doing very much saving of results to some file format, then there is another opportunity for concurrency. Spin off a new task to save results into a db or bin file, while the simulation code goes on to the next solution.
It’s very interesting stuff, but programming for concurrency is still pretty confusing and difficult to do well - and very easy to spend too much time optimizing the wrong thing – so profile first.
@samhatchett this is one of the most insightful comments I have read on the subject. Thanks for sharing your experience with everybody!
It seems that parallelizing tasks, such as randomized simulations during Monte-Carlo analysis, is possible by making direct calls to the epanet executable and producing binary output files (but caution is required not to overwrite solutions by using the same file-names). I suppose we can do the same with EPANET-MSX. Of course, this solution will not be relevant to all problems
In any case, this could also be useful for distributed computations (e.g. for big-data analysis).
MATLAB has some nice features for parallelization, that we are currently experimenting, to see how we can use it efficiently with EPANET.
by the way, if anyone is interested in running monte-carlo scenarios, @hyoungmin_woo can assist - he probably knows the current whereabouts of the EPANET-MCX (monte carlo extension) code.