foreach - Parallel processing in R done wrong? -


i have code trying process in parallel using foreach-package. code working when run on computer 4 cores takes 26 min , when switch 1 32 cores, still takes 13 min finish. wondering whether doing wrong since using 8 times cores, reduce time 1 half. code looks this:

no_cores <- detectcores() cl <- makecluster(no_cores) registerdoparallel(cl) xenopus_data <- foreach(b=1:length(newly_populated_vec),.packages = c("raster", "gdistance", "rgdal","sp")) %dopar% { xenopus_walk(altdata=altdata,water=water,habitat_suitability=habitat_suitability,max_range_without_water=max_range_without_water,max_range=max_range,slope=slope,start_pt=newly_populated_vec[b]) } stopcluster(cl)   

for computer 4 cores following time:

time_of_start [1] "2016-07-12 13:07:23 cest" time_of_end [1] "2016-07-12 13:33:10 cest"  , 1 32 cores: time_of_start [1] "2016-07-12 14:35:48 cest" time_of_end [1] "2016-07-12 14:48:08 cest" 

is normal ? , if so, know how speed additionally, maybe using different packages? type of appreciated!

edit: these times after applying corrections suggested. 32 cores:

user      system     elapsed  5.99       40.78      243.97 

for 4 cores:

user  system  elapsed    1.91    0.94  991.71  

note before, did calculation multiple times via loops, that's why computation time decreased drastically, 1 can still tell difference between 2 computers has increased, believe.

try , let me know if problem solved:

library(doparallel) library(foreach) registerdoparallel(cores=detectcores()) n <- length(newly_populated_vec) cat("\nn = ", n, " | parallel workers count = ", getdoparworkers(), "\n\n", sep="")  t0 <- proc.time() xenopus_data <- foreach(b=1:n,.packages = c("raster", "gdistance", "rgdal","sp"), .combine=rbind) %dopar% {          xenopus_walk(         water=water,         altdata=altdata,         habitat_suitability=habitat_suitability,         max_range_without_water=max_range_without_water,         max_range=max_range,         slope=slope,         start_pt=newly_populated_vec[b])  } time <- proc.time() - t0 

also, try monitor logical cores in pc/laptop check if cores involved in computation. (taskmanager windows , htop linux)

please mindful doubling number of cores not lead having double performance.


Comments

Popular posts from this blog

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project.Error occurred in starting fork -

windows - Debug iNetMgr.exe unhandle exception System.Management.Automation.CmdletInvocationException -

configurationsection - activeMq-5.13.3 setup configurations for wildfly 10.0.0 -