## Parallel and Distributed Sparse OptimizationPublished in Asilomar’13 Conference C codes (bugs fixed in November 2014)
## BackgroundModern datasets usually have a large number of features or training samples, and they are usually stored in a distributed manner. Motivated by the need of solving sparse optimization problems with large datasets, we propose two approaches including (i) distributed implementations of prox-linear algorithms and (ii) GRock, a parallel greedy coordinate descent method. ## Codes and demos## Three parallel C solvers for LASSOP-FISTA: parallel fast iterative soft-thresholding algorithm PD-ADMM: parallel dual alternating direction method of multipliers GRock: parallel greedy coordinate descent method (bugs fixed in November 2014)
## More codes and applications will be uploaded in the coming weeks## Solving LASSO with a 170GB matrix on Amazon EC2## Test dataset: dense matrix with 100K rows and 200K columns, 20 billion entries in total, 170GB in size : 200K entries, 4K nonzeros (2 percent), Gaussian values
## Requested systems on Amazon EC220 “high-memory quadruple extra-large instances” each instance has 8 cores and 60GB memory
## LASSOWe tried to recover by solving in which and were set so that is the exact solution ## Tested algorithms*p D-ADMM*:*parallelized*dual ADMM, solving the dual formulation of LASSO*p FISTA*:*parallelized*FISTA, solving the dual formulation of LASSO*GRock*: parallized greedy coordinate-block descent
ADMM's performance depends on a certain penalty parameter . We picked as the best out of only a few trials (we cannot afford more). It is likely that the performance of PD-ADMM can be further improved
step size for FISTA is evaluated based on the spectral radius of
stopping creterions include: maximum number of iterations: 2500 a relative error of 1E-5
## Sparse logistic regression on a cluster## Test datasetThe Adult Data Set from the UCI Machine Learning Repository
## SystemRice cluster STIC We tested our codes on 1, 8, 16, 64, and 128 cores
## Sparse logistic regression modelWe tried to recover a hyperplane for classification with a for given feature data points and labels , . ## Tested algorithms*p FISTA*: our*parallel*implementation of FISTA. FISTA is a prox-linear algorithm with Nesterov's acceleration by Beck and Toubelle.*GRock*: parallized greedy coordinate-block descent
## Citation
« Back |