A Hybrid Parallel Techniques for Large-Scale Text Classification with Symmetric ADMM and 1-Factorization

Abstract

Distributed computing has become essential for handling large-scale datasets in machine learning. This study focuses on the implementation of two prominent multiclass Support Vector Machine (SVM) algorithms—Crammer and Singer (CS) and Weston and Watkins (WW)—in a distributed computing environment using symmetric Alternating Direction Method of Multipliers (ADMM). Designed for multiclass classification, these algorithms extend traditional binary SVMs by optimizing a single objective function that captures all class relationships simultaneously. These algorithms are adapted to leverage the computational power of multi-node clusters, ensuring scalability and efficiency. Symmetric ADMM is employed to decompose the optimization problem across multiple nodes, enabling parallel processing and efficient convergence for large datasets. The implementation distributes the optimization problem across a multi-node cluster, enabling parallel computation for handling large-scale dataset LSHTC. Each node processes a subset of the data, and symmetric ADMM ensures coordination and convergence by exchanging updates between nodes in a balanced manner. The optimization leverages polynomial kernels to handle non-linear separability effectively. The distributed framework reduces computational overhead and memory constraints compared to traditional single-node methods, achieving efficient scaling with the number of nodes. Results from the implementation highlight improved training times and robust classification accuracy, demonstrating the effectiveness of symmetric ADMM in balancing workload and accelerating convergence. This implementation serves as a scalable solution for high-dimensional datasets in resource-intensive environments.