Community detection is an important challenge in network analysis. We develop a novel influence diffusion model to exploit the implicit knowledge of influence-based connectivity and proximity encoded in the network topology, and approach community detection by clustering together nodes with similar influence profiles. Using this model, we proposed an effective influence-guided spherical K-means (IGSK) algorithm for community detection in binary networks. In this paper, we first present an influence-based weighting scheme and extend our influence diffusion model to weighted networks. We then propose a novel influence-guided label propagation (IGLP) algorithm, in which each node is assigned the same community label as its most similar neighbor. Our algorithm does not require any parameters or convergent iterations, and produces a deterministic initial configuration of community partitioning. Further, we perform agglomerative hierarchical clustering to uncover the hierarchical community structure of the network using a new cluster proximity measure. Extensive tests of our method on a set of real-world networks and synthetic benchmarks show excellent performance in terms of both accuracy and efficiency in both undirected/directed and unweighted/weighted networks. More interestingly, our method achieves nearly linear time complexity and manifests promising scalability for large-scale networks.