AI accelerator is a specialized hardware processing unit that provides high throughput, lower latency, and higher energy efficiency compared to existing server-based processors available in the market. Some AI accelerators are NPU, GPU, FPGA, and ASIC. As compared to other accelerators, ASICs are much more efficient technology as they consume very low power and can be readily customized for specific activities. The AI accelerators can be used in cloud servers as well as at the edge devices. Nowadays, the cloud provides an ideal environment for Machine Learning as it gathers a massive amount of data from various sources. At the same time, edge computing or in-device computing is the ideal option for inference that requires quick output. AI accelerator architecture is necessary for advanced data centers to address the ever-increasing demands of processing and handling massive datasets workloads such as machine vision, deep learning, AI, etc. Moreover, it is necessary to consider the servers’ power consumed and the data center’s power budget while designing the AI accelerators. This chapter discusses various AI accelerators in the cloud, data centers, servers, and edge computing. © The Editor(s) (if applicable) and The Author(s),