Background

When developing a new application, especially an Artificial Intelligence (AI) based application, one of the questions that comes into mind: should the application be executed on the CPU (Central Processing Unit) or the GPU (Graphical Processing Unit).

CPU is the main processing unit in every computer while GPU is a separate processing unit that its main purpose is to efficiently handle graphical work. Apparently the GPU can also efficiently run basic operations in AI software like matrix multiplications. So should an AI based application rely on GPU?

The main advantage of using GPU for AI based application is the good chance of reducing overall processing time and taking the load of the main processing unit. But there are some disadvantages that needs to be taken into account.

Power

Compared to the CPU, GPU consumes significant amount of power and executing a software on the GPU might end up with a significant increase of the overall power consumption. This is not good for your environment, electricity bills but in addition it might be very disturbing especially when you are running on batteries.

Reduced display quality

The main purpose of the GPU is to enable smooth graphics processing on the PC. If a third party AI software uses the GPU resources it might overload it and as a result it might have a bad impact the basic screen functionality of the PC. For example, you might experience phenomena like a lag.

Availability

GPU is not always available and therefore any AI software cannot rely on GPU only. If GPU is not available or is not powerful enough, the AI application must be efficient enough to be executed on the main CPU without overloading it.