Efficient Use of AI
Use the model that is appropriate to your task
...don’t use Einstein to assemble a grocery list.
Online vs Local AI
First, there is a trade-off between running locally and online.
Running locally gives you less latency and more control. It also guarantees access, as long as you have battery backup. There is a cost to acquire, power, and maintain local hardware.
The resulting cost is not a function of the amount of usage (with the proviso that more usage may mean consuming more utility-provided electricity, and larger, more expensive hardware). You do
have to maintain the local system(s) and follow the emergence of new models/hardware.
Running online has the risk of no access, greater latency, less control, and expense based on usage. However, it may give you access to larger, more powerful models. Most existing computers
and phones can access it. There is no maintenance expense. As newer, more powerful models appear, they may be made easily available.
Efficient Online AI Usage
If you choose to use an online AI service, you are likely to have a choice of a number of different AI’s at a number of different costs. For example, Anthropic today has five different levels of AI’s that you can use. In Anthropic’s (Frontier model company) documentation, the company recommends that you use the model that is appropriate to your task. Essentially, they are saying, don’t use Einstein to assemble a grocery list.Each one of the five different models that Anthropic offers reasons at different levels. Using the highest-level model consumes the most tokens to execute the same prompt language.
If the intent is to have an AI simply assist with a Web search, then one of the simpler models may be the most efficient, similarly with a simple IA.
On the other hand, trying to execute an extremely complex task or request may need the highest-power model that uses way more tokens. There are also models that have been customized for specific tasks such as software coding. Using a coding model to do a grocery list is also likely to be inefficient.
Less expensive online AI services (not Frontier model companies) are beginning to appear. They tend to fall into one of two approaches: offer open source/open weights models (OS/OW Ms) running in a large data center; or offer older versions of Frontier model companies. OS/OW Ms coming from China are beginning to get close to the performance of Frontier model companies. Some are concerned about using models developed in China, fearing theft of information. There are others who maintain that these models are perfectly safe.
Across all of the alternatives, the trick is to get staff to use the right model for the right task. There are three ways to do this.
First, access to models may be limited depending on function. That is, only give finance staff access to finance-customized models. Only give software developers access to models that have been customized for coding. Etc. Only give staff access to the lower-cost models. Or combinations of these.
Second, staff can be trained to select the right model for the right task.
Finally, an AI may be used to suggest, or actually route, to the most efficient model.
Efficient Local AI Usage
There are three ways that models can be run locally: In an organization data center or a cloud contracted provider; In smaller servers distributed around the organization; and finally on Edge devices such as notebook computers, phones, and small desktop devices like the Mac Mini and Asus version of the DGX Spark.
