Analyst To AI

As with many things, there is a cost to using AI. We have all seen the push back from local communities to the building of data centers. Residents near these facilities protest to protect the land that these massive complexes will require as well as the strain that it will put on the local infrastructure.

While there is a societal cost, there’s another cost that I wasn’t aware of until recently. I’ve started looking into platforms to experiment with building AI agents. Most of the pricing was presented in buying credits for a specific monthly or annual fee. I wondered what depletes the credits and learned that the credits represented a certain quantity of tokens. When I dove deeper to see what a token was, this is what I learned.

A token is a unit of data that AI algorithms use to process data more efficiently. The number of tokens AI uses for a unit of data varies but I’ve read that a word averages roughly just less than one token. Every time you do a search, it costs a certain amount of tokens, so a long search with a lot of back and forth questions and clarifications can cost a LOT of tokens.

This is where the economics of the data centers come into play. Economics teaches us about economies of scale. These huge data centers will be able to process an incredible amount of data and expand the capacity beyond what is currently available. All of this available capacity will lower the per unit currency cost for each token.

I’m not going to share my opinion about this issue because this blog is intended to inform and learn what AI is all about. I do find it interesting though, to be able to understand the real costs and issues at play as AI is quickly becoming part of our everyday life.