Creating (and refining) prompts used by LLMs has been a side-hobby for the past year.  Below are three non-technical prompts that illustrate using an LLM to answer casual questions.

For instance, my young daughter currently plays hockey. Last March one of her coaches mentioned that there were outsized scholarship opportunities from NCAA Division III colleges for women’s hockey. To start fact-checking that statement, I began with a prompt:

There are 67 teams in NCAA Division III women’s hockey. Please list them in order based on the school rankings from U.S. News & World Report.

At the time, ChatGPT 4 said it did not have access to UNSWR or Forbes and couldn’t complete the ranking.  

Fast forward to today, while ChatGPT 4o still does not have direct access to these two ranking indices, it did pivot by asking if it could provide a different sorting (although not by academic ranking).

During the summer, when Ask Venice launched, I tried asking what the etymology of a “no-coiner” was.  While it didn’t provide the history of the phrase, it did provide some footnotes which at the time few LLMs did.  Fast forward to today, the same question still does not provide a satisfactory answer (e.g., it lacks history: it doesn’t say who the first people were to use the term).

This past fall I queried:

What was the participation in the PSAT / SAT over the past 10 years. And, assuming birth rates / immigration trends continue, what the participation rate would be the next 15.  

Following those I asked what the previous participation in the AMC 8, AMC 10, and AMC 12 were the past 10 years. And, likewise, what the trend lines look for the next 15 years.

ChatGPT 4 drew some decent line charts to visualize those trends.  The motivation for this were various parents discussing the rat race of cram schools in the Boston area during a time in which some matriculation numbers for the city are declining.

A number of other casual searches involved finding local activities for my daughter and even writing or reading contests.  The answers typically are often better than what Google provides.

The refinement process for prompts is probably something that will not go away even with more accurate reasoning models (such as DeepSeek R1) are deployed.  Does that mean people being paid $200k a year for their help chatting with bots in 2023 translates into a long-term moat for creative prompt writing?  No, but there does seem to be an art for conjuring and revising them at this time.  

Counting GPUs

We have discussed the flexibilities of GPUs over the years… I even wrote a “How-to” guide for mining Dogecoin over a decade ago.

I saw a germane tweet this past week that piqued my interest in part because it was a little inaccurate:

Source: David Holz

Someone is (a little) wrong on the internet!

My quibble, and one that was echoed by a couple of others in that thread, is that there are a lot more than 5 million GPUs manufactured each year.  For instance, in Q1 2024 around 70 million discrete and integrated GPUs were shipped by Nvidia, Intel, and AMD combined.  These were for both desktops and laptops.  In Q2 2024, approximately 9.5 million discrete GPUs for desktop PCs (not laptops) were shipped by the same three manufacturers.

And one other GPU segment that David appears to have missed in his calculation are video game consoles.  For example, more than 60 million Playstation 5’s have been sold since 2020.  Likewise around 31 million Xbox Series X and Series S have been sold in that same time frame.  While neither of these are particularly powerful relative to a new GPU from Nvidia today, when they were first released they were all considered high performance.

As LLMs become more accurate, with better reasoning abilities, a scenario that David and others rightly ask is: wouldn’t the demand for GPUs outstrip the supply?  That is to say, if an AGI (however defined) is capable of running on a discrete GPU, how many people with the means, will purchase and install one at home versus renting them off a cloud provider?  Maybe the TAM for on-premise AGI would be as large as on-premise e-mail users?

Let’s check in again on this in a couple of years to see what level of artificial intelligence can run off the forthcoming 5090 and its successor.  Hopefully they’ll be more accurate, because during the drafting of this post, ChatGPT 4o used outdated video game console numbers (without footnotes!) that I eventually tracked down with… Google.

And if you have access to the newly released DeepResearch, feel free to leave a comment about what it assesses are the merits and demerits of Jeffrey Emanuel’s essay on Nvidia as well as the energy and resource usage estimates from Rohit Krishnan.