First: Really love the podcast. Keep doing what you’re doing! .
Question about ChatGPT Impact on the world:
I think you were first to point out that Chat CPT doesn’t necessarily tell you truth, it tells you what masses of people on the internet *think* is the truth. Sometimes this is correct, and the information can be really useful. Other times those answers can be off in subtle or even not even so subtle ways. This is clearly a problem if Chat CPT style does become the main way that people get answers to their questions.
Do you know if this is something that is measured or can even be measured? Like can we measure how accurate ChatGPT is in a scalable / systematic way?
Is this is a known problem in the AI/ML world? What is to be done to stop this from happening?
This is a very good question. From what I've seen most AI companies are both trained and tested on curated datasets of human responses. This could be data that humans are paid to make, or something generated in mass, like social media posts with corresponding number of upvotes.
With regards to accuracy, we can't even measure how accurate humans are in a scalable / systematic way. That being said, there were a few attempts to do so:
- Academia: did well for awhile when it better selected for general competence, now falls prey to very simply exploits and social climbing (with the exception of a few fields which continue to select for general competence)
- Markets: this is not quite a judge of truth but a judge of what people are willing to pay for, which can discern some truths. Where applicable, commercial success is likely going to be a good metric for AI.
With regards to the broader problem of people getting their answers from an opaque, difficult to observe system, this is unfortunately how the majority of people have gotten information in the past and how they continue to do so today. That being said, the incentives, tendencies, and methodology of different ML models do have implications for what kinds of measurement errors we can expect. I'll be happy to expand on this in the long form version.
I appreciated your comparison of the way ai image generation works by adding noise and the way the programmers worked to have it generate language. I feel like i have done this with photoshop occasionally, where I have a flat color and I need to bring it to some kind of recognizable texture, but simply running filters on it won’t produce a believable result. Either they just shift colors around or they plaster some prefab picture/pattern onto my color field. So I have to add noise and then push things around a bit and then the texturizing filters have something to grab onto.
The process isn’t drawing exactly, it has the feel of “this is what the machine is good at” so they made filters to do it. I just have to use it to approach making the image I want from a sort of angle.
When Richard Hanania talked to his philosophers about ai, they mentioned image generated ocean waves that are not built the way physical waves are. They used a word like “agonic” and also “non-agonic”. What do those mean? Will language generation have the same machine-centered feel when we are working with it?
First: Really love the podcast. Keep doing what you’re doing! .
Question about ChatGPT Impact on the world:
I think you were first to point out that Chat CPT doesn’t necessarily tell you truth, it tells you what masses of people on the internet *think* is the truth. Sometimes this is correct, and the information can be really useful. Other times those answers can be off in subtle or even not even so subtle ways. This is clearly a problem if Chat CPT style does become the main way that people get answers to their questions.
Do you know if this is something that is measured or can even be measured? Like can we measure how accurate ChatGPT is in a scalable / systematic way?
Is this is a known problem in the AI/ML world? What is to be done to stop this from happening?
Thanks!
This is a very good question. From what I've seen most AI companies are both trained and tested on curated datasets of human responses. This could be data that humans are paid to make, or something generated in mass, like social media posts with corresponding number of upvotes.
With regards to accuracy, we can't even measure how accurate humans are in a scalable / systematic way. That being said, there were a few attempts to do so:
- Academia: did well for awhile when it better selected for general competence, now falls prey to very simply exploits and social climbing (with the exception of a few fields which continue to select for general competence)
- Markets: this is not quite a judge of truth but a judge of what people are willing to pay for, which can discern some truths. Where applicable, commercial success is likely going to be a good metric for AI.
With regards to the broader problem of people getting their answers from an opaque, difficult to observe system, this is unfortunately how the majority of people have gotten information in the past and how they continue to do so today. That being said, the incentives, tendencies, and methodology of different ML models do have implications for what kinds of measurement errors we can expect. I'll be happy to expand on this in the long form version.
All good questions! Some quite difficult.
Thanks - Looks like the AI Pluralism Newsletter will be good start to get more into the overall subject matter as well. Good timing!
I appreciated your comparison of the way ai image generation works by adding noise and the way the programmers worked to have it generate language. I feel like i have done this with photoshop occasionally, where I have a flat color and I need to bring it to some kind of recognizable texture, but simply running filters on it won’t produce a believable result. Either they just shift colors around or they plaster some prefab picture/pattern onto my color field. So I have to add noise and then push things around a bit and then the texturizing filters have something to grab onto.
The process isn’t drawing exactly, it has the feel of “this is what the machine is good at” so they made filters to do it. I just have to use it to approach making the image I want from a sort of angle.
When Richard Hanania talked to his philosophers about ai, they mentioned image generated ocean waves that are not built the way physical waves are. They used a word like “agonic” and also “non-agonic”. What do those mean? Will language generation have the same machine-centered feel when we are working with it?