You are basically giving all the reasons for why this hasn’t been done yet without AI, but none of the reasons for why this can’t be done with AI.
I know we like to be cynical about the advances of things like chat gpt, but I have found many uses that are very similar to what you describe below. Taking a problem that could be solved with tedious brute force and combining data from multiple sources and knowledge of a scripting language, but instead I ask chat gpt in just the right way and it will get me the answer.
Also worth noting, grocery store prices are easily accessible online now whereas 10 years ago they were not. It’s just a matter of time before AI gets access to this data and can integrate into whatever models it uses.
Well, no, I’m just saying the text generation stuff did not change anything about that process.
It can try to generate the right text for the requests to grab this data, but since there’s going to be practically no documentation for that out there, it will struggle to do so from just its training data alone.
So, what you do instead is that you have a human figure out the API of each store that needs to be integrated + ideally a transformation of the returned data into a shared, documented format. And then you tell the text generation a trivial way for it to generate the text to make use of that.
So, basically you preface the whole user conversation with “If I ask for prices of Todd’s Tater Tots, run ./prices_todds_tater_tots.sh for that and use the result according to the JSON schema in prices_store.schema.json.”.
And then you repeat that for all the other stores, for some math API and some navigation API and then you’ve got a chance that the text generation figures out the right semantics of how these things should be called.
Semantics is what it’s good at. But the rest is still the same process as ten years ago.
I think you’re under selling what chat gpt is capable of. It is able to take outside data in and use it with the rest of the model. Bing does it with its web index data. I was able to ask what the cheapest gas station near me is and bing gave me a list, likely coming from gas buddy.
Yeah, it can easily do that, if such a comparison service already exists. Then it’s just yet another API that it calls, or in this case, it more likely just does a Bing search and recounts the top results. But I haven’t yet heard of such a service existing for groceries, so it would still need to be built.
I understand what you’re saying. In theory, it’s possible. But in practice, it is not just a matter of linearly improving LLMs and then at some point, they’ll just do it on their own. Any task that takes more than a few steps means that the error rate of the LLM multiplies.
The error rate would need to get magnitudes lower for that multiplication to not explode with many steps. The alternative is removing steps that the LLM needs to do, as I described. Some colleagues at my dayjob do basically nothing else now.
You are basically giving all the reasons for why this hasn’t been done yet without AI, but none of the reasons for why this can’t be done with AI.
I know we like to be cynical about the advances of things like chat gpt, but I have found many uses that are very similar to what you describe below. Taking a problem that could be solved with tedious brute force and combining data from multiple sources and knowledge of a scripting language, but instead I ask chat gpt in just the right way and it will get me the answer.
Also worth noting, grocery store prices are easily accessible online now whereas 10 years ago they were not. It’s just a matter of time before AI gets access to this data and can integrate into whatever models it uses.
Well, no, I’m just saying the text generation stuff did not change anything about that process.
It can try to generate the right text for the requests to grab this data, but since there’s going to be practically no documentation for that out there, it will struggle to do so from just its training data alone.
So, what you do instead is that you have a human figure out the API of each store that needs to be integrated + ideally a transformation of the returned data into a shared, documented format. And then you tell the text generation a trivial way for it to generate the text to make use of that.
So, basically you preface the whole user conversation with “If I ask for prices of Todd’s Tater Tots, run
./prices_todds_tater_tots.shfor that and use the result according to the JSON schema inprices_store.schema.json.”.And then you repeat that for all the other stores, for some math API and some navigation API and then you’ve got a chance that the text generation figures out the right semantics of how these things should be called.
Semantics is what it’s good at. But the rest is still the same process as ten years ago.
I think you’re under selling what chat gpt is capable of. It is able to take outside data in and use it with the rest of the model. Bing does it with its web index data. I was able to ask what the cheapest gas station near me is and bing gave me a list, likely coming from gas buddy.
Yeah, it can easily do that, if such a comparison service already exists. Then it’s just yet another API that it calls, or in this case, it more likely just does a Bing search and recounts the top results. But I haven’t yet heard of such a service existing for groceries, so it would still need to be built.
I understand what you’re saying. In theory, it’s possible. But in practice, it is not just a matter of linearly improving LLMs and then at some point, they’ll just do it on their own. Any task that takes more than a few steps means that the error rate of the LLM multiplies.
The error rate would need to get magnitudes lower for that multiplication to not explode with many steps. The alternative is removing steps that the LLM needs to do, as I described. Some colleagues at my dayjob do basically nothing else now.