185

All that water, all those resources, everything will be sacrificed to keep the line going up. (hexbear.net)

submitted 2 months ago by Super_Lumalo@hexbear.net to c/chapotraphouse@hexbear.net

69 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[-] AssortedBiscuits@hexbear.net 9 points 2 months ago* (last edited 2 months ago)

I wanna see the results if you ask ChatGPT the same question a million times. What percentage of responses would actually get the correct number?

[-] purpleworm@hexbear.net 9 points 2 months ago

I think that heavily depends on whether it gets the initial answer right, since it will use that as context

[-] Cysioland@lemmygrad.ml 4 points 2 months ago

When you're calling it through an API then you can simply choose not to pass it any context

[-] purpleworm@hexbear.net 2 points 2 months ago

Fair enough

[-] gay_king_prince_charles@hexbear.net 2 points 2 months ago

It depends on the temperature. There's a variable you can play with that adds since randomness to the responses (LLMs are fully deterministic when temperature is 0). Sometimes the F1 or F2 score is used to determine correctness of many questions, but I don't have a great understanding of how that metric works and what ChatGPTs is.