this post was submitted on 04 Oct 2023
22 points (86.7% liked)

Asklemmy

43877 readers
1399 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy ๐Ÿ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_[email protected]~

founded 5 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
22
Does Bing Chat give reliable answers to math and physics questions? If not is it possible to make it more reliable? (lemmy.ml)
submitted 1 year ago by [email protected] to c/[email protected]
16 comments fedilink hide all child comments
 

I realize and understand the criticisms of ChatGPT and I have personally seem how bad it can be. Once I asked to count the number of days till a random date giving the present date and it failed miserably, again and again. Trust me! I get the criticism. But, what about Bing Chat Bot?

Have you ever tried to ask you Physics and Maths related questions to it? I was coding a while ago and I had a pretty complex questions which could not be solved by a very popular reddit coding community but Bing Chatbot gave an answer to it in an instant! I was genuinely impressed. Apparently it checks for answers on multiple webpages on the internet, it reads and understands what it reads and it gives the answer to it after combining the knowledge it gained from it's search. Again, the question I asked was pretty complex but it was able to answer it in an instant and it was the right answer! It was coding, it's pretty hard to get the right answer in the first try, I have found it's more "trial and error".

So yeah!

  1. Can I rely partially on Bing Chatbot for math questions?
  2. If not can I ask it to form a query which encapsulates my question perfectly?
  3. If not, should I ask it to "Answer this question and site your sources"?
  4. Can I do something more? i.e., like I did in 3? What are your thoughts on this?

I won't be able to reply to each of your comments anytime soon, but know that I deeply appreciate this community and it's members and their help :')

you are viewing a single comment's thread
view the rest of the comments
[โ€“] [email protected] 3 points 1 year ago* (last edited 1 year ago)

I have experience with GPT-4, and in particular I've used to for math questions in my work occasionally. I'm not sure how Bing chat compares.

For GTP-4, I've noticed the following:

  1. How reliable the answer is depends on how easy or obscure the question is. It hasn't lied to me on easy or introductory material, but once your questions start becoming more obscure, and it's less likely to have the answer in the training set, it starts making things up.
  • I think of it as search to an extent - it needs to have the answer in the training data to find it. Unlike google, it can usually find an answer even if you don't use the proper terms. But if it doesn't find an answer, it might make something up.
  • "Easy or introductory" is relative - I have been able to get good answers for some masters-level math, and some wrong ones for lower-level things. Ultimately it depends on how much resources on the topic have been in the training set.
  1. It's actually much more reliable in detecting errors than it's in generating text. So you can open a new chat and ask, "Is the following true: ..." and it will catch most of its own errors. Once it starts catching error, you should know you've left the reliable "easy questions" territory, and even if it can still be useful, exercise much more care.
  2. The way you phrase a prompt matters a lot. For example, if you ask it to explain its reasoning step by step, it becomes much more accurate.
  3. It is generally good in rephrasing questions to use better terminology.

.

Bing chat might be different in some regards. I know that it automatically searches the web for sources, and when generating an answer, and bases its answer on the contents of the sources it found - but I don't have experience with it.

That said, asking for additional sources (besides the search results it found) shouldn't improve the accuracy. It might just give you something you can use to fact-check it.