Uneasy lies the head that wears a Crown*

Bard - A quick review vs GPT

Useful Links for Bard vs Chat GPT
Bard
Chat GPT
Google AI Principles

Sample Prompts submitted to Bard

• "How many words in this sentence"

Bard failed on this prompt, like GPT. Adding extra text to mention "this same sentence" helped. Score this one as a draw between the two LLMs.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• "Write python code to find gaps in a set of time series data which has been loaded into a pandas data frame"

Bard generated several versions of the python code. But the code didn't compile when run. Score this one as a FAIL by Bard as the code produced by GPT worked first time and had no issues.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• "Create code for a web site which provides reviews of Roblox games. The code should use HTML, Javascript, CSS and it store reviews in a database like sqlite. The web site will allow users to add comments on reviews. Please do not suggest using PHP language or any web coding tools such as React."

Bard produced some static html only. GPT was able to generate code that included services and a database for storing comments. Possibly some more edits required on the prompt to allow Bard to show its real capabilities but given the above it did not produce results as good as GPT.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• "Please complete this phrase : "Bacco, tabacco e Venere riducono l’uomo in"

Bard does not support multiple languages as yet. Score a win for GPT on this feature.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• "I want to build a system for trading crypto coins based on API published by the Binance exchange. Please share the following UML diagrams based on that API - sequence diagram, state diagram, class diagram, context diagram. Also include a swagger document with API from Binance plus set of tests. The language for coding will be python so include python class model. please share the UML diagrams in mermaid standard so I can view the UML diagrams at mermaid web site."

Bard was not able to produce a design that included any UML diagrams in working mermaid format. It generated a hiugh level system design but was not able to create working design documents for detailed parts of the system. A swagger document was generate don demand but it listed just endpoint names with no info on parameters, json objects or anything useful for developers. It seems to read only the available functions in the API content and not understand what lies under these API calls. GPT wins on this test.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• "Please describe the book by oswald spengler called "Decline of the west" and share some reviews with links and academic references"

Next I asked each LLM(AI) to rate the content produced by their counterpart. Bard viewed GPT content as "excellent" while GPT thought Bards content was "adequate".

Bard: "I would rate your summary of Oswald Spengler's book as excellent. It is comprehensive and well-organized, and it accurately reflects the main points of the book. I particularly appreciate the way you have provided links to academic references, which will be helpful for readers who want to learn more about Spengler's work."

GPT: "The summary you provided gives an adequate overview of Oswald Spengler's "The Decline of the West." It briefly explains the main thesis of the book, highlights its controversial nature, and provides examples of reviews and academic references. However, there is room for improvement."

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Other Features for comparison

• AI Principles

One feature which Google mentions is their set of AI principles which can be viewed [here][1], OpenAI has its equivalent set of guidelines/goals available for viewing [here][2]. We could score this item as a draw since both have a set of priciples but we are not able to test fully at this time. Perhaps set aside a separate test of gaardrails and goals for the two LLM's later on, possibly adding any other LLMs like Jurassic if they can be tested.

• Cost

  GPT costs $20 per month while Bard is free (for now)

Summary

Bard had some good points, such as ability to import text from web sites by sharing the URLs. And its better at prividing references to underlying contnt. But GPT performs much better at a range of tasks and appears to be more advanced that Bard by a couple of orders of magnitude. For example, its quite easy to send prompts to Bard and see a response like this "Something went wrong. Bard is still experimental". [1]: https://www.blog.google/technology/ai/ai-principles/

Conclusion as of today is that GPT is more advanced and functional as an AI Chatbot based on LLM. This test can be rerun when either of the companies releases a new LLM version.

* A quote fromWilliam Shakespeare who was and is known as the “Bard”. The quote comes from his play “King Henry IV Part 2” and is relevant to the position of Google as it sees the first possible challenge to its legacy function of internet search by LLM such as GPT.