Tech publisher CNET quietly started an experiment with articles written by ChatGPT and, despite many industry experts lauding the software’s remarkable power, they were littered with fairly basic errors.
In heartening news for anyone, anywhere who has to write anything with even a modicum of complexity, the initial results of CNET‘s remarkably wide-reaching experiment have been far from positive.
According to a post by the site’s editor in chief, Connie Guglielmo, CNET published 78 articles that were written by the ChatGPT robot since November, at some points, it was churning out 12 posts per day. However, the publisher has been forced to issue large and embarrassing corrections to many.
The idea was simple, the site’s Money editorial team would use the tech on basic explainers around financial services topics in order to free up time for editorial staff on less trivial matters. Articles titled “How to Cash a Check Without a Bank Account,” “Alternatives to Traditional Savings Accounts,” and “How to Get Your Overdraft Fees Refunded” were published under the CNET Money Staff byline with no mention of AI involvement on the articles themselves. Readers would be able to find out that an AI had written the articles if they checked the byline’s bio page.
However, a bunch of other publishers and journalists caught on and called out the site for not disclosing that the articles were written by a machine. Then, the corrections came.
I think about shit like this a lot as somebody laid off from a copyediting job because some people think AI tools can do the work for you.
I can’t believe I’m saying this but I hope Google steps in https://t.co/NUG8vThTCm
— Carli Velocci (@velocciraptor) January 11, 2023
One article, discussing compound interest contained a basic but important error.
“To calculate compound interest, use the following formula:
Initial balance (1+ interest rate / number of compounding periods) ^ number of compoundings per period x number of periods
For example, if you deposit $10,000 into a savings account that earns 3 per cent interest compounding annually, you’ll earn $10,300 at the end of the first year.”
Not only is it wordy but it is wrong, as well. It implies that, if you stuck ten grand into a savings account with a 3 per cent interest rate, you would earn $10,300 per year — effectively doubling your money. In reality, you would have $300 more than you did at the start of the year.
In response, Guglielmo wrote that CNET had built its “reputation testing new technologies and separating the hype from reality” for more than “two decades.”
She also said that while the AI engine would “compiled the story draft or gathered some of the information in the story,” every article would be reviewed, fact-checked, and edited by an editor with topical expertise before publish. If errors were found post-publish, the site would issue a correction.
Here’s the correction for the story on compound interest in full:
“Correction, 1:55 p.m. PT Jan. 16: An earlier version of this article suggested a saver would earn $10,300 after a year by depositing $10,000 into a savings account that earns 3% interest compounding annually. The article has been corrected to clarify that the saver would earn $300 on top of their $10,000 principal amount. A similar correction was made to the subsequent example, where the article was corrected to clarify that the saver would earn $304.53 on top of their $10,000 principal amount. The earlier version also incorrectly stated that one-year CDs only compound annually. The earlier version also incorrectly stated how much a consumer would pay monthly on a car loan with an interest rate of 4% over five years. The earlier version also incorrectly stated that a savings account with a slightly lower APR, but compounds more frequently, may be a better choice than an account with a slightly higher APY that compounds less frequently. In that example, APY has been corrected to APR.’
That’s some statement.
The article now also states that it was edited by Liliana Hall, who covers , and was “assisted by an AI engine and reviewed, fact-checked and edited by our editorial staff.”
Other publications, such as the Associated Press, use AI to fill in the blanks on pre-formatted stories. The Financial Times does something similar with quick turnaround financial results reporting — though the analysis of those results is done by a human.
So, if you do anything involving writing, it seems the robots won’t be coming for your job just yet, at least.
