False Positives Abound
Spent three weeks on the first chapter of a manuscript. Zero AI input other than checking for tense (it's in present tense, and with flashbacks, this can cause brain hemorrhaging).
Sent it off to an editor who responded, "If you're going to use AI to write your work, why don't you simply use AI to edit it for you."
Asked which program he had used to determine that my work, he replied, "Sapling. It's what I always use."
I then plugged my work into six other AI Detector models, and the highest amount of AI I received back from them was 14%. Four of the six were 0%. The fifth was 2%.
There will come a day when one of these companies gets sued into oblivion, and I can't wait for it to happen.
I think
Antwoord van Sapling Intelligence
Hi there --
Sorry this happened. We're very explicit that the output should just be used as one cue, and that human judgement should be used: https://sapling.ai/ai-content-detector#:~:text=No%20current%20AI%20content%20detector%20(including%20Sapling%27s)%20should%20be%20used%20as%20a%20standalone%20check%20to%20determine%20whether%20text%20is%20AI%2Dgenerated%20or%20written%20by%20a%20human.%20False%20positives%20and%20false%20negatives%20will%20occur.
However, we do benchmark on a real validation set for all accuracy numbers we report, something not all detectors (or for that matter, "humanizers") do. We do also offer fine-grained information on what portions of the text are indicative of AI, which should be used during final human evaluation.
