Sonnet: Company and product name extraction
| Text | Companies | Products |
| 10 steps to smarter Google account security Give yourself some added peace of mind by giving your Google account a thorough set of security reinforcements, both on Android and your desktop. | Google account | |
| How ChatGPT works with iOS 18.2 iPhones and Siri Apple Intelligence is about to receive its first important update — introducing ChatGPT access. | Apple | iOS 18.2, iPhones, Siri, ChatGPT |
| OECD: GenAI is affecting jobs previously thought safe from automation Though the technology will likely lead to new jobs, they may not benefit those who lost work due to automation. | OECD | GenAI |
| Microsoft moves to stop M365 Copilot from ‘oversharing’ data The generative AI assistant can surface sensitive information in over-permissioned files, a growing concern for businesses testing the technology. Microsoft is adding new features to SharePoint and Purview to make it easier to control what the tool can access. | Microsoft, SharePoint, Purview | M365 Copilot, SharePoint, Purview |
When I tried the pricey Opus model, it got rows 2-4 correct but added Android as a company on the first row. My conclusion: I either need to give these models better prompts and more examples for a task like this or tolerate some inaccuracies.
However, models continue to improve, and a task that’s beyond their capabilities now may work better a few months down the road.
For the sake of comparison, I gave all this text to OpenAI’s new o1 model in the ChatGPT Plus chatbot and asked it to extract products. Those results were better: o1 correctly identified Google account, Android, ChatGPT, iOS 18.2, iPhones, Siri, M365 Copilot, SharePoint, and Purview as products.
And it was the only model to extract just Google, Apple, and Microsoft as “companies” — OECD, the Organisation for Economic Co-operation and Development, is not technically a company. When I asked o1 to extract both companies and organizations, it responded: Google (company), Apple (company), OECD (organization), and Microsoft (company).
(OpenAI doesn’t have its own extension for Google Sheets, although there are some paid third-party applications.)
Meanwhile, though, if you can tolerate some imperfect results and less than top-speed performance, you can start doing LLM-based natural language processing right within a spreadsheet. As Ethan Mollick, author of Co-Intelligence: Living and Working with AI, advised on Bluesky recently:
“I think firms worrying about AI hallucination should consider some questions:
1) How vital is 100% accuracy on a task?
2) How accurate is AI?
3) How accurate is the human who would do it?
4) How do you know 2 & 3?
5) How do you deal with the fact that humans are not 100%?
Not all tasks are the same.”