Check out [[Daily Reading]] for context. ## Today's Pick of the Internet #### [Stuff we figured out about AI in 2023](https://simonwillison.net/2023/Dec/31/ai-in-2023/) In this article on his blog, Simon Willison summarises AI news from 2023 (last year). I really enjoy Simon's style of writing and therefore, slightly tweaking my format of this summary. I will write down the section header and quote Simon in each as a fitting summary. ##### Large Language Models "In the past 24-36 months, our species has discovered that you can take a GIANT corpus of text, run it through a pile of GPUs, and use it to create a fascinating new kind of software." ##### They're actually quite easy to build "So training an LLM still isn’t something a hobbyist can afford, but it’s no longer the sole domain of the super-rich. I like to compare the difficulty of training an LLM to that of building a suspension bridge—not trivial, but hundreds of countries around the world have figured out how to do it." ##### You can run LLM's on your own devices "Today there are literally thousands of LLMs that can be run locally, on all manner of different devices." ##### Hobbyists can build their own fine-tuned models "The best overall openly licensed LLM at any time is rarely a foundation model: instead, it’s whichever fine-tuned community model has most recently discovered the best combination of fine-tuning data. This is a huge advantage for open over closed models: the closed, hosted models don’t have thousands of researchers and hobbyists around the world collaborating and competing to improve them." #### We don't yet know how to build GPT-4 "Still, I’m surprised that no-one has beaten the now almost year old GPT-4 by now. OpenAI clearly have some substantial tricks that they haven’t shared yet." ##### Vibes Based Development "I find I have to work with an LLM for a few weeks in order to get a good intuition for it’s strengths and weaknesses. This greatly limits how many I can evaluate myself!" ##### LLMs are really smart, and also really, really dumb "Offer it cash tips for better answers. Tell it your career depends on it. Give it positive reinforcement. It’s all so dumb, but it works!" #### Gullibility is the biggest unsolved problem "Language Models are gullible. They “believe” what we tell them—what’s in their training data, then what’s in the fine-tuning data, then what’s in the prompt." ##### Code may be the best application "On the other hand, as software engineers we are better placed to take advantage of this than anyone else. We’ve all been given weird coding interns—we can use our deep knowledge to prompt them to solve coding problems more effectively than anyone else can." ##### The ethics of this space remain diabolically complex "Law is not ethics. Is it OK to train models on people’s content without their permission, when those models will then be used in ways that compete with those people?" --- #### Like what you see? Would you like to support me? Easy! Head over to [this link](https://refind.com/?invite=7b7e76f6e0) and subscribe to receive **Refind** newsletters. Every day Refind picks the most relevant links from around the web for you. Loved by 400k curious minds.